UIS-Digger: Towards Comprehensive Research Agent Systems for Real-world Unindexed Information Seeking
Authors: Chang Liu
Chuqiao Kuang
Tianyi Zhuang
Yuxin Cheng
Huichi Zhou
Xiaoguang Li
Lifeng Shang
Venue: ICLR 2026 Poster (2026)
Keywords: Dataset
Agent
Information Seeking
Primary area: datasets and benchmarks
TL;DR: We identify the unindexed-information seeking problem which is omitted by previous deep research agents, and provide a new benchmark called UIS-QA, together with a proposed baseline agent system surpass all previous methods.

Abstract: Recent advancements in LLM-based information-seeking agents have achieved record-breaking performance on established benchmarks. However, these agents remain heavily reliant on search-engine-indexed knowledge, leaving a critical blind spot: Unindexed Information Seeking (UIS). This paper identifies and explores the UIS problem, where vital information is not captured by search engine crawlers, such as overlooked content, dynamic webpages, and embedded files. Despite its significance, UIS remains an underexplored challenge. To address this gap, we introduce UIS-QA, the first dedicated UIS benchmark, comprising 110 expert-annotated QA pairs. Notably, even state-of-the-art agents experience a drastic performance drop on UIS-QA (e.g., from 70.90 on GAIA and 46.70 on BrowseComp-zh to 24.55 on UIS-QA), underscoring the severity of the problem. To mitigate this, we propose UIS-Digger, a novel multi-agent framework that incorporates dual-mode browsing and enables simultaneous webpage searching and file parsing. With a relatively small $\sim$30B-parameter backbone LLM optimized using SFT and RFT training strategies, UIS-Digger sets a strong baseline at 26.36\%, outperforming systems integrating sophisticated LLMs such as O3 and GPT-4.1. This demonstrates the importance of proactive interaction with unindexed sources for effective and comprehensive information-seeking. Our work not only uncovers a fundamental limitation in current agent evaluation paradigms but also provides the first toolkit for advancing UIS research, defining a new and promising direction for robust information-seeking systems.

Published as a conference paper at ICLR 2026
UIS-DIGGER: TOWARDS COMPREHENSIVE RE- SEARCH AGENT SYSTEMS FOR REAL-WORLD UNIN-
DEXED INFORMATION SEEKING
Chang Liu'> 5 Chuqiao Kuang', Tianyi Zhuang Yuxin Cheng", Huichi Zhou', Xiaoguang Li', Lifeng Shang'
HHaawei Technologies Ltd. "The University of Hong Kong University College London (LiuChang730, kuangchuqias zhuangtianyl @hwwwwei.co.m com yxcheng&connect. hku.hk hui chichi Zhou. 250uc1 ac..uk { [Lixiaoguangl1, Shang. Lineng @whawwii....mm. com
ABSTRACT
Recent advancements in LLM-based information-sccking agents have achieved becord-breaking performance on established benchmarks. However, these agents remain heavily reliant search-engine-indeeed knowledge, leaving critical blind spot: Unindexed Information Seeking (UIS). This paper identifies and ex- plores the UIS problem, where vital information is not captured by search engine crawlers, such as overlooked content, dynamic webpages, and embedded files. Despite its significance, UIS remains underexplored challenge. To address this gap. we introduce UIS-QA, the first dedicated UIS benchmark, comprising 110 expert-annotated QA pairs. Notably, even state-of-the-a of-the-art agents experience a drastic performance drop on UIS-QA (e.g., from 70.90 on GAIA and 46.70 on BrowseComp-zh to 24.55 on UIS-QA), underscoring the severity of the prob- lem. To mitigate this, we propose UIS-Digger, novel multi-agent framework that incorporates dual-mode browsing and enables simultaneous webpage searching and file parsing. With relatively small 330B -arammeete backbone LLM opti- mized using SFT and RFT training strategies, UIS- UIS-Digger sets a strong baseline at 27.27%, outperforming systems integrating sophisticated LLMs such O3 and GPT-4.1. This demonstrates the importance of proactive interaction with unin- dexed sources for effective and comprehensive information-sseeing. Our work not only uncovers fundamental limitation in current agent evaluation paradigms but also provides the first toolkit for advancing UIS research, defining new and promising direction for robust information-seeking systems.
INTRODUCTION
With the emergence of Large Language Models (LLMs) augmented by tool calls and agent-based workflow designs, modern AI systems have demonstrated impressive capabilities in performing complex real-world information seeking tasks (OpenAl, 2025). These methods usually lever- age powerful tools such as search engines and crawlers for retrieving external knowledge (Team) 2025b at Li et al. 2025a), which term Indexed Information Seeking (ISS.. While existing benchmarks such as GAIA (Mialon et al.) (023a) and BrowseComp (OpenAI Team, (025) sug- gest current agent system's 's advancement information seeking, these benchmarks do explicitly measure the extend of agent's reliance on search engines or the ability in discovering information scattered across unindexed pages.
In real-world scenarios, however, many tasks involve unindexed information seeking (UIS). where necessary information is hidden in obscure corners of the Internet, embedded in files, excluded from search engine indices due to crawling and ranking limitations. As shown in Fig. [ for UIS questions, search engines may return related pages but fail to provide the direct content needed and interactions such date selection and visual graph reading. Recognizing that existing benchmarks
"Equal contribution.

---

Published as a conference paper at ICLR 2026
was maximum ridel range meterss) port day of the Mid-Antumn Festivu?"
UIS- Agent:
date selecter and option 2o. page
tine answar 20 2:34 2255m
Search
Undexed-Informaatee
Deep-Interactions
Unindexed-nnoormation
Search
Brmak antieied "oa.
"xoc 2014 and cutumn tide range" "NoO recedds"
Nayr The Prott..
"2024 mid
Sagges 'D Snaeeppoennn
Sonr there boaedteea io
Agent:
axcodding to Aouartaa
Figure 1: UIS problem. Previous information agents (bottom) focus primarily indexed information and thus often fail to gather the evidence needed to answer complex queries, either rejecting to answer generate hallucinations. In contrast, UIS agents (top) equipped with additional tools and fine-tuned to excavate unindexed information, thus capable of interacting with websites deeply and solve UIS tasks reliably.
overlook the intrinsic distinction between IIS and UIS, which leads to insufficient adaptation and evaluation on agent's UIS capability, we introduce UIS-QA, the first benchmark explicitly designed for UIS capability evaluation. It consists of 110 carefully annotated and cross-vall validated test samples, ensuring correctness, objectivity, and temporal stability. The tasks UIS-QA cover a wide range of action spaces including search, crawl page, download files and webpage interaction (e.g., select option), requiring agents to skillfully interact with webpages and excavate unindexed information.
Our experiment results reveal that even the top information-sseekig agents struggle in UIS-QA, where the strongest baseline yielding only 25.45% accuracy, much lower than their GAIA and BrowseComp-zhh (Zhou al. (0255) performance (over 70% and 45%, respectively). This find- ings highlight the significance of an UIS benchmark towards comprehensive evaluation of infor- mation seeking agents. With comprehensive analysis the failure modes, we also identify two major causes of methods that perform poorly on UIS tasks, namely, the insufficient action space and limited foundation models.
As mentioned above, solving UIS tasks usually involved a wide range of actions, which can be out of the action space for search-engine-basee agents (Li et al. 2025a Team 20256 openPanGu Team 2025 Shi et al. 2025). making UIS problems theoretically unsolvable. While the action space affecting the upper bound, the capability of foundation models sets the lower bound, determining whether the agent can take correct choices and forms a rational strategy within the large action space.
To this end, we also propose UIS-Digger, a multi-agent system for deep research tasks, with a ver- satile framework and tuned inner L.M,, supporting affluent actions of searching and deep brows- ing. The dual-mode browser within UIS-Digger allows the agent to dynamically switch between visual (screenshot) and textual modes, providing richer and more efficient webpage understanding. Furthermore, UIS-Digger also incorporates file readers and parallel tool execution, significantly strengthening its UIS- UIS-solving ability. We also tuned the underlying LLM using synthesized QA pairs through two stages: an initial supervised fine-tumin tin-ttning (SFT) round for cold start, followed by rejection sampling fine-tuning (RFT) for bootstrapping UIS capability. The final system achieves 27.27% accuracy on UIS-QA using ~30B backhone LLMs, surpassing all existing baselines, in- cluding those integrating sophisticated LLMs such as O3 and GPT-4.1.
We summarize the principal contributions of this work as follows:
We identify and formalize the overlooked problem of Unindexed Information Seeking (UIS), highlighting its intrinsic distinction from IIS and demonstrating that even state-of- the-art iiloommttoonseekiing agents remain limited in UIS scenarios.
We introduce UIS-QA, the first benchmark dedicated to UIS, featuring a rigorously val- idated dataset for systematically evaluating agent systems. Alongside, we propose UIS-
2

---

Published as a conference paper at ICLR 2026
Digger, versatile multi-agent framework that serves as strong baseline, achieving best-in-class score of 27.27%.
We conduct detailed analyses of fallure cases and agent behavior evolution across training stages, offering concrete insights and resources to guide future research in advancing the UIS domain.
2 UIS-QA
Since there are scarce previous studies explored the UIS problem, how to evaluate an agent system's s ability under UIS task setting is still a missing piece of the puzzle. Therefore, we propose new benchmark named UIS-QA. In this section, we will elaborate the construction of UIS-QA in three parts: the problem formulation, the data collection procedure, and the UIS filtering.
2.1 PROBLEM DEFINITION
To begin with, the whole Internet can be understand as structured collection of vast number of webpages P. As one of the most prevalent entrances of the Internet, search engines normally have crawled and organized large portion of the webpages, denoted as p(E), which is also known 'in- dexed pages'. The information retrieved by search engine is thus defined as 'indexed information'. We formalize this concept as follows:
p(e) = {p.) ((u,, s:) un E u, E Es}
(2) where each webpage retrieved by the search engine is represented as a tuple of a URL and snippet 8,, The collections of all the URLs and snippets from p(E) are represented as s and u. respectively. In other words, all the information present in the page snippets or in the results of one-step crawling from indexed pages can be considered as indexed information. Unless specified otherwise, in the following sections we Google Serper as the default search engine for E.
II= = [I rrl crrww(u))),
Conversely, 'unindexed information' refers to all other information on the Internet excluded ZZ:
UI
{xr x E (P II)}
In practical terms, it is infeasible to examine all the pages indexed by a search engine. Thus, the above definition serves a theoretical model. In reality, due to computational constraints, only small set of search queries can be fed into the search engine and only few top pages returned from the searches will have chance be visited. Hence, we introduce approximations for II and UI:
A( P = (E(g)))=11)) m {(ūū II = {- x E ( crawl(ūj)
(5)
uI {- I II}
(6)
Here, A denotes an arbitrary information-sseking agent system that receives a task Q from the user and formulates m search queries qi for searching via E. II represents the practically accessible indexed information based the search engine & and queries {q.), which is subset of the ideal II. Consequently, the remainder of P not included in II becomes unindexed information L/I.
Compared to the ideal definition, in practice, II is much smaller than II, making it more likely that the target information necessary solve the user's task is located in WII This practical limitation highlights the widespread and critical nature of the UIS problem.
Based the definition of unindexed information, further formalize the UIS problem follow:
(2,C)=
- C ue(V) = -cIce IzJ Uc/ Volcceuit where |c(U)] > 0, and (Q, (()) #
(8)
(9)
(10)
https///www.serperdde
3

---

Published as a conference paper at ICLR 2026
Real-Wold Wold Web Environment
Unknown Startpoint
Unindexed- Information Dependence
Final Answer- oriented Evaluation
Task Type
WebArena
Computer Use
Mind2Web
Computer Use
Mind2Web-Lin
Computer Use
Online-Mind2Wer
Computer Use
X
nowsocomp-en/zil xbench- DeepSearch
Info Socking
Info Seeking
GAIA-textual-10
Seeking
UISQQA
Info Secking
Table 1: Comparison of our UIS-QA and existing benchmarks.
To solve a user's question Q, a context C consisting of both indexed and unindexed information is required, denoted as ((1) and C(U), respectively. If the required unindexed information empty and the correct answer cannot be inferred from (()), then the is UIS problem.
We compare UIS-QA with existing information-seeking and computer-use datasets along five key dimensions, as summarized in Tab. T Task Type: Information-seekkao datasets (e.g., Browsecomp- en/zh (OpenAl Team 2025 Zhou et al. 2025b). xbench-DeepSearch([ii
et al. 2025b). GAIA- rextual-10((Mialnn al. 2023b)) require multi-step exploration on the open web, emphasiz- ing search strategy and information extraction. In contrast, computer-use datasets (e.g.. We- bArena(Zhou al.) 2024), Mind2Web(Deng et al. (023). Mind2Web-LiveePPne et 2024), Online- Mind2Web(Xue et al. 2025)) focus on performing interactive browser actions (c.g.. click, type) to accomplish user goals, prioritizing tool operation proficiency. Real-World Web Environment: UIS-QA evaluates in the live public Internet. This exposes agents to real- complexities such as outdated information, distracting content, complex layouts, and wwweerttsements-cchllleeeee absent in controlled settings. Unknown Startpoint: UIS-QA provides no predefined starting point. Agents must initiate searches using general-purposs engines (e.g.. Google) and navigate the entire web, without being restricted to specific sites. mnindexed-Informatiie uniquely requires reliance on information not directly accessible via standard search results. Final Answer-Oriented Evaluation: The benchmark employs deterministic short-form answers for eval- uation, minimizing subjective judgment and enabling fully automatic scoring. In summary, UIS-QA holistically evaluates the integration of information-seckiin and computer-use capabilities under realistic and demanding web interaction settings.
largely
Dependence: UIS-QA IA
2.2 DATA COLLECTION
Following the definition of UIS tasks, we form an expert group to manually annotate question- answer (QA) pairs, and filter out those can be solved using only indexed-information solely. This process resulted in a test set of 110 high-quality UIS data samples. Specifically, the team is asked navigate deeply authoritative or official websites, performing interactive actions such as multi- round clicks, option list selection, setting filters, setra-searching, and downloading files. Afterward, the annotators arrive at an information source such as specific webpage or file. Based on the content of this page or file, the annotator then formulated a question, whose answer could be found or inferred from the available content. For each website, we restrict the annotators to compose maximum of two QA pairs to ensure diversity. To further improve the quality of the annotation process, we emphasized the following principles:
Objectivity: unlike open-ended or subjective questions, our setting requires answers in the form of factual fill-in-the-blee blank questions. Thus, the answer 2 to each question Q is expected to be objective, deterministic, and unique.
Authritativeness: our golden answers are strictly derived from authoritative sources. Duc to the intrinsic nature of UIS, such sources often not searchable and demand strong world modeling ability to know which websites contain the appropriate authoritative information. This challenges the model to identify reliable sources amid abundant secondary and confticting information.
Static Nature: given the dynamic nature of the internet, some content may change significantly over time (e.g., "What is today's weather?"), making it unsuitable for our benchmark. Therefore,
4

---

Published as a conference paper at ICLR 2026
WebSurfer
Question
WebSearcher
Groweer
Mas Saniigg
pall regiort
Planner
Actton
Answer
4432
Reader
File
Texd
parter
Coselcaa No
Figure 2: UIS-Digger multi-agent system. Planner, web searcher, web surfer and file reader works together to solve UIS problems. The web surfer can switch between textual- and visual-mode to observe webpages and hence make next-step decisions. Zoom-in for better view.
annotators were instructed to ensure that answers are static, so that the comparisons between agents could be fair across different testing times.
Verifiabilliy: to assess the performance of agent systems UIS-QA, a verification tool. Consequently, the answers must be verifiable. Most of the "golden" answers are presented in the form of numerical values, dates, logical statements, proper nouns. Some answers are also defined by unambiguous rules (e.g., "including either A or B can be considered correct").
a ruse-based LLM as
Accessibility: annotators asked avoid posing questions that would trigger human verification (e.g., CAPCHAs) during the browing process. Similarly, websites with access-restricted content requiring login also excluded from consideration.
2.3 UIS FILTERING
Even under the strict collection rules described above, some questions are still inevitably solvable using only indexed information. Therefore, we design UIS filtering pipeline to remove IIS ques- tions. Firstly, each question is independently examined by three annotators, who use Google Search to check whether the target content can be directly retrieved from the search engine. If the search engine result page does not directly contain the target content but contains a link that redirects to the actual content page, the question is still considered UIS. In addition to manual verification, we employ z.ar as an automatic verifier to filter out IIS questions. However, if question can be swered by z.ai only after downloading a file, we classify as UIS, since file access requires explicit browsing actions beyond indexed snippets. Next, we leverage offline LLM (e.g.. Deepseek-R11 to filter out questions answerable from LLM's inner knowledge (DeepSeek-Al 2025). Finally, we obtain 110 high-quality samples that constitute UIS-QA.
Among the 110 samples in UIS-QA, 84 questions are written in Chinese and the remaining in En- glish. The questions span variety of domains, including government announcements, official prod- uct introductions, source code repositories, games, and company annual reports.
3 UIS-DIGGER: A MULTI-AGENT FRAMEWORK FOR UIS PROBLEMS
As mentioned above, there are few existing works that have studied the UIS problem. Therefore, propose new agent-system framework for UIS problem solving, named UIS- UIS-Digger, which can serve as a fundamental methodology for UIS-QA. In this section, UIS-Digger will be elaborated in three aspects of agent design, framework architecture, and training process.
3.1 AGENT AND ARCHITECTURE DESIGN
In Fig. 2 we introduce the overall architecture of UIS UIS-Digger. UIS-Digger is a multi-agent system engaging four agents, planner, web searcher, web surfer and file reader. Each agent equipped
htpp:
5

---

Published as a conference paper at ICLR 2026
with set of tools and assigned to a specific category of sub tasks. For every new instruction, the agent initializes an empty memory and works in an iterative problem-sol solving process inspired by the ReAct paradigm (Yao et al. 2023). The agents communicate with each other and corresponding tools via a request-response message system.
Planner: upon receiving new query, the top-level planner decomposes it into set of sub- tasks, coordinates the execution among the three subordinate agents, and delivers the final answer to the user.
Web Searcher: the web searcher concurrently employs search engines and crawling tools to retrieve indexed information (Eq. 5). and may further delegate sub-tasks to the web surfer and file reader to obtain unindexed information from web URLs or files.
Web Surfer: The web surfer starts from a URL and operates a browser access unindexed infor- mation. Its action space covers common interactions with websites, including clicking, scrolling, typing, selecting, navigating, submitting forms, downloading files, locating elements, and taking screenshots. Unlike previous browser- -itteerrntteratted methods with either purely textual or visual obser- vation about the webpage (Zheng et al., 2025) CAMEL-AL.rgg. 2025p, we introduce a dual-model memory-shared browsing strategy, to balance both completeness functionality and high efficiency. Crucially, unlike previous multimodal agents, our surfer maintains a shared memory and consistent browser state across textual and visual modes. This design preserves a unified working history, climinates synchronization overhead, and encourages efficient inference by prioritizing textual mode while reserving visual inspection for essential cases.
File Reader: both the information seeker and web surfer can download files, which are then pro- cessed by the file reader supporting formats such PDF, XLSX, and DOCX. When content exceeds the context window, it is incrementally read chunk by chunk, following Yu et al. (025b))
3.2 AGENT TRAINING
UIS-Digger requires specialized capabilities from its inner LLMs, including task decomposition, tool usage, and integrating diverse information for UIS tasks. To this end, we construct synthesized training data and tune the inner LLMs in two stages of SFT and RFT.
3.2.1 TRAINING DATA CONSTRUCTION
For efficiency, synthesize QA pairs rather than rely solely manual annotation. We draw upon both real-world information from the internet and simulated environments, as illustrated in Fig. 3
To construct QA pairs from real-world information sources, over one hundred base websites are collected, across domains such as public companies, product catalogs, government announcements, data dashboards, and code repositories. UIS-Digger is instructed to roam within these websites and extract five informative sections about a chosen entity, forming a context as defined in Eq. [ is designed gather information from deeper webpages by performing various browsing actions. Then we deploy another LLM to compose a question and label the corresponding answer based on this context, followed by an LLM judge filtering out ambiguous or subjective questions. The prompts used for information collection and query generation are provided in Appendix B
To address early weaknesses in handling interactive web elements, such as selecting a date in datetime picker, we further developed three types of virtual websites that simulate flight booking and statistical data lookup scenarios. These websites incorporate specific interactive elements that posed challenges to the earlier version of UIS-Digger. Each virtual site is provided fictitious JSON database (e.g., synthetic shopping records). QA pairs can be directly derived from the database, while UIS-Digger must solve them by interacting with the simulated website. This simulation strat- egy significantly enhances the agent's ability to manipulate widgets such as radio buttons, date selectors, filters, and graphs.
Based on the constructed QA pairs from real and virtual websites, we employ UIS-Digger to solve these questions and collect the trajectories, which are then filtered with reject-sampling method and used for tuning the inner LLM of UIS-Digger. The final result trajectories are used in two stages of SFT and RFT, with disjoint question sets allocated each.

---

Published as a conference paper at ICLR 2026
Real-world QA pairs
Simulared QA pairs
Homepage Collection
Difficult Action Space Identification
Narm some officie ssel peboiies of (ortegurs))
Public Companies
Code repositorics
Goverment Announcements
Unindexed Information Collection
Webpage Generation
Pleas
official (hommsoege))
Question Generation
Question Generation
Qu... A....
Qu.. Ain
It currently (month)) Based or the spoefic Information in the provided text sogrremts...
A
Figure 3: QA Pairs Construction Pipeline. (Left): Procedure for constructing QA pairs using real- world information. First, homepages potentially containing deep navigation structures and informa- tive content are collected. UIS-Digger then explores these homepages to extract information from pages requiring multiple navigation steps. The collected information subsequently serves as context for query generation. (Right): Procedure for constructing QA pairs based on simulated webpages. We identify browsing actions that UIS-Digger struggles to perform, and generate webpages (along with a JSON database containing relevant statistics) that incorporate these actions. QA pairs are then generated using the information from the JSON database of these simulated webpages.
3.2.2 TWO-STAGE TRAINING
In the SFT stage, we integrate powerful teacher model X to solve sampled questions with temper- ature 0, producing one trajectory per question. A separate L.M judge verifies (1) the correctness of the final answer and (2) whether the question is trivial, i... if first-round reply already contains the golden answer z. We adopt reject sampling, retaining only those correct and non onn-trivial trajectories. The resulting SFT-tuned model is denoted as X²", which is then used for RFT trajectory generation.
In the RFT stage, X is deployed to solve the remaining training questions, with temperature 0.4 and sampling group size of four, encouraging exploration. The same reject-sampling strategy is applied. To emphasize challenging tasks, we reweight samples by difficulty, measured by the number of correct attempts. Specifically, trajectories from challenging questions be retained than those from easier ones. Bootstrapping with these RFT trajectories yields the final model X",, which integrated the default LLM in UIS-Digger. Unless otherwise specified, all subsequent experimental results are reported with X".
likely
4 EXPERIMENTS
In this section, we present results and analyses on the proposed benchmark UIS-QA and the UIS- Digger system. Across nearly all baseline methods, observe substantial performance gaps be- tween UIS-QA and prior non-UIS tasks, underscoring the significance UIS-QA -AA as a novel bench- mark. Furthermore, through error analysis across different models, identify several key factors that determine an agent system's 's success in UIS.
4.1 EXPERIMENTAL SETTINGS
To distinguish the different nature of the proposed UIS-QA benchmark from exisiting ones, we conduct affluent evaluations on existing advanced information seeking agents:
Direct API Inference These methods directly query a base LLM through provider's APIs, with action space (e.g., whether models such as DeepSeek- V3.1 (DeepSeek-Al) 2024), Claude-sonnet-4 (Anthropic 2025)
search tools) not fully disclosed. We evaluate
and GPT-5 (OpenAl 2025)

---

Published as a conference paper at ICLR 2026
Names
Action Space
Backbone
UIS-QA
GAIA
BC-2h
cravi(s)
visual
file
hrowser
Direct Inference
DeepScck-V3.1 clande-sommct44
DeepSeck-V3.1
Claude-S4
GPTS
GPT-
Commercial System
GLM-4 G Saaettiiinn, vehvaart
GLM4,5
11.8
Doubaorwehttt
Geminii 2.5 pro
proge seash
Gemini 2.5-pro
4.5
ReAct Agentic Framework
WebSailor Tongyi-DR
x
x
WebSaike-32B 32B + Qwen3-720 TongyiDR330B-N DR-330B--3BBB 33 GGPT-40
7.3
53.21
25.5
23.6
70,91
46.7
Multi-agent Pramework
DDv2
x
Pangu-3SB
8.2
34.6
03-mini Claude-S31
MinoThinker v0.1
/
MiroThinkers 32B DPO-DPR GPT-4.1 =Chnde-S117
7,3
57.9F
Memento
+ GPT-411
25.53
79.4
AWorld
Gemini-233 2.5--225ppp-- GPT-4o
5.5
32.2
TTS Digger (Pangu)
/
PanGu-388
27.3
50.5
32.5
UIS Digger (Qwen)
Qwen3-32B
27.3
47.6
32.5
Table 2: Evaluation results on UIS-QA, GAIA, and BrowseComp zh (BC-zh). indicatess reasoning- oriented LLMs. denotes results measured on GAIA-text rather than the full GAIA benchmark. $ indicates that the UIS-QA score for Memento (Zhou et al. (225a) is reported without using its case bank, since UIS is task type and only limited cases have been previously allocated. Action spaces including crawl (read webpage content), visual (read images), download file and opearte browser included.
Commercial Systems. Beyond single LLM, these systems adopt more sophisticated architectures that theoretically enable broader action space such as searching. GLM- 4.5 (Team et al.] (205)), Doubao (Seed team 2025), Gemini-2.55ppr (DeepMind) 2025) belongs to this category.
ReAct-based Frameworks. A straightforward agent design that couples reasoning and ac- tion, represented by WebSailor (Li al., 2025a) and Tongyi DeepResearch (Team 2025b).
Multi-agent Frameworks. These methods implement multi-agent architectures where specialized agents handle different tasks such as webpage crawling, visual signal interpreta- tion, file reading, and browser operation. Many systems in this group achieve strong results traditional benchmarks like GAIA and BrowseComp. Examples include DDv2 (open-
PanGu Team 2025), OWI (CAMEL-AL.orf mento (Zhou et al. 2025a), and AWorld Yu et al. (025a)
2025), MiroThinker (Team 2025a). Me-
The proposed UIS-Digger with backbone X is also evaluated in this part. We trained two ver- sions of X", a 38B-Pangu model (Chen al., (225aa) and Qwen3-32B model (Yang al. 2025). During training, only LLM -LM-generated tokens are updated with gradient backpropagation, while tool responses are excluded. Implementation details of the two stages are provided in Appendix
4.2 MAIN RESULTS ON UIS-QU -ASQQA
In Tab. - we present the evaluation results of baseline methods and UIS-Digger. UIS- UIS-Digger achieves the highest score of 27.27% on the UTS-QAbenchmark, outperforming even sophisticated systems powered by 03. In addition, delivers competitive results on conventional information- secking benchmarks such GAIA and BC-zh, demonstrating strong generality. These findings suggest that UIS-Digger establishes a solid baseline for advancing research on the UIS problem.
By contrast, all baseline methods suffer substantial accuracy drops under the UIS setting. Even strong systems such as Tongyi-DR and Memento, which exceed 70% accuracy on GAIA, drop to only 23.6% and 25.5% UIS-QA- QA-ooorrepoondini to declines of 47.3% and 53.9%, respectively. This sharp degradation reinforces our central motivation: UIS remains an underexplored and insuf- ficiently addressed capability in current agent systems.
8

---

Published as a conference paper at ICLR 2026
Beyond the ranking of baseline methods, it is also worthy to note that methods that achieve higher scores on general iifoommation-seeiing tasks such as GAIA also tend to perform relatively better on UIS-QA. This correlation suggests that a strong foundation model (e.g., 03 in Memento) is still essential for UIS tasks.
Nevertheless, When comparing ReAct-style methods with more complex agent frameworks, ob- serve that the relative distribution of UIS and IIS scores is not fundamentally different. Even within the same framework type and similar action spaces, these methods exhibit large performance dispar- ities, with gaps of up to 17.3% and 20.9%, respectively. We hypothesize that while larger action space theoretically enables more diverse strategies, it also expands the search space and introduces new challenges. The main bottleneck, therefore, shifts to the underlying LLM's M fundamental ability.
4.3 ANALYSIS
To systematically analyze the challenges faced by agent systems in solving UIS tasks, we conduct a detailed examination of their searching and browsing behaviors. Fig. 4 illlustrates two key aspects: the proportion of trials successfully grounded to the golden information source (left), and the action frequeney distributions across correct and incorrect samples after different training stages (right).
Gains from SFT and RFT Training Both SFT and RFT training stages lead to substantial ac- curacy improvements UIS-QA, demonstrating the effectiveness of the two-stage tuning strategy- For instance, UIS-Digger with a PanGu backbone achieves gains of 13.6% from SFT and an addi- tional 4.6% from RFT. Further details and extended results are provided in Appendix D.
Error Analysis On the left side of Fig. 4 we analyze the searching behaviors of four representative methods- Tongyi-DR, WebSailor, and UIS-Digger-on UIS-QA. We evaluate whether an agent successfully retrieves and accesses the root website of the annotated golden information source. The root website is defined as the domain name of the ground-truth webpage, and actions such as crawling, surfing, or downloading the golden webpage URL are counted as visits. The three concentric rings in each pie chart, from the innermost to the outermost, denote: (1) final answer correctness, (2) whether the golden root website is retrieved during search, and (3) whether the golden root website is subsequently accessed.
Observed from different parts of the pie charts, we identify several key patterns. For clarity, sections are denoted by their colors from the inner to outer rings (e.g., BBR stands for Blue-Blue-Red),
More illustrative examples provided in the Appendix E
Missing retrieval (RRR) and knowledge sourcing (RBR) are two dominant failure modes. Without retrieving the root page, solving UIS problem becomes theoretically impossible. underscoring the need for robust search capabilities. Even when homepages are retrieved, agents often fail to select the correct knowledge source among the results, highlighting the importance of precise source identification. These patterns emphasize the value of UIS Mis-QAin exposing uss-sccccfic weaknesses in agent behaviors.
UIS remains difficult even when the source page is reached (RBB). Another substantial fraction of cases involve correctly retrieving and visiting the root website but still producing incorrect final an- swers. Such failures stem from the inherent complexity of UIS action spaces: even when starting from the correct source, agents must execute intricate operation sequences- such as multi-step navi- gation, filter adjustments, or repeated back-and-forth exploration. This calls for stronger continuous reasoning and ong--horizon planning capabilities future agent systems.
Intrinsic knowledge and alternative sources offer only limited shortcuts. We also observe small number of correct cases where the golden root website is neither retrieved nor visited. Our man- ual inspection suggests two explanations: (I) agents occasionally leverage intrinsic knowledge of URLs to directly access relevant pages, and (2) third-party websites sometimes redundantly host the required information. While such cases reveal that prior knowledge or external redundancy can occasionally "hack" " UIS tasks, their rarity indicates they do not fundamentally mitigate the UIS challenge.
Tool Usage Across Training Stages We observe clear shifts in tool-utilization patterns as the agent advances through training. As shown in Fig. 4 (right), the frequency of search tool calls in-

---

Published as a conference paper at ICLR 2026
30.0%
25.5%
19.1%
20.0%
Zero-shot
30.
After SFT
25,55
2.7%
1.4%
8.2%
3.4
2223
1.0
2,60 To
net seath wut
accesed
Incorrect Conea
Zero-shot
20.0%
After SFT
After RFT
Rebballo
Digger
19.1
60.5%
92.7%
72. 7%%%%
9,0g
47.3%
search
1/ Haader
cried
browee
tslal
Figure 4: Action analysis. (left): Search behaviors of UIS-Digger and three baseline methods. The pie charts show the proportions cases where the agent successfully retrieves the root URL via search, and whether the root URL is subsequently accessed through crawling or browsing. (Right): Action frequency distributions of correct and incorrect cases for Pangu-38B UIS-Digger at different training stages. Zoom-in for best view.
creases across both correct and incorrect trajectories, reflecting the growing reliance on external re- trieval, which is believed to potentially reduce hallucination. In contrast, file-parsing actions remain largely unchanged, consistent with their role as follow-up step once relevant files are downloaded.
A critical difference emerges in the use of the crawl tool. The untrained model fails to invoke it altogether, whereas this capability appears after SFT and further improves with RFT, underscoring the importance of staged training for acquiring essential behaviors. Browsing actions reveal another important trend: in successful trajectories, browsing attempts sharply decrease over training, indicat- ing more targeted and efficient navigation. Conversely, unsuccessful trajectories show increasing number of attempts, suggesting heavy unsuccessful exploration.
Overall, correct trajectories follow trajectory of "learn then streamline": tool usage rises after SFT as the agent learns to solve more complex tasks with longer tool-use sequences, then declines as navigation efficiency improves with RFT. Incorrect trajectories, however, show a monotonic increase in tool calls, reflecting prolonged retries that fail to converge to a correct solution.
CONCLUSION
In this paper, identify the overlooked problem of Unindexed Information Seeking (UIS), where indispensable information resides beyond the reach of search engines. To systematically evalu- this UIS capability, we introduce the UIS-QA benchmark, which provides dedicated set for assessing agent systems on UIS tasks. Although existing agents achieve strong performance on conventional Eiformation-seeking benchmarks, their ability to solve UIS problems remains lim- ited. Consequently, we propose UIS-Digger, an agent system with enhanced web-interactive tools and trained through sequential SFT and RFT stages. Our results demonstrate that with an appro- priate action space and tailored training strategy, UIS ability can be effectively bootstrapped, en- abling UIS-Digger to achieve state-of-the improvements, the absolute accuracy of UIS- at 27.27% remains far from satisfactory, under- scoring the difficulty of UIS. We hope that UIS-QA will encourage further research this direction and inspire the development of more practical and generalizable deep research agents.
performance on UIS-QA. Nevertheless, despite these
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Moonshot AI. Kimi-researccer: End-to-end rl training for emerging agentic capabiiitie,, 2025, URL httpp:://noonshoaal..ttttt iooioo/iiii -Reeieeheer
Anthropic.
Introducing claude 4, 2025,
URL
claude-2
CAMEL-ALorg. Owl: Optimized workforce learning for general multi-agent assistance in real- world task automation. https:://githii com/ came watpe-aameellaaaaaeeeaaaaaaaaaaa ai/owl 2025. Accessed: 2025- 03-07.
Sky CH-Wang, Darshan Deshpande, Smaranda Muresan, Anand Kannappan, and Rebecca Qian. Browsing Lost Unformed Recollections: A Benchmark for Tip-of-theeTooguee Search and Rea- soning, March 2025. URL http:://aaaaae org/ /tbb//2503 19193 arXiv:2503.19193 [cs].
Hanting Chen, Jiarui Qin, Jialong Guo, Tao Yuan, Yichun Yin, Huiling Zhen, Yasheng Wang. Jin- peng Li, Xiaojun Meng, Meng Zhang, Rongju Ruan, Zheyuan Bai, Ychui Tang, Can Chen, Xing- hao Chen, Fisher Yu, Ruiming Tang, and Yunhe Wang. Pangu light: Weight re-initialization for pruning and accelerating Ilms, 2025a. httpttss: /ttp:s://arxiie http://wwww...../.aaaaoa/a
wwwp:////aaaaaaaaaae.oo.ee aww.aabb./555.2 20155
Kaiyuan Chen, Yixin Ren, Yang Liu, Xiaobo Hu, Haotong Tian, Tianbao Xie, Fangfu Liu, Haoye Zhang, Hongzhang Liu, Yuan Gong, Chen Sun, Han Hou, Hui Yang, James Pan, Jianan Lou, Jiayi Mao, Jizheng Liu, Jinpeng Li, Kangyi Liu, Kenkun Liu, Rui Wang. Run Li, Tong Niu, Wen- long Zhang. Wenqi Yan, Xuanzheng Wang. Yuchen Zhang, Yi-Hsin Hung, Yuan Jiang. Zexuan Liu, Zihan Yin, Zijian Ma, and Zhiwen Mo. xbench: Tracking agents productivity scaling with profession-aligned real-world evaluations, 2025b. URL //tp:///awvvvvv... wwwpwwwwww.//b///55550e33
13651
Google
DeepMind.
Gemini model & thinking updates
(march
2025). gemini ni model - thinkngtinking-- http://aaaaaaacccooooooo///aaaaa 2025.
http:////oog.oooo//////ooooooooo deepomiin
DeepSeck-AL. Deepseek-v3 technical report, 2024. URL
19437
DeepSeek-AL. Deepseck-rl: Incentivizing reasoning capability in ilms via reinforcement learning,
2025. URL httpttts///xii...crtttt http://axiiv..rr//a/////00..111.
Xiang Deng. Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web, 2023. URL https://arxiv.
org/ hrg//bs/2an 230//3366.000 06070
Mingxuan Du, Benfeng Xu, Chiwei Zhu, Xiaorui Wang, and Zhendong Mao. Deepresearch bench: A comprehensive benchmark for deep research agents, 2025. URL https::/arxiv..or
abs/ 0t///2566..111
Google. Deep research is now available on gemini 2.5 pro experimental.
Google Blog. 2025.
URL ddeep-resee resp-reesaach gemini-iiiiiiie 2-1----5-------------- pro- extx-eexerimmtal/////
Liang Hu, Jianpeng Jiao, Jiashuo Liu, Yanle Ren. Zhoufutu Wen, Kaiyuan Zhang, Xuanliang Zhang, Xiang Gao, Tianci He, Fei Hu, Yali Liao, Zaiyuan Wang, Chenghao Yang, Qianyu Yang, Mingren Yin, Zhiyuan Zeng, Ge Zhang, Xinyi Zhang, Xiying Zhao, Zhenwei Zhu, Hongseok Namkoong, Wenhao Huang, and Yuwen Tang. Finsearchcomp: Towards realistic, expert-level evaluation of financial search and reasoning. 2025. URL /ULK::/tee/aarr....o///////....
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training 1ims to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.095/66 2025.

---

Published as a conference paper at ICLR 2026
Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baix- uan Li, Zhengwei Tao, Xinyu Wang, Weizhou Shen, Junkai Zhang. Dingchu Zhang, Xixi Wu, Yong Jiang, Ming Yan, Pengjun Xie, Fei Huang, and Jingren Zhou. Websailor: Navigating super- human reasoning for web agent. arXiv preprint arXiv:2507.080 220.0299... 2025a.
Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yutao Zhu, Yongkang Wu, Ji-Rong Wen, and Zhicheng Dou. Webthinker: Empowering large reasoning models with deep research capability, 2025b. wttptttts///aii..ctot
abs/ 2504. 21776
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations, 2023a.
Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general assistants, 2023b. URL https://axxiv.ort
abs/ 12983
OpenAL. Introducing gpt-5, 2025.
URL https://openai, comm:ndex/ ndex
Introducing-gpt 57
OpenAl. Introducing deep research, 2025. URL https: https://openal..omm/reee
introducing-ded -dep- research/ Accessed: 2025-09-13.
OpenAl Team. Browsecomp: a benchmark for browsing agents. https::/opeeai.com
index//rowssecomp: 2025. Accessed: 2025-04-29.
openPanGu Team. Deepdiver v2 technical report, 2025. URL https::/ai.gitcomm
Enbededddnnngndddddddddddddddddd 7B /Bepepppiieeeebbbo///aaa////////
Seengaangu-deepplleee--------eee repprueeerre-ttteeerrrrrr.....
Yichen Pan, Dehan Kong, Sida Zhou, Cheng Cui, Yifei Leng, Bing Jiang, Hangyu Liu, Yanyi Shang, Shuyan Zhou, Tongshuang Wu, and Zhengyang Wu. Webcanvas: Benchmarking web agents in online enviromments, 2024. URL BRRhhtps:: http://awxwww.../aeeeeeeee
12373
Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, and et al. Sean Shi. Humanity's 's last exam, 2025. URL
attps::/arxiis org/ wtwgaas/////ooooooooggg/ogeeeee/
ByteDance Seed team. Doubao-1.5-pro, 2025. URL htttp:://seed.prrt
com/e/cceccoce/eo com/er.
$peccia1/buuba____5__2 pro
Wenxuan Shi, Haochen Tan, Chuqiao Kuang, Xiaoguang Li, Xiaozhe Ren, Chen Zhang, Chen Hant- ing, Yasheng Wang, Lifeng Shang, Fisher Yu, and Yunhe Wang. Pangu deepdiver: Adaptive search intensity scaling via open-wer web reinforcement learning. arXiv preprint 23X3222505. 2025.
Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Yan Zhang, Fei Huang, and Jingren Zhou. Zerosearch: Incentivize the search capability of lims without searching, 2025a.
http//aaavvrrg/aaa///////// www./oor///////.... abs./505..2.. 04588
Shuang Sun, Huatong Song, Yuhao Wang, Ruiyang Ren, Jinhao Jiang, Junjie Zhang, Fei Bai, Jia Deng, Wayne Xin Zhao, Zheng Liu, Lei Fang. Zhongyuan Wang, and Ji-Rong Wen. Sim- pledeepsearcher: Deep information seeking via web-powered reasoning trajectory synthesis, 2025b. URL https: http://axxvvv.. org. abs/2505 1834
5 Team, Aohan Zeng. Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng. Jiajic Zhang, Kedong Wang, Lucen Zhong, Mingdao Liu, Rui Lu, Shulin Cao, Xiaohan Zhang, Xuancheng Huang, Yao Wei, Yean Cheng, Yifan An, Yilin Niu, Yuanhao Wen, and et al. Yushi Bai. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models, 2025. URL https: / wwwxiviiivv..... org/ abs httg/abs//508.0477777 255./
MiroMind AI Team. Mirothinker: An open-source agentic model series trained for deep search and complex, long-horizon problem solving. htttps://gil com/ wwtteiioe///aaaeee Mwt/MMMooMMid//// MiroThi nker) 2025a.
12

---

Published as a conference paper at ICLR 2026
Tongyi DeepResearch Team. Tongyi-deepresearch. http:://ottab...m///abbbbbbbbb/ DeepResearch, 2025b.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique Multi- hop questions via single-hop question composition, 2022.
2108. 00573
Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. Measuring short-form factuality in large language models, 2024. URL BtLLhtts://aaaae wttp://wwwrr//bbb/111.0.222
Ryan Wong, Jiawei Wang, Junjie Zhao, Li Chen, Yan Gao, Long Zhang, Xuan Zhou, Zuo Wang, Kai Xiang, Ge Zhang, Wenhao Huang, Yang Wang, and Ke Wang. Widesearch: Benchmarking agentic broad info-seeking. 2025.
07999
Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Gang Fu, Yong Jiang, Pengjun Xie, Fei Huang, and Jingren Zhou. Webdancer: Towards autonomous information seeking agency, 2025. URL
22648
XAI. Grok beta - the age of reasoning agents.
gro. 2025.
Tianci Xue, Weijian Qi, Tianneng Shi, Chan Hee Song, Boyu Gou, Dawn Song, Huan Sun, and Yu Su. An illusion of progress? assessing the current state of web agents, 2025. URL https:
7/ aaaai. org. abs 2abba//2544013.
An Yang. Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL wttph:tts///mm /ttps:://axxvvvi org/ abs /2505. 09388
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio. William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering, 2018. URL
als///as///// 809 09600
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
Chengyue Yu, Siyuan Lu, Chenyi Zhuang, Dong Wang, Qintong Wu, Zongyue Li, Runsheng Gan, Chunfeng Wang, Siqi Hou, Gaochi Huang, Wenlong Yan, Lifeng Hong, Aohui Xue, Yanfeng Wang, Jinjie Gu, David Tsai, and Tao Lin. Aworld: Orchestrating the training recipe for agentic ai. 2025a. URL http:: //ap:ii////a.vaaaaaa owg//ogg///....oooo.... 2500. 20404
Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang, et al. Memagent: Reshaping long-context 11m with multi-cony rl-based memory agent. arXiv preprint arXiv:2507.0225) 2025b.
Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deepresearcher: Scaling deep research via reinforcement learning in real-world environ- ments, 2025. URL htttps://aaxii.rrr/////// http://xxxxivrcraaa////aa/
//tpn/////////////////////////// 03160
Huichi Zhou, Yihang Chen, Siyuan Guo, Xue Yan, Kin Hei Lee, Zihan Wang, Ka Yiu Lee, Guchun Zhang, Kun Shao, Linyi Yang, and Jun Wang. Memento: Fine-tuning Ilm agents without fine- tuning Ilms, 2025a. (tt.ttts:::/ peatpa://arxivae wtteeiiioorg///eo http://ss///..0.0 16153
13

---

Published as a conference paper at ICLR 2026
Peilin Zhou, Bruce Leon, Xiang Ying. Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, Yuxin Gu, Sixin Hong. Jing Ren, Jian Chen, Chao Liu, and Yining Hua. Browsecomp-zhh Benchmarking web browsing abilitty of large language models in chinese, 2025b. URL htl.nttpp:///rrxii.orr////. 2504. 19314
Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. Webarena: Á realistic web environment for building autonomous agents, 2024. URL htl.https:://rxii.orrg///// 2307 13854
A RELATED WORK
Information Seeking Benchmarks Early benchmarks primarily focused on multi-hop question answering. For instance, HotpotQA Yang et al. 2018 was designed to evaluate multi-hop retrieval and question answering, while SimpleQA (Wei et al. 2024) focused short-form factual queries. Musique (Trivedi et al. 2022) further tested multi-hop reasoning over single-hop evidence.
More recent benchmarks demand deeper and more persistent search behaviors. Benchmarks such as BrowseComp-en/zh (OpenAl Team, (2025) Zhou et al. (2025)) and Blur (CH-Wang et al.] (025) incorporate deliberate information obfuscation, requiring agents persistently navigate the web to locate hard-to-find information. Similarly, xbench-Dece Seeechh (Chen et al. (025b) measures reasoning, tool usage, and memory in extended interaction chains. common strategy among these benchmarks is to increase task difficulty by lengthening the reasoning chain, thereby necessitating multi-step browsing and tool invocation. However, they do not explicitly evaluate an agent's ability to retrieve wnindexed information that is not easily accessible via search engines. As a result, systems excelling at conventional search may perform well on these benchmarks but fail dramatically on our proposed UIS-QA
Other works approach complex information--sekinng from different angles. WideSearch (Wong et al., 2025) evaluates broad-scale information retrieval across multiple domains and time spans, often drawing from official websites. HLE (Phan et al., 2025) focuses on challenging academic rea- soning, while GAIA (Mialon al.] 2023a) emphasizes long-horizon tool use. More recently, bench- marks like WideSearch (Wong et 20.5)), FinSearchComp (Hu et al.] (025). and DeepResearch Bench (Du et al., 2025) tackle -omainn-peccfii information needs and, doing so, occasionally involve unindexed sources through historical financial reports or specialized official data. Never- theless, such exposure remains incidental. In contrast, our work systematically isolates Unindexed Information Seeking (UIS) as a core capability dimension, offering a principled evaluation frame- work.
Information Seeking Agents Recent years have witnessed significant progress in the develop- ment of information-secking agents. Technology companies have released deep research prod- ucts, such OpenAl Deep Research (OpenAl 2025). Google Gemini Deep Research (Google) 2025), Kimi-Researcher (Al, 2023). and Grok-3 Deep Research (xAI, 2025). In parallel, the re- search community has explored multi-agent architectures for complex task orchestration. For exam- ple, OWL (CAMEL-ALorg)) (025) proposes a hierarchical framework of planning and specialized execution, while AWorld (Yu et al. 2025a) offers an open-source platform for large-scale agent- environment interaction.
Several studies focus on enhancing reasoning and exploration capabilities during web search. Web- Thinker (Li et al. 2025b) integrates reasoning processes with web exploration. Search-R1 (Jin) et al. 2025) employs reinforcement learning to enable L.MMs to autonomously generate search queries during multi-step reasoning. To address training data scarcity, methods such as Sim- pleDeepSearcher (Sun et al.) 2025b) synthesize data by simulating realistic user interactions live search environments, and ZeroSearch (Sun et al. 2025a) uses LLMs simulate search engine during training. WebDancer (Wu et al. 2025) creates challenging training tasks that demand deeper multi-hop reasoning. Furthermore, DeepDiver V2 (openPanGu Team, (025) trains multi-agent system on both closed-ended problems requiring extensive information gathering and verification, and open-ended tasks aimed at producing comprehensive long-form content.
14

---

Published as a conference paper at ICLR 2026
To explore the unindexed information seeking capabilities of agents, propose agent architec- ture that supports flexible interaction between planner and specialized subagents capable di- rectly manipulating web elements. Additionally, enhance the backbone model through carefully curated synthetic data using both supervised fine-tuning (SFT) and rejection sampling fine-tuning (RFT).
B PROMPTS USED FOR QA PAIRS GENERATION
The prompts utilized for collecting information from homepage browsing and for generating the final queries are presented follows.
Prompt for Information Collection
Please explore the official website [homepage). You are encouraged to conduct searches and to select in-tepth pages rich in substantive content for browsing. Finally, paraphrase the content of at least **vive specific articles on different topics that contain a wealth of detailed entity information**
For example:
Visit the Investor Relations section of corporate website, locate the Q3 2024 report, download it, and record the shareholding percentage of the largest individual sharcholder. Visit a museum's official website, find the "Treasures of the Museum" tab, click on and record all the listed treasures.
Note:
1. Paraphrase the specific content; do not use statements such as "for specific details, please refer to X document."
1.1 You encouraged to paraphrase information containing numerical values and names.
The collected content should ideally not be directly searchable via search engines.
2.1 You encouraged to visit related detailed pages. For example: Access "X Com- pany's accounts payable for Q2 2025 is..." and collect the specific content, then access "X Company's 's accounts payable for Q1 2025 is... and paraphrase both pieces of information.
2.2 If documents are available on the website, you are encouraged to paraphrase the specific content within those documents.
3. The source of the specific content must be the original text you actually saw; DO NOT fabricate anything!!! Paraphrase these contents verbatim directly.
4. Select objective and specific content.
4.1 The information provided by the content must be objective and definitive. For example: "In 2025, X's revenue rate was...". "X's standard numbers include...", "X was included in the National Patent Industrialization Demonstration Enterprise Cultivation Pool month of year y."
4.2 The information provided by the content cannot be vague or allow for other possi- bilities. For example: "X's advantages include...", "X's main goals are...", "X focuses aspects and z.", "The reasons does Y are..".
4.3 The information provided by the content cannot be overview/summary in nature. For example: "X's key measures include..." " "The difficulties in X's research include..." "X's prospects for the future include...".
4.4 Do not select speech-type, manifesto-type, or address- type webpages.
Maintain rigor.
5.1 For all content, considering the current date, version, etc., the collected content must not say "Sales X's flagship model were y yuan"; add conditions and change it to, for example, "Sales of X's 2024 flagship model in Mainland China were y yuan". Do not say "X has a total of 41 characters"; add conditions and change it to, for example, "Version 5.2.3 of X has total of 41 heroes".
include specific qualifying statements.
5.2 For content specific to a particular institution or enterprise, include the institution enterprise condition. Do not say "Investment meetings each quarter"; add the enterprise condition and change it to, for example, "Enterprise X's investment meetings held the last day of each quarter".
held on the last day of
15

---

Published as a conference paper at ICLR 2026
Prompt for Query Generation
currently [month). Based on the specific information in the provided text segments, please create 5 objective questions from different perspectives that have definitive answers. Attach the answers and the rationale for each question, separating multiple rationales with semicolons. Use the format: 1. Question Design: XXX Question: XXX Answer: XXX Rationale: XXX
Text Segment: [context)
Note:
1. Ask questions targeting specific information; do not focus on "task" descriptions.
The questions should ideally not be answerable by directly searching search engine.
2.1 You are encouraged to design multi-hop questions based on the text segment. For example: combine "What were Company's accounts payable for Q2 2025?" and "What were X Company's accounts payable for Q1 2025?" into "By how much did X Company's 's accounts payable increase in Q2 2025 compared Q1 2025?"
3. The source for the questions must be the original text you actually saw; DO NOT fabricate anything!!!
4. Ensure the objectivity and specificity of the questions.
4.1 A question is objective and specific if it has an objective, definitive answer. For example: "What was X's revenue rate in 2025?", "What the standard numbers for X?", "When was X included the National Patent Industrialization Demonstration Enterprise Cultivation Pool?"
4.2 A question is *not* objective and specific if the answer is open to reasonable interpre- tation. Avoid questions like: "What are the advantages of X?", "What are the main goals of X?", "What aspects does X focus on?", "Why does X do Y?"
4.3 question is *not* objective and specific if multiple non-equivalent answers could be considered accurate. Avoid questions like: "List the key measures of X."
4.4 A question *not* objective and specific it is overview/summary nature. Avoid the research difficulties in X?"
questions like: "What was reported in X?", "What
5. Maintain rigor and ensure the uniqueness of the answer.
5.1 For all content, considering the current date, version, etc., include specific qualifying statements. Do not ask "What were the sales of X's flagship model?"; add conditions and ask, for example, "What were the sales of X's 2024 flagship model in Mainland China?". Do not ask "How many characters does X have?": ; add conditions and ask, for example, "How many heroes in version 5.2.3 of X?"
5.2 For content specific to particular institution or enterprise, include the institution or enterprise as a condition. Do not ask "On which day are investment meetings held each quarter?"; add the enterprise condition and ask, for example, "On which day of the quarter does Enterprise X hold its investment meetings?". Do not ask "on the official website": specify which official website.
Do not include specific webpage titles or file names the questions. The answers must not contain phrases like "for specific details, please refer to X link".
C IMPLEMENTATION DETAILS
SFT Training. For supervised fine-tuning (SFT), we use learning rate of 3 x 10-6 with a hatch size of 32. Each training instance is packed to sequence length of 128k tokens. We train the model for a total of epochs. After filtering for correct teacher answers, we retain 1,482 training queries, corresponding to 4,501 trajectories in total. Since our framework multi-agent system, a single query may correspond to multiple trajectories.
RFT Training. For reject-sampling fine-tuning (RFT), use the same learning rate and batch size as in SFT. After filtering for correct responses, the full RFT dataset contains 12,959 trajectories associated with 3,317 queries. After applying difficulty queries and undersampling simpler ones), the final number of trajectories actually used for training 4,467.
sampling (oversampling difficult
16

---

Published as a conference paper at ICLR 2026
UIS-QA
BC-zh
GAIA
FinSearchComp(12273)
Pangu-38B 9.1
12.1
25.2
48/3.4
Pangu-SFT 22.7
30.8
42.7
69.0/5.7
Pangu-RFT 27.3
32.5
50.5
73.0/11.4
Table 3: Performance of each training stage across different benchmarks.
D ABLATION STUDY
In this section, we analyze the contributions UIS-Digger's modules and technical choices. Over- all, the results confirm both the robustness and the effectiveness of our framework in tackling UIS problems.
D.1 PERFORMANCE GAINS FROM SET AND RFT TRAINING
Beyond UIS-Digger's strong performance on UIS-QA, we conduct ablations assess how different training stages contribute to accuracy. As shown in Fig. S performance consistently improves after each stage of SFT and RFT, though with diminishing returns. The most significant gain comes from the SFT stage, supporting our claim that vanilla agents lack awareness of UIS and perform poorly at the outset.
RFT further improves performance by enabling the agent to explore diverse solving strategies and re- inforce successful ones. This finding is encouraging: even under the UIS setting, self-improvement through reinforcement remains effective. Nevertheless, UIS-Digger's absolute accuracy after RFT still unsatisfactory, indicating substantial room for future works. We hypothesize two key limi- tations: (1) distribution gap between synthesized QA pairs and the real test set, which weakens transfer, and (2) sparse supervision from reject sampling, where feedback is based only on final answers, potentially reinforeing low-quality trajectories. We also evaluate our method other benchmarks to assess its generalizability. As shown in Tab. consistent performance gains across various benchmarks are observed. Notably, some benchmarks exhibit even larger improvements than on UIS-QA, validating the broad effectiveness of our SFT and RFT stages.
D.2 BACKBONE MODELS
To disentangle the impact of the backbone LLM from that of the UIS-Digger framework, we com- pare several models (Tab. 4). Both Pangu-38B and Qwen3-32B, when trained under UIS-Digger, achieve high score of 27.3%, demonstrating that the framework and training pipeline generalize across backbones. Similarly, Claude-sonnet-4 reaches 23.6%, showing substantial improvement over its original performance and indicating that UIS-Digger benefits even relatively weaker back- bones.
In contrast, directly deploying GPT-4o as the main LLM leads to a dramatic drop to 8.2%, while the similarly untuned O3 yield 30.9%, which even surpass the tuned small models of Pangu and Qwen3. This finding suggests that raw foundation model capability alone is critical and compatibil- ity with the framework can also significantly affect performance.
For the dual-mode web surfer, we also ablate the choice of VLM used to interpret visual signals. By replacing GPT-4o with Qwen VL-max, UIS-Digges still achieves 25.5%, close to the original 27.3%. This demonstrates that UIS-Digger is robust to different VLM choices, with only minor performance variation.
E
CASE STUDY
This section provides detailed case analyses corresponding to the error categories discussed Sec- Each case (translated into English) illustrates specific mode, detailing the agent's actions.
tion 4.3
17

---

Published as a conference paper at ICLR 2026
Qwen
Table 4: Ablation results on backbone models
Pangu
and VLMs.
227
Soore on UIs
backbone
VLM
UIS-QA
Pangu-38B
GPT-4o
27.3
Pangu-38B
OwenVL-max
25.5
Qwen3-32B
GPT-40
27.3
O3-mini
GPT-4o
7.6
GPT-4o
GPT-4o
8.2
Qwen3-235b-a22b
GPT-4o
13.9
Training
GPT-oss-1206
GPT-4o
19.9
Claude-sonnet-4
GPT-4o
23.6
Figure 5: The UIS-QA score changing curve for UIS-Digger at different stages of training.
03
GPT-4o
30.9
Knowledge Sourcing Error This case demonstrates critical failure in source selection. Despite the presence the correct, authoritative website the search results, the agent committed a knowl- edge sourcing error by opting for third-party source. The selected source compromised by outdated historical data that was irrelevant to the query's specific context. Consequently, the agent produced a highly confident yet incorrect answer. Blue indicates the correct source, red indicates third-party source and orange indicates the outdated information
Question
What committees under the Academic Committee of the Chengdu Giant Panda Base?
www.panda.org. org.cn
Gold Root Website
Chengdu Research Base of Giant Panda Breeding
clas.cas.cn
Browsed Webpage
Chengdu Library and Information Center, Chinese Academy of Sciences
Action
## Observation
Through an in-depth exploration of the website of the Chengdu Docu- mentation and Information Center, Chinese Academy of Sciences, have obtained detailed organizational structure information regarding the Aca- demic Committee of the Information Service Center for the Chengdu Giant Panda Breeding Research Base. The primary sources of information in- clude: 1. Successfully accessed the detailed page for the Academic Com- mittee (http://clas.com http://claas.ca..n../es wwwpw.aw..........caaaa 1m0/zztx2016/llx22200 6/gltx201 6/xswyh2016/ 2. Acquired the complete historical evolution and organizational structure of the Academic Committee from the 1st term (1981) the 8th term (revised) (2019 present)...
## Reflection & Reasoning
have successfully completed the user's task requirements. Through sys- tematic exploration of the website, I located an authoritative official in- formation source and obtained complete organizational structure informa- tion for the Academic Committee of the Information Service Center for the Chengdu Giant Panda Breeding Research Base. Key Findings:
1. The Academic Committee is institution with long history, estab-
lished 1981, spanning over 40 years...
Successful Recall of the Gold Root Website from Internal Knowledge This case demonstrates scenario where the agent successfully leverages its internal knowledge of authoritative domains to navigate directly to the correct (gold root) website, bypassing the need for a potentially noisy web search.
18

---

Published as a conference paper at ICLR 2026
Question
According to the Industrial and Commercial Bank of China's RMB spot exchange rate, what was the buying price of the Euro 2025-05-01, and what was the specific release time?
www.ichc.com.cn
Gold Root Website
Industrial and Commercial Bank of China
LLM Input (Re- ceived from Plan- ner)
Please help me complete the following task. When carrying out the task, please carefully meet the requirements specified in the system prompt. Your task is: <task>Visi the official website of the Industrial and Com- mercial Bank of China (ICBC) and find the euro's RMB spot exchange buying rate for May 1, 2025, along with the specific release time. Please focus checking sections such 'Foreign Exchange Rates' or 'Finan- cial Markets' on the ICBC website to locate the euro/RMB spot exchange buying rate data.</task>
LLM (Tool Call)
Output
"task": "Visit the official website of Industrial and Commercial Bank of China (ICBC) to find the Euro to RMB spot foreign exchange buying rate for May 2025, and the specific release time. Focus on checking sections like Foreign Exchange Rates', 'Financial Markets', related sections on the ICBC website to locate the Euro/RMB spot forex buying price data.", "start url": "https//wwwwicbb.com "max num steps": 20
Failure to Interact with Webpage Elements In this case, the agent successfully navigated to the correct (gold root) website. However, failed to complete the task due to an inability to navigate to the specific page that shows the exhibition hall map.
19

---

Published as a conference paper at ICLR 2026
Question
Which heroes in the game Honor of Kings had the highest number ap- pearances, ban rate, and average kills per game during the 2025 KPL Spring Season, and what were the respective figures?
www.chnmuseum.co Nation Museum of China
Gold Root URL
Search Action
"query_ _ ["Number of restrooms/toilets on B1 floor of National Mu- seum of China", "National Museum of China basement level toilets", "Na- tional Museum China floor plan restrooms")
Search Response
"title": "Partial Toilet Renovation and Upgrading Completed at the Na- tional Museum of China Enhance Visitor Experience", "link":
https//www..chmmuseeee h/wp:///wwmm///////////////////0
372269.shtmll"
"snippet": "Humanized services were the focus of this renovation. Dur- ing the renovation project, additional facilities such as toilet armrests and height-adjustable hand wash basins were added to facilitate the disabled, children, and other groups: small shelves and coat hooks were installed inside the toilet stalls to meet visitors' needs for placing personal items..." "date": "Jul 28, 2025", "position":
Browsed Webpage
Browsed Webpage Content
In terms of functional layout, addressing the restroom distribution issues raised by the audience, this renovation involved swapping the men's and women's 's sections of the ground-floor restroom on the north side. While maintaining the same number of men's stalls, the number of women's stalls was increased. Additionally, the cleaning room was relocated outward, further improving space utilization efficiency...
Gold Wehpage
Specific
https//www..hnmussum.cco
Gold Wehpage mation
Specific Infor-
This the service page the National Museum of China, which contains comprehensive visitor information. Key details include the opening hours, reservation rules, and map displaying the exhibition halls and public fa- cilities (e.g., restrooms).
F LIMITATIONS AND FUTURE WORK
This paper highlights the overlooked problem of UIS, introducing the dedicated benchmark UIS-QA QA and strong baseline method, UIS-Digger. While the results presented above are promising, several limitations remain to be addressed.
First, as shown in Fig. 5 UIS- UIS-Digger continues to improve after RFT training, but the gains are lim- ited. This suggests that despite our careful data generation and filtering pipeline, the synthesized QA distribution may still differ from real-world cases. Moreover, the sparse supervision Segnal-focused solely on final answers restricts the model's ability to distinguish between trajectories that are equally correct but vary in quality.
Second, because websites evolve unpredictably, even carefully chosen time-invariant sources may shift in accessibility. For example, new third-parc party websites might replicate the target information, effectively transforming a UIS case into an IIS one and altering the problem difficulty.
Looking forward, plan enhance UIS- UIS-Digger with more advanced self-improo improvement techniques such as reinforcement learning, and to synthesize higher-quality QA pairs that better reflect the complexity of real-wor world UIS scenarios.
20

---

Published as a conference paper at ICLR 2026
G STATEMENT ON THE USE OF AI
Al techniques were employed solely to assist with language polishing and improving sentence flu- ency during the writing of this paper. All ideas, methods, and experimental results were conceived, designed, and executed entirely by the authors.
H ETHICS STATEMENT
This work involves human annotators in the data collection process. All annotators were compen- sated above the minimum wage specified by the local government. The primary goal of this research is to support the community in advancing UIS-capable agents. To promote transparency and repro- ducibility, the dataset will be open-sourced. No commercial or confidential information is included in the dataset.
21


Reviews:
[{"review_id": "iMEe9jmEQV", "reviewer": "Reviewer_RP5P", "summary": "The paper tackles the interesting task for Unindexed Information Seeking (UIS), which involves answering questions based on vital information that is not directly surfaced by the search engines. The paper addresses this with a new UIS-QA benchmark along with the UIS-Digger approach that has a dual-browsing mode for interactively accessing information. The authors further finetune multiple backbone models using synthetic trajectories and demonstrate that the trained models outperform existing approaches on this task.", "questions": "1) For the SFT and RFT processes, are the trajectories for the Planner, WebSearcher, WebSurfer and FileReader agents separately aggregated and then used for training?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1) The experiments results are impressive, with exploration of the SFT and RFT finetuning approaches\n2) The paper has a comprehensive analysis of the different actions and tool calls used.", "weaknesses": "1) The proposed UIS-QA benchmark is a bit limited in size with only 110 examples (with a split of 84 questions in Chinese and 26 questions in English). The authors should consider expanding the dataset to ~300 instances. Moreover, it would be worthwhile to also report expert human performance on this dataset to give a sense of upperbound. \n\n2) While the proposed UIS-Digger models excel on the UIS-QA dataset, the performance is low on GAIA and BrowseComp-zh datasets, bringing into question the generality of the approach. The authors should also show performance of zero-shot, SFT and RFT variants on the GAIA and BrowseComp-zh datasets to give a sense of whether the finetuning degrades performance on other benchmarks. \n\n3) The paper is missing a lot of important experimental details, particularly in how the SFT and RFT finetuning was done. Moreover, I have concerns about the reproducibility of Section 3.2.1 and Section 3.3 as minimal details have been provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aE0UkoOtD0", "reviewer": "Reviewer_xVJL", "summary": "This paper introduces the concept of \"Unindexed Information Seeking\" (UIS), which refers to information that cannot be directly retrieved through search engine queries and requires deeper interaction with websites (e.g., clicking, form filling, file downloading). The authors formalize the distinction between indexed information (II) - content available in search snippets or one-step crawling - and unindexed information (UI) - everything else requiring interactive navigation. They contribute: (1) UIS-QA, a benchmark of 110 expert-annotated QA pairs designed to test UIS capabilities, (2) empirical evidence showing state-of-the-art agents experience 47-54% performance drops on UIS-QA compared to existing benchmarks like GAIA, and (3) UIS-Digger, a multi-agent system with dual-mode browsing trained via SFT and RFT that achieves 27.27% accuracy, outperforming baselines including those using o3 and GPT-4.", "questions": "1. WebArena and Mind2Web already test interactive navigation. What would be the difference between UIS-QA and them?\n2. What would be the performance of UIS-Digger with the SoTA LLMs (like GPT, Claude) as the backbone?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. **Valuable dataset contribution with clear curation criteria**: The manually created dataset of 110 validated QA pairs that explicitly avoid search engine shortcuts represents a concrete contribution.\n2. **Valuable empirical evaluation**: Testing 13+ baseline systems across multiple categories provides valuable comparisons.\n3. **Detailed failure analysis**: The breakdown of error modes and tool usage evolution across training stages offers useful insights.", "weaknesses": "1. **Insufficient differentiation from existing web agent research and benchmarks**: The paper claims that UIS represents an \"underexplored challenge\" and a \"critical blind spot\" in current agent systems, yet the capabilities required (e.g., interactive navigation, form filling, file downloading, multi-step exploration) are precisely what existing web agents can already do. The action space that UIS-Digger employs (search, crawl, click, scroll, type, download files) already exists in prior systems. Benchmarks like WebArena and Mind2Web also evaluate complex web interactions across diverse websites. The distinction appears to be one of benchmark curation methodology (filtering out search-solvable questions) rather than a discovery of a new research problem requiring novel capabilities.\n2. **Critical missing baselines**: The experimental evaluation contains glaring omissions that prevent assessment of whether UIS is actually an unsolved problem for current systems. Most notably, the paper cites OpenAI Deep Research, Google Gemini Deep Research, and Grok-3 Deep Research in the related work but does not evaluate any of these systems. Recent web agent systems are absent. Without evaluating actual state-of-the-art deep research systems and web agent systems, the paper cannot substantiate its central claim that UIS represents a fundamental limitation or that the proposed 27.27% score represents competitive performance.\n3. **Limited technical novelty**: UIS-Digger employs standard components from existing work: multi-agent architecture with specialized sub-agents; dual-mode browsing switching between textual and visual observation; the tools (search, crawl, browser automation, file readers); and the SFT + RFT training pipeline is a standard approach. More problematically, the system achieves only 27.27% accuracy, barely outperforming the Memento baseline at 25.5% (which was tested in a handicapped configuration without its case bank). This marginal improvement despite task-specific training on synthesized data similar to the test distribution might suggest that the paper has not discovered an effective solution methodology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yYmDrn30ye", "reviewer": "Reviewer_9Jka", "summary": "This paper tackles the underexplored challenge of Unindexed Information Seeking (UIS) — retrieving essential information unavailable through standard search engines. The authors introduce UIS-QA, a benchmark specifically designed to evaluate agent performance on UIS tasks, and propose UIS-Digger, an agent system equipped with enhanced web-interactive tools and trained via sequential supervised and reinforcement fine-tuning. Experiments show that UIS-Digger achieves state-of-the-art results on UIS-QA, though overall accuracy (27.27%) highlights the inherent difficulty of the task. The work is valuable for formalizing UIS as a benchmarkable problem and demonstrating a viable training strategy, but the gains remain modest and practical effectiveness is still limited.", "questions": "1,  Is there any overlap between UIS-QA examples and the pretraining or fine-tuning corpora (e.g., cached webpages or documentation dumps)? Can using DeepSeek to filter out data address this?\n2. Can UIS-Digger transfer to new domains or unseen data environments, or is it overfit to UIS-QA’s structure?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This work identifies an interesting unindexed information-seeking problem, which is practical in reality and still under-explored.\n2. This work introduces a dataset to study this problem and also constructs an agent to solve this challenge correspondingly.", "weaknesses": "1. The scope definition of UIS is not crystally clear. How is “unindexed” information precisely defined? Does it include API-gated, or private web data?\n2. The boundary with traditional search is not super clear as well. How to distinguish UIS tasks from regular information-seeking tasks where the answer is simply poorly ranked or paraphrased online?\n3. This work claims to build a benchmark, but the characteristics of the benchmark are not enough. For example, what fraction of examples involve dynamic pages, paywalls, or databases?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pa0ILwn3v4", "reviewer": "Reviewer_oSEj", "summary": "This paper introduces the problem of Unindexed Information Seeking, where vital information is inaccessible to search engines. \nThe authors propose UIS-QA, a benchmark of 110 expert-annotated questions requiring interactions beyond indexed snippets, and present UIS-Digger, a multi-agent system with dual-mode browsing and file reading capabilities. \nExperiments show that UIS-Digger achieves 27.3% accuracy, outperforming stronger LLMs and existing multi-agent systems on UIS-QA, highlighting the importance of addressing UIS for real-world research agents.", "questions": "- How do you ensure UIS-QA’s representativeness given its small scale? Though 100+ samples are common, how do you ensure that diversity?\n- How sensitive is UIS-Digger to backbone choice? Results suggest raw model quality still dominates. Does the framework generalize to weaker LLMs?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- It identifies a novel and important gap that existing benchmarks overlook, and proposes the first so-called UIS benchmark with carefully validated data.\n- Presents a multi-agent architecture with dual-mode browsing and file parsing.\n- Strong empirical evaluation with comparisons against diverse baselines.\n- Detailed error analysis and ablation studies provide useful insights.", "weaknesses": "- Absolute performance on UIS-QA remains very low (27.27%), raising concerns about practical utility.\n- Training relies heavily on synthetic QA pairs, which may not fully reflect real-world UIS complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)