Tools are under-documented: Simple Document Expansion Boosts Tool Retrieval
Authors: Xuan Lu
Haohang Huang
Rui Meng
Yaohui Jin
Wenjun Zeng
Xiaoyu Shen
Venue: ICLR 2026 Poster (2026)
Keywords: tool retrieval
information retrieval
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce Tool-DE, a document-expansion benchmark with two tailored models (Tool-Embed, Tool-Rank) that achieve state-of-the-art tool retrieval and provide a foundation for future research.

Abstract: Large Language Models (LLMs) have recently demonstrated strong capabilities in tool use, yet progress in tool retrieval remains hindered by incomplete and heterogeneous tool documentation. To address this challenge, we introduce Tool-REX, a new benchmark and framework that systematically enriches tool documentation with structured fields to enable more effective tool retrieval,  together with two dedicated models, Tool-Embed and Tool-Rank. We design a scalable document expansion pipeline that leverages both open- and closed-source LLMs to generate, validate, and refine enriched tool profiles at low cost, producing large-scale corpora with 50k instances for 
embedding-based retrievers and 200k for rerankers. On top of this data, we develop two models specifically tailored for tool retrieval: Tool-Embed, a dense retriever, and Tool-Rank, an LLM-based reranker. Extensive experiments on ToolRet and Tool-REX demonstrate that document expansion substantially improves retrieval performance, with Tool-Embed and Tool-Rank achieving new state-of-the-art results on both benchmarks. We further analyze the contribution of individual fields to retrieval effectiveness, as well as the broader impact of document expansion on both training and evaluation. Overall, our findings highlight both the promise and limitations of LLM-driven document expansion, positioning \textsc{Tool-REX}, along with the proposed Tool-Embed and Tool-Rank, as a foundation for future research in tool retrieval.

Published as a conference paper at ICLR 2026
TOOLS ARE UNDER-DOCUMENTED. SIMPLE DOCU- MENT EXPANSION BOOSTS TOOL RETRIEVAL
Xuan Lu Haohang Huang;: Rui Mengi Yaohui Jin', Wenjun Zeng Xiaoyu Shen
Shhnngaai Jiao Tong University
Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative Institute of Digital Twin, Eastern Institute of Technology, Ningbo
lux199 9970 70 jtu.. etu.juuuuucco en xyshen@eitech. edu.co cn
ABSTRACT
Large Language Models (LLMs) have recently demonstrated strong capabilities in tool use, yet progress in tool retrieval remains hindered by incomplete and het- erogeneous tool documentation. To address this challenge, we introduce TOOL- REX, a new benchmark and framework that systematically enriches tool docu- mentation with structured fields to enable more effective tool retrieval, together with two dedicated models, Tool-Embed and Tool-Rank. We design scal- able document expansion pipeline that leverages both open- and closed-source LLMs generate, validate, and refine enriched tool profiles low cost, pro- ducing large-scale corpora with 50k instances for umbedding-basse retrievers and 200k for rerankers. On top of this data, we develop two models specifically tai- lored for tool retrieval: Tool- -mobd,, dense retriever, and Took-Rank, an LLM- based reranker. Extensive experiments on ToolRet and TOOL-REX demonstrate that document expansion substantially improves retrieval performance, with Tool- Embed and Tool-Rank achieving new sfate-of-the-aa results on both benchmarks. We further analyze the contribution of individual fields to retrieval effectiveness, as well as the broader impact of document expansion on both training and eval- uation. Overall, findings highlight both the promise and limitations of LLM- driven document expansion, positioning TOOL-REX, along with the proposed Tool-Embed and Tool-Rank, foundation for future research in tool retrieval.
INTRODUCTION
Tool use has emerged critical capability of Large Language Models (LLMs), allowing them to interact with external tools and APIs to accomplish complex real-world tasks (Hsieh et al. Qin et al., 2023 Liu et al. 20024a). This paradigm, often referred
2023
tool learning (Qin al.
2024
or tool-augmented agents (Wang et al. 2024), unleashes the power of LLMs to access real-time fac- tual knowledge, perform complex computations, and interact with a vast ecosystem of applications. Within this paradigm, tool selection is particularly crucial. Specifically, tool retrieval- the task of identifying the most relevant tools from large repositories in response user queries-see serves as the gateway to effective tool use. As the number and diversity of available tools grow to thousands of APIs, the challenge of retrieving the right tool at the right time has become major bottleneck, often hindering reliable utilization.
To assess and improve this capability, several dedicated benchmarks have been developed in re- cent years, such as ToolBench (Qin et al. 2023), ToolACE (Liu et al.) 2024a), MetaTool (Huang et al.) (004c), and ToolRet (Shi et al.) 20256). While these benchmarks have advanced research, they have also highlighted fundamental challenge: tool retrieval suffers from persistent semantic gap between user queries and tool documentation. Queries are often ambiguous, under-specified, phrased in ways that do not align with the formal, technical descriptions of tools. This "lower
* "qqual contribution
Working at Google Cloud AT Research. & Corresponding author. We release our code httpt:///giiiieeeee www./...o//.///////////.. REX

---

Published as a conference paper at ICLR 2026
semantic overlap" places heavy demands retrieval models, which must bridge the gap between user intent and tool uunctionlity-particulaaly for dustrf-distiibution (OOD) queries with diverse or overlapping phrasing. To narrow this gap, the community has explored range of query/docu- ment expansion and augmentation strategies. For example, Re-Invoke (Chen et al. 2024b) augments tool documents with L.LM-generated pseudo-queries and extracts latent intents; EASYTOOL (Yuan iet al., (2025) restructures verbose tool documentation into concise, standardized forms; MassTool (Lin et al. 2025 formulates retrieval as a multi-task search problem combining tool-use detection and retrieval, and ScaleMCP (Lumer et al. 2025) explores dynamically weighting different doc- ument segments to produce more informative embeddings. However, these approaches primarily work around the problem of poor documentation rather than fixing it at the source.
We argue that these efforts, while valuable, overlook the root cause of poor performance: the tool documentation itself is inherently flawed, suffering from both a lack of standardization and missing critical information. This lack of standardization is critical issue; in the ToolRet (Shi et al. (025b) dataset, for example, we identify at least seven different phrasings for the same type of function (see Appendix A.2). introducing ambiguity that directly complicates retrieval. Furthermore, the docu- mentation often incomplete, omitting crucial context such clear "when-to-use" scenarios and operational limitations. This forces LLMs to guess, often leading to the generation of invalid param- eters and subsequent tool execution failures (Yuan et al. 2025). This foundational data problem has created a performance ceiling for existing models and, more importantly, has stifled the development of powerful, dedicated tool retrievers and rerankers.
To address this data-centric challenge, we introduce TOOL-REX (Tool-Retrieval with EXpansion), new benchmark and framework for systematically enriching tool documentation. We develop low-cost, LLM-based pipeline that augments raw documentation with structured, high-value fields including function description, when-to-use, limitations, and tags. The value of this enriched data is twofold. First, it directly boosts the performance of existing baseline models. Second, it en- ables the creation of large-scale training corpora, from which we build two specialized models: Tool-Embed, a dense retriever trained on 50k examples, and Tool-Rank, a reranker trained on 200k examples. On TOO,-REX, our models achieve 56.44 N NDGG@1,, 67,81 Recall@ 10, and 56.60 Completeness@@ 10, corresponding improvements of +10.23, +10.29, and +9.08 over the MTEB SoTA open-source model
In summary, contributions
We design a low-cost tool document expansion pipeline that systematically enriches tool docu- mentation with structured fields (e.g., function description, when-to-use, limitations, and tags), improving completeness, readability, and utility for both retrieval and reranking.
We introduce Toor. the first benchmark dedicated to document-expanded tool retrieval, built upon 35 well-established tool-use datasets. Alongside the benchmark, we release two large-scale training corpora: Mhol-Embed-Treee (50k examples for retrievers) and Tool-Rank- Train (200k examples for rerankers).
We develop two dedicated models, Tool-Embed (retriever) and Tool-Rank (reranker), which establish new state-of tifee-ofthhe of-the-art performance TOoL-REX.
Finally, our work includes systematic analysis of document expansion's impact on tool re- trieval, providing insights for building the next generation of accurate and efficient tool selec- tion systems.
DOCUMENT EXPANSION FOR TOOL RETRIEVAL
In this section, we introduce TOOL-REX together with two dedicated models. We first discuss the limitations of existing tool documentation in 2.2 then we provide an overview of its construction pipeline in [2.3] and analyze the impact of document expansion fields on retrieval performance in $ 2.44 finally present two retrieval models specifically developed for this task: Tool-Embed and Tool-Rank in 2.5
AAs of September 20, 2025, the sfate-of-the the---ar open-source model on the MTEB leaderboard is Twen3-Rmbeddd ng- 88 (see MTEB leaderboard: Teaderboard).
co/ http//paccssmmeee mmteb/
2

---

Published as a conference paper at ICLR 2026
2.1 DATASET COLLECTION
TOOLRET(Sh) et al. 2025b) serves as the foundation of our dataset construction, as it is currently the most comprehensive benchmark for tool retrieval. consolidates 35 publicly available tool- use datasets, including ToolBench((U) (Qin et al. (023). APIBank (Li et al. (203a)). MetaTool (Huang al. 2024c), and TaskBench, resulting in 7,615 retrieval tasks over 43,215 unique tools. Building upon this unified resource, Tool-REX preserves the original dataset coverage and categorization. Following TEOLRET, all tools are grouped into three categories:
Web APIs: 36,978 tools / 4,916 tasks
Code functions: 3,794 tools / 9550 tasks
Customized applications: 2,443 tools 1.749 tasks
This organization enables controlled evaluation across both in-domain (Web) and out-of cut-of-distriie de-distribution (OOD) settings, as our training data sourced exclusively from APIGen (Liu et al.] (024b) and ToolBench (Qin et al. 2023) (Web category), ensuring strict separation between training and evalu- ation without any data leakage, while the test sets additionally include Code and Customized tools. Further details of datasets are provided in Appendix A.
2.2 LIMITATIONS OF EXISTING TooL DOCUMENTATION
This breadth of Tool-Ret also introduces substantial heterogeneity across tool documents. shown in Appendix A.2 the same function can be described in up distinct formulations across sources, yielding hetcrogeneous- and often inconsistent- mnonsistent-rcepeeecttiiio of semantically equiva- lent functionality.
Field Coverage across TOOLRET Datasets
name
description
parameters
responses
when to use
linitations
splienk
apigeni
relerstcchain hain
revens
ratterch
Leval diatog
Leval stop
Peeehennc dai
boulACE
ruelaaru
tooibmoct
Intern
conleyes
metitools Enser
woteztoff, music
anzttootl, sport
sostscolls trusto
Cog enttornis Weath
craft Matit oig
craft, Bibmay
craft vite
gostio, huppg
gertia, pytor
gartia tarstar
tooling
appleecc
goidoosi
metatonl whtch
mutbbncch hugp
go
tanébernll, nut
tool horrest
Lasistpara
tootbarch sam
Ukbb
Customized
Figure 1: Field coverage across the 35 datasets in TOOLRET.
Beyond inconsistency, many tool documents are incomplete or underspecified. Fig. T shows the 35 datasets that constitute the TooLRET benchmark differ widely in field coverage: some provide rel- atively rich documentation, while others omit essential components. For example, certain datasets lack critical attributes such as the description field (e.g., MNMS), leaving tools without even a basic textual summary of their functionality. More generally, we observe substantial variation in granularity: some entries include exhaustive parameter lists or implementation notes but omit the tool's operational intent, whereas others present only a broad purpose without clarifying inputs/out- puts, preconditions, limitations, or usage contexts.
2.3 PROCESS OF TOOL DOCUMENT EXPANSION
TOOL-REX is built upon TOOLRET, which unifies 35 established tool-use datasets categorized into three domains: Web, Code, and Customized. We provide detailed list of the datasets and their sources in the Appendix A.1
We construct TOOL-REX through four-stage pipeline: expansion, judgement, refinement,and hu- man validation, as illustrated Fig. 2 This design balances cost and quality by combining efficient mid-sized models with stronger models for validation and refinement.
3

---

Published as a conference paper at ICLR 2026
Determine whe it is valid
Generate tool prafile
Valid tool profile
P-
Funccion:.. Tagsi.. When uu.
Step
1Is the content empty? 2.Is the format JSON? D.Ds the confent come from the tool itseif?
Step
Limitation Example usogei.)
Original tool documents
New tool documents
Step 3
Invalid tool profile
Figure 2: Document expansion pipeline for constructing TOOL-REX.
Step 1: Expansion. We use Qwen3-32B, a model with strong instruction-following capabil- ity, to expand raw tool documents into structured profiles, During generation, we enable its reasoning mode to improve the quality and consistency of outputs. Given the original docu- mentation dorigimal as input, the model produces an expanded profile dpruile containing the fields weeetiiondessrpttiiii
tags, then_to_use, 1imi tations, and example _usaae. Among them, funct sennctiiineeccppttiio and tags are always required, while the other three fields are generated only explicitly supported by the documentation. The expansion is query- independent and strictly grounded in the tool doc, with unsupported fields omitted. Specifi- cally, function description summarizes the tool's core functionality; tags capture key concepts categories retrieval keywords; when to use specifies realistic usage scenar- ios; limi tat ions records constraints such as input/output limits or domain restrictions; and gaample.usage illustrates API calls or queries derivable from the documentation. These structured fields enrich tool docs with semantically precise content, facilitating more effective query-document alignment in retrieval. These structured fields are then encapsulated into tool-profile object dprolite which is merged with the original documentation doriginal to form the expanded document:
dexpansion doriginal U dpeotile
The prompt used for generating document expansions is provided in Appendix B.1
Step 2: Judgement. We employ L.L.M-3.1--7OB to verify the quality of expanded profiles. Before LLM-based judgement, we first apply rule-based checks to ensure that the generated tcol profile is non-empty and conforms to valid JSON format. Subsequently. LLaMa-3.1 70B serves as a semantic judge to assess whether each expanded field is faithful the original tool documentation, thereby ensuring that no hallucinated or unsupported content is introduced. The prompt used for judgement is provided Appendix B.2
Step 3: Refinement. For the small fraction of cases that fail Step (about 1.5% of the corpus, roughly 600 examples), we apply refinement process using GPT-4o, a stronger closed-source model with superior reasoming and consistency capabilities. We reuse the same prompt as in Step to regenerate the expansions, ensuring that the outputs remain strictly grounded in the original tool documentation.
This three-stage process enables efficient and low-cost document expansion, forming the basis of the TOOL-REX benchmark. Examples of generated document expansion are listed Appendix [D]
To verify the quality of the dataset, we conducted sample-based human evaluation to confirm the reliability of the pipeline. We randomly inspected 100 regenerated profiles and checked for faith- fulness, absence of hallucinated fields, and semantic consistency. As detailed in Appendix C all sampled cases were validated by annotators, confirming the robustness our refinement pipeline.
We further investigate the contribution of each field in Section 2.4 The ablation sults show that removing when.to.ust yields better performance. Based on this find- ing, we exclude example.usagy from the expanded profiles used for retrieval, and retain and tags. Table reppots the av-
function ion description when to use, erage token length of tool documents before and after adding the tool profi le field.
4

---

Published as a conference paper at ICLR 2026
Average token length across domains. We report the original document length, the tool profile length, and the expanded document length.
Table I:
Class code
Original Doc
ToolProfile
Expanded Doc
167.62
51.81
219.43
customized
71.55
43.79
115.34
web
156.00
42.05
198.05
Avg.
131.72
45.89
177.61
Training set construction. Leveraging the outputs from Stage and Stage 2, we construct two large-scale training datasets: approximately 50k instances for training embedding- and 200k instances for training rerankers. Importantly, both training sets are derived exclusively from APIGen (Liu et al. 2024b) and ToolBench (Qin et al., 2023). which belong to the Web cat- egory. Consequently, the models are trained only Web-based tools, while the evaluation bench- marks additionally include Code and Customized tools that never appear in the training distribution. This setup establishes a genuine out-of-distribution (OOD) evaluation that reflects real-world de- ployment scenarios, where available tools evolve over time and newly emerging categories must be handled without retraining.
retrievers
2.4 IMPACT GENERATED FIELDS TOOL RETRIEVAL
Introducing newly generated fields into tool documents is not universally beneficial: longer inputs can dilute salient signals and incur truncation under token budgets retrievers. To quantify these effects, we evaluate two complementary retrievers-BMM (sparse, keyword overlap) and Qwen3- Embeddings-8F (dense, cemantic)-under two protocols using standard metries nDCG@ 10.
When-to-use
Expansion
Wher-to-use
Expansion
Add one Onieuul
Add one Oni-ccul
Tegs
Tega
Runction
Liitatiess
Umitations
(a) Add-one VS. One-out (GritLM)
(b) Add-one VS. One-out (BM25s)
Figure 3: Impact of individual fields on retrieval: example-usage usage brings minimal gains in gritim and bm25s, and even hurts performance in gritlm, while function and when-to-use contribute relatively larger improvements.
Protocols. Add-Oni Ond-One: starting from the original document, we add one generated field at time from (funct on wonndescriii i when_to_use, 1 mimi tat ons, tags, example exampl..usaee and measure performance deltas. One-Out: starting from the fully expanded document, we remove one field at a time and re-evaluate.
We visualize the effect of each field under two protocols for both dense retriever (GritLM; Fig. Ba) and a sparse retriever (BM25; Fig. 3b). In the plots, the gray dashed ring denotes the original baseline, the blue dashed ring denotes full expansion, the green polygon traces Add-one, and the orange polygon traces One-out. Two consistent observations emerge. First, full expan- sion is not uniformly optimal: removing example le usage in the One-out protocol yields higher
5

---

Published as a conference paper at ICLR 2026
N@10 than keeping it. Second, exwmple.usageee provides the smallest (often negative) gains in Add-one. In contrast, suncticn.description. and tags are cuutral-to-positive under Add-one, and their removal produces noticeable drops in One-out. Guided by these findings, exclude example usage from the expanded profiles and retain funct on descr pt on, when_to_use, limi tations, and tags for retrieval. Appendix Appendix[Dfurtherr further provides case studies illustrating how expansion improves tool documentation and, in turn, enhances retrieval perfor- mance.
2.5 ENHANCED MODELS FROM ENRICHED DOCUMENTS
We leverage the synthesized training corpora described in $ 2.3 to train our dedicated retriever Tool- Embed and reranker Tool-Rank. All experiments conducted two NVIDIA A100 GPUs (80GB each). The training procedures for both components are detailed below.
Tool-Embed. For retriever training. we adopt Qwen3-Embeddingg0.6 and Owen3-EEmeddiing 4B as the base model. Training is performed using the ms ms-swift framework with the InfoNCE loss. For each query-tool pair in the 50k training dataset, we randomly sample tools from other queries as negatives. The model is trained for one full epoch with full-parameter tuning using DeepSpeed ZcRO-3.
Tool-Rank. For reranking. we build Qwen using the LLaMB-Factonn framework, fine-tuned with cross-entropy objective. employ LoRA parameter-efficiet adap- tation, with rank = 32, 64, and dropout 0.1. Training is conducted for one epoch the 200k reranking dataset. At the evaluation phrase, Tool-Rank takes as input a query-tool document pair and is prompted to output whether the document is relevant. A single forward pass yields logits for the tokens true and false, which are normalized into probabilities:
exp(Cerso)
Prrelevant g, d) exp(Lerue) + exp(Eraise)
where L.ue and Leulse denote the logits of the corresponding tokens. This probability is used the reranking score. The prompt template used for reranking provided in Appendix B.3
To further evaluate the impact of document expansion tool retrieval, used the same datasets but removed the expansion field tool frofi i le. With identical training configurations, we trained Tool-Eameeaaiiginal and Tool-Rank counterparts to Tool-Embed and Bool-Rank.
EXPERIMENTS
This section first presents the experimental settings in $ 3.1 followed by the main results in 3.2 which include overall comparisons of retrieval models and rerankers.
3.1 EXPERIMENTAL SETUP
All experiments are conducted on two NVIDIA A100 GPUs with 80GB memory each. We use the TOOL-REX benchmark described in and ToolRet (Shi al. 2025b). For reranking experiments, we take the retrieval results from the best-performing retriever in the first stage as the initial set.
Baselines We evaluate Gool-Embed-0.66/// and -onk-Ra together with their non- expansion trained counterparts TTool-Embed,niee and Tool- (ool-Rannonginaa) Also, we evaluate 12 rep- resentative retrieval and reranking models on Tool-REX:
Sparse retriever: BM25s Lu 2024)
Dense retriever: GritLM-7B MMuennighoff al. 2024). NV-Embed-vl (Lee al.
2024) nee-Qwen2-1.5B-instee
-istttal-7--instiee (Wang et al. 2023a), Qwenš-Embeeddng series (Zhang et al. 2025): Beeenn--mmedding--.... Qwen3-
(Li et al. 2023b).
Embedding-4B, Qwenn-Emmbeddinng-333
httees///fthhbb.ccoo///eelssssss ft

---

Published as conference paper at ICLR 2026
LLM-based reranking models: jina-reranker-md mm (Sturua al. 2024).
bge-reranker-Ve
gemma (Chen et al.) 2024a), Reranker--B Zhang et al. 2025)
Metries We adopt three widely used IR metrics evaluate tool retrieval performance: (i) NDCG@K K (N K). which considers both the relevance of retrieved tools and their ranking posi- tions; (ii) R (R@K), which measures the proportion of target tools successfully retrieved within the top-K results; and (iii) Completeness@IK (C@K) (Qu et al. 2024), which specifically target tools are included the top-K
assesses retrieval completeness by assigning C&K results and 0 otherwise.
Table 2: Performance comparison of different retrieval models ToolRet and TOOL-REX. N@ 10, @@ 10, and C@ 10 denote NDCG@ 10, Recall @ 10, and Comprehensivenessee 10, respectively.
Web
Code
Customized
Avg.
Model
NØ 10
R@10
cØ10
N R@10 CØ10
N@ 10
R@
C@10 10
N
10
R 10
CØ 10
ToolRet
BM25s
34.13
22.80
41.90
56.47
55.36
41.16
38.90
36.41
46,40
39.
Griil M-78
36.58
46.01
27.65
41.26
53.81
52.07
45.55
54.01
41.40
41.13
51.28
40.37
NV-Embed-v1
31.51
40.52
26.74
47.92
62.07
59.60
48.70
57.69
43.88
42.71
53.43
43.41
gte- 10en2-1..B-instruu
36.73
46,39
28.24
40.56
53.08
51.37
46.52
42.10
41.27
51.62
40.57
e5 emistral-bbiinstee
32.59
42.17
27.51
44.05
57.43
55.09
43.42
50.69
39.16
40.02
50.10
40.59
Owen3- Swen3-Embeddiigg 0.6B
-..6
38.22
46.17
28.47
48.60
62.29
60.06
42.58
49.93
40.38
43.13
52.80
42.97
40.90
50.14
31.97
53.56
71.12
69.61
42.15
50.80
40,22
45.54
57.36
47.27
Qwen3- Bemen---dddddng--
41.61
50.23
32.56
53.43
67.29
43.59
53.13
42.73
46.21
57.52
47.52
Tool Emie---hhednan 0.6B
38.57
48.05
32,39
50.11
62.22
60.52
51.77
60.17
46.28
46.82
56.81
46.40
40.87
50.61
34.51
54.63
68.62
67.58
52.12
60.77
46.31
49.21
60.00
49.47
Tool-Embee--0.6l (ours)
42.35
52.79
34.95
49.37
64.41
62.50
50.34
60.11
46.98
47.35
59.10
14
Tbol-Embed-48 (ours)
45.74
55.54
37.20
54.40
69.09
67.43
54.38
63.76
48.42
51.51
62.80
51.02
Tool-REX
BM25s
28.46
35.85
24.04
45.16
58.59
57.
44.43
50.08
38.99 28.98
39.351
48.171
40.081
GritLM-7B
28.96
55.94
$1.79
45.82
43.541
54.071
43.571
NV- NV-Embed-v1
31.96
41.12
27.03
48.79
62.96
60.05
48.89
57.91
44.05
43.211
54.001
43.711
pe--Owen--L5B-ine
36.09
46.60
29.21
41.39
53. 10
51.09
47.87
56.12
42.81
41.781
51.941
41.031
E--mistral-bb-instuue
31.35
40.77
27.22
42.04
53.94
43.17
38.85L
49.13
40.091
Bemend--mdddingg -.66
38.87
47.24
29.78
48.62
62.72
60.36
42.40
48.71
38.94
43.301
52.891
43.031
4BB
40.52
48.73
30.59
53.12
68.56
67.12
43.32
51.09
40.41
45.65t
56.13|
46.04]
Qwen3- Brenn--Embedding-BB
50.61
32.78
52.38
67.40
65.46
44.29
46.231
56.83
46.70
Hool-Embed-0061 (ours)
42.41
53.07
35.06
49.89
64.11
62.26
51.99
60.29
46. 12
48. 10
59.161
47.811
Tool-Embed-41 (ours)
46.28
55.93
37.61
55.87
70.48
68.81
54.54
62.97
48.40
52.231
63.131
51.61
Table 3: Performance comparison of different reranking models on Tool-REX. All rerankers use the retrieval outputs of Bbol-Embed-44 the initial candidate set, and rerank the 10p-100 results for each query.
Weh
Code
Customized
Avg.
Model
NØ
R@10
C@10
N@
10
R@10
C@10
N@10
R@10
CØ10
N@
R@10
C@
ToolRet
Tool- Fimbed 4B
40.87
50.61
34.51
54.63
68.62
67.58
52.12
60.77
46.31
49.21
60.00
49.47
+ iina-rerankermm0
43.82
54.10
35.10
52.13
65.88
64.89
51.44
62.49
47.27
49.13
60.82
49.09
geg--rrankee-v2-gemee
43.68
53.17
34.84
52.82
67.26
65.84
51.96
62.53
47.54
49,48
60.99
49.41
Qwen3 R-mm-----akk-----
44,00
53.68
34.09
52.02
67.19
65.12
53,12
62.74
47.80 47.84
49.71
61.20
49.02
+ Baoo-Ran--igaa--t
45.46
56.02
38.66
51.75
68.21
66.35
56.42
63. 19
48.91
51.21
62.47
51.31
Mhol-Rank-4B (ours)
57.21
39.96
53.01
67.57
57.85
49.67
52.40
Tool-REX
Bine--Eimee--4B
46.28
55.93
37.61
55.87
70.48
68.81
54.54
62.97
48.40
52.23
63.13
51.61
+ jina-rerakke--nt
46,90
56.58
38.78
54,82
70.07
69.06
56.07
65.67
50.72
$2.60
64.11
$2.85
+ aige-reranker-22
--gemmm
47.96
57.72
39.38
55.51
71.31
69.15
55.09
63.64
49.61
52.85
64.22
52.71
Tween-RReaanker-11
47.48
58.12
40.84
70.33
64.48
58.66
66.45
52.34
64.97
52.55
Tool-Rank Rank-I (ours)
50.66
61.36
44.45
58.69
74.00
72.05
59.97
68.05
53,29
56.44
67.81
56.64)

---

Published as a conference paper at ICLR 2026
3.2 MAIN RESULTS
Documents Expansion Improve Tool Retrieval To assess the contribution of document ex- pansion, we first compare models trained with expansion against non-expansion counterparts un- der identical settings. As shown in Table 2 relative to the baseline Dween--Embedding-O.. (43.13 N N@10), Tool-Embed 0.6B/4B yield gains of +4.97/+6.69, whereas the non-expansion-traine Tool-Embedarigiial 0.6B/4B improve by +.3.69/3.677. Reranking exhibits the same pattern: non- Opaasion-trainee Tool- Tool-Rankeeignall adds +2.00 in N@10 after reranking the retrievel result from Tool-Emmedorrgiaal- -4B, while the Tool-Rank improves the result of Tool- -B-ed--ee- by +4.21.
We then compare the performance of various retrievers both ToolRet and TOOL-REX, with re- sults summarized in Table 2 Among the sparse models, BM25s shows consistent improvements on TOXL-REX, with notable gains in N@10, R@10, and C@110, demonstrating that document expansion enhances keyword overlap between queries and tool documents. For dense retrievers, observe a similar trend: except for E5-Mistral, which slightly decreases in performance, most models achieve higher scores on TOOL-REX. In particular, GritLM achieves the largest gain in N@10 (+2.41), accompanied by improvements in R@10 and C'@10. For Deven3-Emeddding--- and
we note a small decrease in R@10 (-1.23 and -0.69, respectively), yet both models still show slight improvements in N@10, indicating that expansion helps improve the semantic alignment between queries and tool documents even when recall is marginally reduced. For rerankers, as shown in Fig. 3] when initial results come from Tool---mbedariginnl- on ToolRet, gains the original ToolRet are limited fina-rerankkr-mo: -0.08; bge-reranker--2-gemmns +0.27; Swen3-Reranker-4B3 +0.50), suggesting that under-informative documents constrain reranking; on the +0.37; ggg-reranker--2-gemieee +0.62;
TOOL-REX, the same rerankers benefit markedly Mina-reranker-me m0:
ReeRReeer-a:------ +1.20), with Tool-Rank-4B providing the largest improvement (+4.21, reaching 56.44). Collectively, these results show that expansion improves training effectiveness and downstream retrieval quality: it reinforces lexical signals for sparse models and, crucially, supplies additional semantic context that enables dense retrievers and rerankers to capture finer-graine relevance, yielding larger gains than non-expansion training under the same conditions.
Generalization of Tool-Embed and Tool-Rank As shown in Table 2 and Table 3 both Tool- Embed and Tool-Rank demonstrate strong generalization beyond the expanded setting. First, on the non-expanded Tool-Ret benchmark, our models achieve the best performance and even outperform their counterparts trained the original documentation. This indicates that the enhanced fields in Tool-REX provide effective semantic supervision, enabling models to improve retrieval quality even when evaluated on unmodified documents. Furthermore, the results on the Code and Customized categories confirm OOD generalization. As detailed in Section 2.3 both Tool-Rank-tee train are constructed exclusively from Web-category tools, meaning that Code and Cus- tomized tools never appear during training. Despite this distribution shift, our models consistently surpass all baselines on these unseen categories, demonstrating that they learn transferable semantic representations rather than overfitting to surface patterns of the training domain.
cembed-train and
SoTA Performance of Tool-Embed and Tool-Rank Table 2 shows that our retriever trained on expanded documents, Tool-Embed-4B, achieves state-of-the-art performance on TOOL-REX, reaching N@10 52.23, R@10 63.13, and C@10 51.61. It also surpasses all other re- trieval models on the original ToolRet benchmark. Notably, Tool-Embed-0.66 already outperforms baseline models with up to 8B parameters, demonstrating the effectiveness and robustness of the Tool-Embed Table reports results of rerankers on TOOL-REX. All rerankers operate on the initial retrieval results produced by TOOL-EMBED-4B reordering the top-100 candidates for each query. We observe that rerankers consistently refine retrieval performance, with Tool-Rank-al achieving state-of-the-ar results: N N@10 improves by +4.21 (to 56.44). R@10 by +4.68 (to 67.81), and C@10 by +4.99 (to 56.60), compared with the first-stage retriever. These findings confirm the superior performance of Tool-Embed-4B and Tool-Rank-44 4B on TOOL- -RX, establishing strong basclines for future tool retrieval research.
8

---

Published as a conference paper at ICLR 2026
4 ANALYSIS
4.1 DOCUMENT EXPANSION AS AN EFFECTIVE TRAINING SIGNAL
To isolate the effect of document expansion during training, we construct non-expanded counter- parts of our corpora by removing the expansion field tool profile while keeping the num- ber of query-tool pairs and all hyperparameters unchanged. We then train Tool-Emmedanigima and Tool-Rankusginal on these reduced datasets and compare them with the expansion-traines vari- ants. As summarized in Table 2 expansion yields consistently larger gains for retrieval: relative to Owen3-Emmeeddnn--0.66 (43.13 N @1)), Tool-Embeddargnan00..B3
improve by +3.69/43.67,
whereas expansion-traineed Tool-Embed-0.6B44 reranking (Table 3): Tool-Rankungnal adds +2.00 N@10, while expansion-traie trained Tool-Rank im- when evaluated datasets
achieve +4.97/46.69. A similar pattern holds for
proves by +4.21 1 to 56.44. Notably, these improvements persist whose test documents are not expanded (e.g.. ToolRet), indicating that exposure to richer, standard- ized fields during training enhances generalization to heterogeneous documentation. Conceptually, expansion reduces structural heterogeneity (c.g.. inconsistent field naming and narrative styles) and supplies explicit functional, situational, and constraint signals (funct ion, tags, when_to_use which stabilizes optimization and encourages alignment with task-relevant seman- ties rather than surface cues. Taken together, the evidence supports document expansion a princi- pled and query-agnosee training signal that improves both embedding and reranking models under fixed training budgets.
4.2 DOCUMENT EXPANSION AT EVALUATION TIME: DISCRIMINATIVE GAINS DESPITE SIMILARITY DILUTION
We next examine the role of expansion at evaluation time. To this end, we randomly sampled 100 queries from each of the Web, Code, and Customized tasks (300 total), and for each query selected one positive and one negative candidate. Using the GritLM model, we computed their similarities with and without document expansion. As shown in Fig. 4 the absolute similarity scores of expanded documents are consistently lower than those of their non-expanded counterparts, plausibly due to semantic dilution caused by increased length. Importantly, the reduction asymmetric: positives drop only slightly (Weh: -0.0014, Code: -000022, Customized: -0.0027), whereas negatives show much larger decreases (Web: -0.0055, Code: -0.0038, Customized: -0.0152). This asymmetry yields sharper posi- tive/negative separability and improves relative ordering, which directly benefits retrieval quality. The effect becomes even clearer the rerank- ing stage. When Tool-Embed-4B provides the initial pool, rerankers applied on the original, non-expanded ToolRet yield limited or even neg- ative changes in @0 (c.g.. jina-reranker-mle -0.56; Dgg-reranker--22gemmm- +0.27: Qwen3- Reranker-4B: +0.50), suggesting that under- informative documents constrain reranker util- ity. In contrast, rerankers consistently bene- fit the expanded Tool-REX: jina-reranker- m0 (+0.37), bge-reranker-2-geemnns (+0.62), Bwenn-Reranner-4B (+1.20), with Tool-Rank- 4B achieving the largest gain (+4.21, reach- controlled toggle experi- ment-fixing the retrieval pool while switching between expanded and non-expanded document furth-further confirms that removing expansions sharply attenuates reranker improvements. Taken together, these results demonstrate that expansion serves not only as an effective training signal but also crucial evaluation-time enabler: it reduces semantic dilution asymmetrically, sharpens the separation between positives and negatives for retrievers, and provides the structured semantic hooks that rerankers require for fine-gra finenne relevance judgments.
0.89
Pos ToolRet
Neg ToolRet
Pos Tool-DE
Neg Tool-DE
0.87
Mean Similarity
0.85
0.0014
0.0022
0.0027
0.83
A. 00.5055
A.000038
+0,0152
0.81
0.79
0.77
0.75
web
code
customized
ing 56.44).
Figure 4: Mean similarity scores of positive and negative in TOOL-REX and TOOLRET.

---

Published as a conference paper at ICLR 2026
5
RELATED WORKS
5.1 TOOL RETRIEVAL BENCHMARKS
Tool retrieval aims to identify the most relevant tools from large repositories given a user query. To enable systematic evaluation, several tool-use benchmarks have been proposed. ToolBench (Qin et al. 2023) comprises over 16,000 web APIs crawled from RapidAPI with LLM-generated queries and labeled ground truth tools. APIBank (Li et al.; 2023a) focuses web APIs for personalized ap- plications such as alarm booking and database login. ToolACE (Liu et al. 2024a provides compre- hensive evaluation of tool utilization capabilities across diverse scenarios. Tooa (Huang al.) 2024c) evaluates whether LLMs can correctly decide when and which tools to use. TaskBench (Shen al. 2024) assesses tool-use capabilities across multimedia, daily life, and deep learning domains. T-Eval (Chen et al. 2023) offers line-graines evaluation across multiple aspects including instruction following and planning. RestGPT (Song et al. 2023) focuses on connecting LLMs with RESTful APIs for real-world applications. Building upon these resources, ToolRet (Shi et al., 2025b) ag- gregates over 30 well-established tool-use datasets into unified format, resulting in 7.6k diverse retrieval tasks and a corpus of 43k tools. Unlike existing benchmarks that manually pre-select small sets relevant tools, ToolRet evaluates tool retrieval in realistic scenarios with large-scale toolsets, categorizing tasks into three groups: web APIs, code functions, and customized applications.
5.2 QUERY AND DOCUMENT EXPANSION
Traditional information retrieval has long explored query and document expansion to reduce noise and enhance semantic matching Li et al. (2025). For instance, Query2doe (Wang et al.) 2023b) en- riches query representations by generating pseudo-documents while EnrichIndex (Chen al. 2025) decomposes documents into multiple semantic views such as summaries, purposes, and QA pairs. Recent advances extend these paradigms tool retrieval, aiming to bridge the semantic gap be- tween user requests and formal tool documentation. On the query side, MCP-Zero (Fei et al. 2025) converts natural language inputs into structured tool requests for more accurate alignment, and Re- Invoke (Chen et al. 2024b) incorporates an intent extractor to filter tell-relevant intents from verbose queries. For complex or multi-step tasks, TURA (Zhao et al.] 2025) performs multi-intent query decomposition, while Mass'Tool (Lin et al.) 2025) enriches query representations through search- based user intent modeling with nearest neighbors, improving robustness against -iittt-dittriutti queries. On the document side, EASYTOOL (Yuan al.) 2025) restructures lengthy and incon- sistent tool documentation into concise, standardized instructions with usage examples. In-oke also augments tool documents with synthetic queries during offline indexing, and ToolBench (Qin) eet al. 2023) leverages large-scale LLM prompting to generate diverse instructions for APIs, creating training corpora for retrievers. More advanced strategies enrich documents with topies and intents, as in Advanced RAG-Too Fusion, or dynamically weight heterogeneous segments (e.g.. name, de- scription, synthetic queries) form discriminative embeddings, as proposed in ScaleMCP (Lumer iet al. 2025). PLUTO (Huang et al. 2024b) extends retrieval beyond static expansion by introduc- ing Plan-and- and Edit-and- Ground paradigms, combining neural retrieval with L.LM-based task decomposition and scenario-driven document edits to enhance tool utilization. Despite these advances, retrieval quality remains a bottleneck end-to-end tool use. Multiple studies show that retrieval errors propagate and significantly degrade downstream agent performance, underscoring the centrality retrieval as a foundation for scalable tool-augmented systems.
6 CONCLUSION
We present TOOL-REX, the first benchmark and framework dedicated to document-expandee tool retrieval. Through scalable LLLM- expansion pipeline, we enrich the raw tool documentation with structured fields, producing more complete and consistent resources for training and evaluation. Building on these resources, we introduce Tool-Embed and Tool-Rank which achieve state-of- art results on both ToolRet and TOXL-REX. Our analyses show that document expansion not only reduces structural heterogeneity during training but also sharpens positive negattiv separability at evaluation, thereby improving reranker effectiveness. We hope TOO,-REX, together with Tool- Embed and Tool-Rank will serve as a foundation for future research in tool retrieval and tool- augmented intelligence.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Tool-REX studies LLM-based document expansion for tool retrieval. Our data are collected from publicly available tool-use corpora/APIs, and we do not include sensitive, personal, proprietary information. Since LLM-ggnnrrted expansions may contain inaccuracies, we mitigate this risk via automatic validation and spot-checked human inspection. We encourage responsible use and verifi- cation of expanded tool descriptions in downstream systems.
PRODUCIBILITY STATEMENT
We release the Tool-REX benchmark with both original and expanded tool documents, and we will provide code/scripts for preprocessing, training, and evaluation to reproduce all reported results. Implementation details (models, hyperparameters, hardware, and evaluation protocols) are specified in the main paper and supplementary material.
ACKNOWLEDGEMENTS
We thank EIT and IDT High Performance Computing Center for providing computational resources for this project. This work was supported by the 2035 Key Research and Development Program of Ningbo City under Grant No. No.20252034.
REFERENCES
Jianly Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3-embedding: Multi-lingual, fulti-functionaaiiyy culltiggranllarity text embeddings through self-knowledge dis- tillation, 2024a.
Peter Baile Chen, Tomer Wolfson, Michael Cafarella, and Dan Roth. Enrichindex: Using Ilms to enrich retrieval indices offline. arXiv preprint arXiv:2504.03598, 2025.
Yanfei Chen, Jinsung Yoon, Devendra Singh Sachan, Qingze Wang, Vincent Cohen-Addad, Mo- hammadhossein Bateni, Chen-Yo Yu Lee, and Tomas Pfister. Re-invoke: Tool invocation rewrit- ing for zero-shot tool retrieval. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 4705- 4726, Miami, Florida, USA, November 2024b. Association for Computational Linguistics. doi: 10. 10.118633/1/2224.11 1il.:666//112222..mmnn-----mm mani.inseemm...270... URL
Eindings emm.e.mmmppp2.. 270/
Zehui Chen, Weihua Du, Wenwei Zhang, Kuikun Liu, Jiangning Liu, Miao Zheng, Jingming Zhuo, Songyang Zhang, Dahua Lin, Kai Chen, et al. T-eval: Evaluating the tool utilization capability of large language models step by step. arXiv preprint wrXiv:2312.14033 2023.
Xiang Fei, Xiawu Zheng, and Hao Feng. Mcp-zero: Proactive toolchain construction for I1m agents from scratch. arXiv preprint arXiv:2506. 00.566000066...... 2025.
Cheng-Yu Yu Hsieh, Si-An Chen, Chun-Liang Li, Yasuhisa Fujii, Alexander Ratner, Chen-Yy Yu Lee, Ranjay Krishna, and Tomas Pfister. Tool documentation enables zero-shot tool-usage with large language models, 2023. URL https:::: wtt/vaiivvv.. http:///aaaiiv.o//a//////// 2308. 00675
Shijue Huang, Wanjun Zhong, Jianqiao Lu, Qi Zhu, Jiahui Gao, Weiwen Liu, Yutai Hou, Xingshan Zeng, Yasheng Wang. Lifeng Shang, et al. Planning, creation, usage: Benchmarking Ilms for comprehensive tool utilization in real-world complex scenarios. arXiv preprint arXiv: 2024a.
Tenghao Huang, Dongwon Jung, Vaibhav Kumar, Mohammad Kachuee, Xiang Li, Puyang Xu, and Muhao Chen. Planning and editing what you retrieve for enhanced tool learning. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Com- putational Linguistics: NAACL 2024, pp. 975-988, Mexico City, Mexico, June 2024b. Associ- ation for Computational Linguistics. doi: 1..18653/v1/2244 www.dfinnnnnnnccle find: ngs naac 66/
URL https:

---

Published as a conference paper at ICLR 2026
Yue Huang, Jiawen Shi, Yuan Li, Chenrui Fan, Siyuan Wu, Qihui Zhang, Yixin Liu, Pan Zhou, Yao Wan, Neil Zhenqiang Gong, et al. Metatool benchmark for large language models: Deciding whether to use tools and which to use. In ICLR, 2024c.
Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catan- zaro, and Wei Ping. Nv-embed: Improved techniques for training Ilms as generalist embedding models. arXiv preprint arXiv:2405. /7228, 2024.
Minghan Li, Xinxuan Lv, Junjie Zou, Tongna Chen, Chao Zhang, Suchao An, Ercong Nie, and Guodong Zhou. Query expansion in the age of trained and large language models: A com- prehensive survey, 2025. btRt::tt:://an
http://wwxv..////a...a///// 2atpas/2299.0. 07794.
Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. API-bank: A comprehensive benchmark for tool-augmented LLMs. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 3102-3116, Singapore, December 2023a. Association for Computational Linguistics. doi: 10. 005/5//11202.emmill main. URL 187/
https: //ap: anthe ogy. org/ 2023. emnlp-mmmmmmmm. ma i
Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308. 032887 2023b.
Jianghao Lin, Xinyuan Wang, Xinyi Dai, Menghui Zhu, Bo Chen, Ruiming Tang, Yong Yu, and Weinan Zhang. Masstool: A multi-task search-based tool retrieval framework for large language models, 2025. (RRhhtps::/ wttp:/a.vv......./a///// absp:///b////// 2507. 00487
Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, et al. Toolace: Winning the points of Ilm function calling. arXiv preprint Ttt::249900092000 2024a.
Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh Murthy, Liangwei Yang, Silvio Savarese, Juan Carlos Niebles, Huan Wang, Shelby Heinecke, and Caiming Xiong. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets, 2024b. URL |tttps://arxiv.
stg/abs// /b////ba/4.... 226.6 18518
Xing Han Lù. Bm25s: Orders of magnitude faster lexical search via eager sparse scoring. arXiv preprint arXiv:2407.0366 2024.
Xuan Lu, Haohang Huang, Rui Meng, Yaohui Jin, Wenjun Zeng, and Xiaoyu Shen. Rethinking rea- soning in document ranking: Why chain-of-thought falls short. arXiv preprint arXiv:2510.08985 2025a.
Xuan Lu, Sifan Liu, Bochao Yin, Yongqi Li, Xinghao Chen, Hui Su, Yaohui Jin, Wenjun Zeng. and Xiaoyu Shen. MultiConIR: Towards multi-condition information retrieval. In Chris- tos Christodoulopoulos. Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 13471 Suzhou, China, November 2025b. Association for Computational Linguistics. ISBN 979-8-891766-35-77 doi: 10.18653/v1/2025 http:s/w//2225..innnneeemmmmmm findings-emmlp.226. URL http:: httpp:://claantilllgg. wwwp:oyy..rg..000..
F Findings emmlp. 7267
Elias Lumer, Anmol Gulati, Vamse Kumar Subbiah, Pradeep Honaganahalli Basavaraju, and James A. Burke. Scalemep: Dynamic and auto-synchronizzing model context protocol tools for Ilm agents, 2025.
06666
Zixian Ma, Weikai Huang, Jieyu Zhang, Tanmay Gupta, and Ranjay Krishna. m & m's: A bench- mark to evaluate tool-use for m ulti-step m ulti-modal tasks. In European Conference on Com- puter Vision, pp. 18-34. Springer, 2024.
Niklas Muennighoff, SU Hongjin, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. Generative representational instruction tuning. In The Thirteenth International Conference on Learning Representations, 2024.
12

---

Published as a conference paper at ICLR 2026
Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. Gorilla: Large language model connected with massive apis. Advances in Neural Information Processing Systems, 37:126544 126565, 2024.
Cheng Qian, Chenyan Xiong, Zhenghao Liu, and Zhiyuan Liu, Toolink: Linking toolkit creation and using through -hain-of-solving on open-source model. arXiv preprint arXiv:2310.05155, 2023.
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023. URL https: http://axivvvvv. org/ / 2377.167899
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xuanhe Zhou, Yufei Huang. Chaojun Xiao, et al. Tool learning with foundation models. ACM Computing Surveys, 57(():1-40, 2024.
Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. Towards completeness-orieneee tool retrieval for large language models. In Pro- ceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 1930-1940, 2024.
Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J Maddison, and Tatsunori Hashimoto. Identifying the risks of Im agents with an Im-emulated sandbox. arXiv preprint arXiv:2309.188) 2023.
Yongliang Shen, Kaitao Song, Xu Tan, Wenqi Zhang, Kan Ren, Siyu Yuan, Weiming Lu, Dongsheng Li, and Yueting Zhuang. Taskbench: Benchmarking large language models for task automation. Advances in Neural Information Processing Systems, 37:4540 4574, 2024.
Zhengliang Shi, Yuhan Wang, Lingyong Yan, Pengjie Ren, Shuaiqiang Wang, Dawei Yin, and Zhaochun Ren. Retrieval models aren't tool-savvy: Benchmarking tool retrieval for large lan- guage models. arXiv preprint arXiv:2503.0176; 2025a.
Zhengliang Shi, Yuhan Wang, Lingyong Yan, Pengjie Ren. Shuaiqiang Wang, Dawei Yin, and Zhaochun Ren. Retrieval models aren't tt tool-savvy: Benchmarking tool retrieval for large lan- guage models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: Linguitssonnll ACL 2025, pp. 24497- 24524, Vienna, Austria, July 2025b. Association for Computational Linguistics. ISBN 979-8- 89176-256-5. doi: 10. erg/2025 findi ngs--gg--n acl. 258/
1258. URL hittps://aclass logy.
Yifan Song. Weimin Xiong, Dawei Zhu, Wenhao Wu, Han Qian, Mingbo Song, Hailiang Huang, Cheng Li, Ke Wang, Rong Yao, et al. Restgpt: Connecting large language models with real-world 2023.
restful apis. arXiv preprint arXiv:2236.0666
Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang. Markus Krim- mel. Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, et al. -mneedbeedd v3: Multilingual embeddings with task lora. arXiv preprint arXiv:2409, 10/73, 2024.
Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, Boxi Cao. and Le Sun. Toolal- paca: Generalized tool learning for language models with 3000 simulated cases. arXiv preprint 05x55v23306.05:01, 2023.
Jize Wang, Ma Zerun, Yining Li, Songyang Zhang, Cailian Chen, Kai Chen, and Xinyi Le. Gta: benchmark for general tool agents. Advances in Neural Information Processing Systems, 37: 75749-75790, 2024.
Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Improv- ing text embeddings with large language models. arXiv preprint arXiv:2401.00366 2023a.
Liang Wang. Nan Yang, and Furu Wei. Query2doc: Query expansion with large language models. arXiv preprint 2303v22303.0777 2023h.
13

---

Published as a conference paper at ICLR 2026
Qiantong Xu, Fenglu Hong, Bo Li, Changran Hu, Zhengyu Chen, and Jian Zhang. On the tool manipulation capability of open- open-source large language models. arXiv preprint arXiv:2305. 16504, 2023.
Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. Gpt4tools: Teaching large language model to use tools via self-instruction. Advances in Neural Information Processing Systems, 36:71995-72000 2023.
Junjie Ye, Guanyu Li, Songyang Gao, Caishuang Huang, Yilong Wu, Sixian Li, Xiaoran Fan, Shihan Dou, Qi Zhang, Tao Gui, et al. Tooleyes: Fine-grained evaluation for tool learning capabilities of large language models real-world scenarios. arXiv preprint arXiv:2401. 0.0041,, 2024.
Lifan Yuan, Yangyi Chen, Xingyao Wang, Yi R Fung, Hao Peng, and Heng Ji. Craft: Customizing Ilms by creating and retrieving from specialized toolsets. arXiv preprint 2309::2309./74220 2023.
Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Kan Ren, Dongsheng Li, and Deqing Yang. EASYTOOL: Enhancing LLM-based agents with concise tool instruction. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Pupers), pp. 951-972, Albuquerque, New Mexico, April 2025, Association for Computational Linguistics. ISBN 179---89176-11111 naacl-long.44 44.
doi: 18.18653/v 1/2025. org. 200 www.ennnaacaca............
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 embedding: Advanc- ing text embedding and reranking through foundation models. arXiv preprinl arXiv:2506.05176 2025.
Yinger Zhang, Hui Cai, Xeirui Song. Yicheng Chen, Rui Sun, and Jing Zheng. Reverse chain: A generic-rule for Ilms to master multi-api planning. arXiv preprint arXiv:2310.0447. 2023.
Zhejun Zhao, Yuehu Dong, Alley Liu, Lixue Zheng, Pingsheng Liu, Dongdong Shen, Long Xia, Jiashu Zhao, and Dawei Yin. Tura: auge-nngee unified retrieval agent for ai search, 2025. URL httthttts//axiiii
04604
14

---

Published as a conference paper at ICLR 2026
A DETAILS OF BENCHMARK
A.1 DATASET CATEGORIZATION
TOOL-REX is built upon ToolRet by applying document expansion. The ToolRet benchmark is one of the most comprehensive benchmarks in the field of tool retrieval, integrating wide range of well- established tool-use datasets collected from top conferences and large-scale projects. It divided into three subtasks: (i) Web APIs, which are standardized JSON-based APIs (OpenAPI format) that can invoked via HTTP requests and cover diverse domains such as movies, music, and sports; (ii) Code Functions, which consist of source code functions focusing computational atomic operations (e.g., tensor calculations, calling Hugging Face models, or using PyTorch libraries); and (ii) Customized Apps, which are described in free-form natural language and typically correspond to user-oriented applications such as sending emails or other personalized tasks.
The datasets corresponding these three subtasks are listed below. For detailed descriptions of each dataset, please refer to their original papers as well as the ToolRet (Shi et al. 20256).
Web AutoTools-Food (Shi al. 2025a). RestGPT-TMDE (Song al. 2023). AutoTools-
(Shi et al. 2025a). AutoTools- Weather (Shi et al., 2025a). RestGPI-Spotily (Song et al.
2023), ToolLens (Qu 2023
ootoTTools-Mo Music (Shi et al.
2025a). ToolBench (Qin al.
2024
APlbank (Li et al., (023a).
MetaTool (ToolE) (Huang et al.
2024c Mnms (Ma et al.
2024
Reverse-Chain (Zhang et al. 2023), ToolEyes (Ye et al. (022)) APIGen (Liu al. 2024b). UI-
traT (Huang et al. 2024a). and T-Eval (Chen et al. 2023
Code Gorilla-PyTorch (Patil et al.
2024), Gorilla-Tensor
(Patil et al
2024), Goriila-uuggingPace
(Patil al. 2024), CRAFTTabbWW
(Yuan et al. 2023) CRAFT QA
Yuan et al. 2023 and
CRAFT-Math-Algehrr Yuan et al.
2023
Customized ToolACE (L.iu et al.
2024a). GPT4Tools (Yang al.
2023).
TaskBench (Shen al.,
2024)
ToolAlpaca (Tang et al.
2023).
ToolBench-sam (Xu et al.
2023
ToolEmu (Ruan et al.
2023
and TooLink (Qian et 2023)
A.2
CANONICALIZATION OF GDOL-DOCUMENT FIELDS
To enable consistent cross-dataset comparison, canonicalize tool-document fields by merging different field names with the same semantic meaning into eight unified categories. Table lists the mapping from raw fields observed the datasets the canonical categories used in our analysis.
Table 4: Mapping from raw fields (left) to canonical fields (right). Representative raw names shown; semantically equivalent variants are grouped.
Raw field names (examples)
Canonical field
name, name. for human
name
#unc.description, description, descripttion.tor.hummn. oug.:/oae....auaaaauuuueeeeeeeor functicnality
description
category, category.name, domain parameters, apiarguments, paw.meeerrreeeerss.....o required inputs, optionallargumentss
tags parameters
responses, response, return.data, outputs, result.arguments,, template.responset output method, api call, url, path cxamplc.usage, example.code limitation, is.transactionall performance, pythaon.ewiiroomee...eennrrerrr
responses
function
when-to-use
limitations
doc.arguments
We retain the following eight canonical fields for all visualizations and analyses:
15

---

Published as a conference paper at ICLR 2026
You are given tool decument containing fields such as "name", "descripticn", "parameters", and "responses".
Task: Judge whether the document is sufficiently complete
for retrieval purpeses. Specifically, check:
(1) Does contain a clear functional statement?
(2) Does it provide contextual usage guidance
(e.g., when to use, typical scenarics, or limitations)
Answer only with:
true (if the document is sufficient)
False (if the document is insufficient)
Figure 5: Prompt template used for LLM-based completeness auditing of tool documents.
The identifier of the tool function (e.g., API name, function name); the minimal unit for reference and invocation.
name:
description: A natural-language summary of the tool's capability and scope; the primary carrier of semantic intent for retrieval.
category: The topical or functional domain of the tool (e.g., movies, music, vision, fi- nance); used for coarse routing and filtering.
parameters: The input schema required to call the tool (names, roles, constraints), including variants such as optional/required arguments.
responses: The output or return schema produced by the tool, including response payloads result fields.
method:
The invocation modality or entry point (e.g., HTTP method+URRI/path, function signature, or API call string).
example _ssgee: Illustrative usage examples (queries plus concrete calls/snippets) derived from the documentation.
limitations: Explicit constraints or preconditions noted in the documentation (c.g., trans- actional restrictions, performance notes, environment requirements).
A.3 AUDIT OF INCOMPLETE DOCUMENTATION
To systematically assess incompleteness, we performed an LL-M-based audit. For each domain (Web, Code, Customized), we randomly sampled 100 tool documents (300 total). The LLM was asked to evaluate whether each document contained (i) a clear functional statement and (ii) contextual usage guidance. A document was flagged as incomplete if either was missing. The audit revealed that 41.6% of the sampled documents were incomplete, confirming that a substantial portion of the corpus lacks sufficient semantic cues for retrieval.
We repeated the same procedure after applying document expansion (see Sec. 2.3 and found that the incompleteness rate dropped 23.5%. This demonstrates that expansion substantially improves coverage of key fields and reduces the proportion of underspecified documents.
B PROMPTS FOR CONSTRUCTING TOOL-REX
B.1 DOCUMENTS EXPANSION PROMPT
Fig. shows the prompt used for tool document expansion.
B.2 JUDGEMENT PROMPT
Fig. shows the prompt used for tool document expansion judgement.
16

---

Published as a conference paper at ICLR 2026
B.3 RERANKING PROMPT
Fig. shows the prompt used for tool document reranking. followed by
(Lu et al. 2025a
C
HUMAN ANNOTATOR AND EVALUATION GUIDELINE
To ensure the reliability our dataset, we employed human annotators with relevant technical and linguistic backgrounds. Each annotator was required have professional experience in software engineering, graduate-level education in computer-related disciplines, and adequate English profi- ciency for reading and evaluating technical documents. The profile of representative annotator shown in Table
Table 5: Background information of representative human annotator.
Name (pseudonymized)
Human Annotator A
Educational background
M.Sc. in Computer Technology
Professional experience
3 years in software engineering
Language proficiency
Fluent English, IELTS overall band score 6.5
Annotation role
Evaluation of query-document pairs, semantic relevance label- ing, and adherence to detailed annotation guidelines
For the validation step, annotators were provided with detailed evaluation guidelines. Each annotator was asked to assess whether the generated cool.profil ill faithfully and completely reflected original tool documentation, while avoiding hallucinated or inconsistent content. The full instruction provided to annotators is reproduced below.
You are given a tool's original documentation and generated tool tool_profii Please carefully check the following aspects:
Human evaluation instruction:
Faithfulness: Do all fields in the tool-profile strictly reflect information sup- ported by the original documentation?
Completeness: Are all important details from the original documentation preserved?
Hallucination check: there any invented functionality, parameter, limitation, or usage example not grounded in the original documentation?
Consistency: Are the fields logically consistent with each other (e.g., no contradictory descriptions)?
Based these criteria, decide whether the generated tool tool_protttee .profi le is acceptable (Pass) not acceptable (F.
D EXAMPLES AND CASE STUDY
After adding the tool profi l field tool doc, we observed that some tools which were orig- inally outside the top 10 moved into the top 10 with the help of the enhanced tool profi i le. The ranking comparison is based on the Qwen3-4p 310-3-44B-mb---- model evaluated on TOOLRET and TOOL- REX.
Web Case. For wwr.taasssteeuueryy..a.. the positive tool improved from outside the top 10 to rank after adding the tool profile (see Fig.
Code Case. For traaft.Tabmm...uerry.ff the positive tool advanced from outside the top 10 to rank 2 after adding the tool,profiln (see Fig. 10].
Customized Case. For appbench query 2, the positive tool rose from outside the top 10 to rank 2 after adding the tool_proof ie (see Fig. 11).
17

---

Published as a conference paper at ICLR 2026
E
THE USE OF LARGE LANGUAGE MODELS (LLMs)
During the writing process of this manuscript, we utilized GPT-5 to support language refinement. The model was specifically applied to enhance fluency, readability, and stylistic consistency by cor- recting grammatical issues and smoothing sentence flow. is important to note that all conceptual contributions, experimental designs, and results presented in this paper were independently devel- oped and validated by the authors. The LLM's role was strictly limited to linguistic polishing, ensuring that the intellectual content of the work remains fully attributable to the researchers while benefiting from clearer academic presentation.
18

---

accurate and Published as a conference paper at ICLR 2026
You will receive a copy of the API documentation.
Your tasks are strictly based on this API. A new "tool_profi field has been added to the docunentation.
The goal is to make the overall documentation complete,
not inconsistent.
## Requi red Fields (Always incl uded in tecl_profile)
"function"; less than 20 words; clearly describe the tool's main function using real terms.
"tags": 3-5 lowercase keywords; no repetition or self-contradictions phrases covering the main topic, primary operations, and synonyms.
Optional Fields (Ineluded only if explicitly supported by the original documentation)
"when_to_use": less than 20 words; actual usage scenarios.
"example_usage": Maximum of 2 Include projects only if the actual example can constructed di ricety from the fields in the original document. Each project may contain:
-qquery: A natural or keyword-based question
-"api_cali": An actual API usage call consistent with the tcol's actual parameters.
"limitation"; Included only the original document explicitly
mentions limitations, rate limits, or special conditions.
*# Iaportant Rules
add functiens, parameters, outputs, limitations, or fields that do not exist in the original document. Consistency: If the original document is about finance, don't mention weather. If it's about image procossing, don't mention stock data. Stick to the relevant fields.
Absolutely fiction:
optional fields Output only valid JSON objects, formatted as follows:
explicitly supported, omit them entirely.
"tool profile":
"function": " .... "tags": [*... "...",, "}, "when_to_use": "..." "example "example_usage" usage":
"query": "...", "api_call";
},, "limitation";
}}
Important:
The output must be directly parsable via 1son.loads () not incluce comments, instructions, Markdown code fences. not use the <think> step.
If the documentation does not explicitly support an optional field, omit it rather than adding it yourself.
The "tool_rrofll" document now generated based on the following API:
(api_document]
Figure 6: Prompt template used for tool profile generation.
19

---

Published as a conference paper at ICLR 2026
You will receive two inputs:
(1) The original API documentation.
(2) An expanded "tcol_profile" derived from the documentation.
Your task is to judge whether the toollprcfile is consistent with the original documentation.
## Field Definitions
"function": The main purpose of the tcel.
"tags": Keywords describing the domain and usage.
"when to to_use": Scenarios where the tcel should be applied. "example_usage":
usage": Example queries and cerresponding API calls. "Limitation"; Constraints explicitly mentioned in the original documentation.
| Important Rule
not invent or assume information.
Base your judgement only on the given API documentation.
output only word: "true" if the profile is consistent, otherwise "false".
Figure 7: Judgement prompt for verifying the consistency of expanded tool profile with the original API documentation.
<lim_stard start J>system Judge whether the Tool Document meets the requi rements based on the Query. Note that the answer only be "true" "false". <lim_end|>
lli__ttar|// |>user
Query: [ILLY_UERY_HERR Tool Document: FILL DOCUMENT HERE <lin_end|>>
<|im_start [Dassistant <think>
</think>
Figure 8: The reranking prompt template used in our experiments.
20

---

Published as a conference paper at ICLR 2026
Web Case
In the scenario of t. eval..stee query 21, the positive tool t eval steep..p.pp...............pp.... t.ol.33.33.. 38 rose from outside the top 10 rank 4 after the addition of the tool.proille.
Query: am interested in the latest movies in China. Please provide me with the details of the top 3 movies currently playing in China. Additionally, would like to know the details of the top 5 upcoming movies in China. Finally, I want to get the description of the movie "The Battle at Lake Changjin'.
Query Instruction: Given a movie information retrieval task, retrieve tools that provide details about currently playing, upcoming, and specific movies by processing param- eters like region, number of movies desired, and movie titles to deliver comprehensive movie information aligned with the query's specifications.
TOOLRET:
"name": "FilmDouban coming_out_fiitee""
"description": "prints the details of the filtered [outNum] coming films in China",
trequiree__paramtterss [],
" "aptiiona__paraneters""
("name": "regicn", "type": "STRING", "description": "the region of search query, must be in Chinese.") "I, "STRING", "description"; "the categery of search query, must be in Chinese. "), ("name": "outNum", "type": "NUMBKR", "description": "the number of search query")
"name": "cate", "catte, "type":
b,
"return :
I"name": "film", "description": list film information, including date, title, cate, zegion, wantWattchFeooleelum, link
TOOL-REX (identical to ToolRet except for the additional tool-profil ile field):
(same as ToolRet)
"tool "tool__rofile":
"function": "Filters and retrieves upcering Chinese films with details like title, date, and region", "tags": I"film", "upcoming", "china", "filter", "movie details"), "when to use": to__tt "When planning movie outings or researching new releases in China",
"limitation"; "Region and category paraneters must be provided in Chinese"
Figure 9: Case study: Web query and tool document expansion example.
21

---

Published as a conference paper at ICLR 2026
Code Case
In the scenario of craft..abbww..quez. _55, the positive tool craft_.abmmp.ttoo.555 rose from outside the top 10 to rank 2 after the addition of the tool.profile.
Query: A business magazine surveyed its readers about their commute times. How many com- mutes are exactly 43 minutes?
Query Instruction: Given data analysis is task. retrieve tools capable of analyzing com- mute time data to count instances equal to specific value, utilizing structured data input ac- cording to the query's parameters and criteria.
TOOLRET:
"name": "count wwwssttanee_wtttsspeeeeeeeeeee sttp::iii_vvvuueeinsssteseeeeeee inte_in__ttttt_tttteeeeeeeeee (data frame @tte__ooy, leaf_col, specitic_vaaue)) "description": "def
(data_frame, stem_col,, leaf_col, specific_value) :\n This function takes in a pandas DataFrame representing stem- and-leaf plot of instances and specific value, and returns the number of instances that have values equal the specific value. \n (content truncated)
(" \"\"\"\n
In return num sum_items"
}
TOOL-REX (identical to ToolRet except for the additional tool.profin field):
... (same ToolRet) "rool_roffle"
"function": "Counts occurrences of a specific value in a dataset
."
"tags": ["stem", "leaf", "plot", "ccunt", "frequency"1, "when_to_use": "Use when you need to count occurrences specific query a dataset.",
"limitation"; "Only works with stem-and-les formatted data."
Figure 10: Case study: Code query and tool document expansion example.
22

---

Published as a conference paper at ICLR 2026
Customized Case
the scenario of appbench. wwt.hh.uee.... 2, the positive tool appbench.tool.22 rose from outside the top 10 rank 2 after the addition of the tool.proflle.
Query: Initiate a private transfer $93 to Wilson using my credit card, then find an affordable Asian fusion restaurant in San Jose, reserve table for at the selected restaurant for 11:30 AM on March 3rd, and conftrm the reservation.
Query Instruction: Given Transaction and Reservation task, retrieve tools that facilitate private money transfers and restaurant operations by processing transaction details (such payment method, amount, and payee) and restaurant-specific information (including location, category, seating, reservation time, and date) to efficiently confirmfinancial financial and dining arrangements.
TOOLRET:
"i ris_transactional" true, "additional_requu red_grunnt_ss
"payment_methle method (str)": : "The source of funds used for payment.
Value can only be one of the following: application balance, eredit debit card", "amount (float)": "The amount to send er request", "reseiver (str)"; "The name of the contact or account making the transaction"
"
"private_visibile (bool) "Whether the transaction is private"
"result arguments";
"payment methed (str)"; "The source funds used for payment.
Value can only one the following: application balance, credit or debit card",
"amount (float)": "The amount to send er request", "receiver (str)": "The name of the contact or account making the transaction", "private_visibilitt) (bool) " "Is the transaction private?" b,
"name"; "makcpayment",, "description": "Send money friend"
TOOL-REX (identical to ToolRet except for the additional tocl-pofile field):
(same as ToolRet) "tool_profile".
"function": "Conveniently send money to recipient via the API.", "tags": ("money transfer", "payment", "api", "transaction", "send money"],
"when_to to__te": "Khen you need to use the API to send money to recipient.",
"limitation"; "Only supports sending meney, not receiving money."
Figure 11: Case study: Customized query and tool document expansion example.
23


Reviews:
[{"review_id": "saxotz1twA", "reviewer": "Reviewer_81wM", "summary": "The authors introduce TOOL-DE, a new framework that systematically enriches tool documentation with structured fields to enable more effective tool retrieval. The framework expands the tool documents by using structured fields like function description, when-to-use, limitations, and trains a dedicated retriever and reranker on top of the expanded documents. The experimental results show the effectiveness of tool expansion as well as the trained retriever and reranker.", "questions": "None", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper effectively addresses the limitations of current tool learning paradigm where existing tool documents are underspecified for effective tool retrieval by using the idea of tool expansion using LLM which shows strong emperical results with and without dedicated trained retrievers and rerankers.\n- It is an end-to-end framework where they create a tool document dataset and train the retriever and reranker. \n- The paper includes extensive ablations studies on impact of each field of expanded document in tool retrieval and how this affects the tool retrieval similarity", "weaknesses": "- While the framework shows strong performance, the idea of revising or augmenting the tool documents has been explored by the previous works [1,2]. \n- To make the data generation more scalable, one might consider replacing human verification to using a strong LLM\n\n\n[1] Huang et al, Planning and Editing What You Retrieve for Enhanced Tool Learning. NAACL 2024 \\\n[2] Chen et al, EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction. ACL 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IPMsyJvGCG", "reviewer": "Reviewer_qfqi", "summary": "This paper tackles an underexplored yet critical problem in tool-augmented LLM systems — the poor quality and inconsistency of tool documentation, which limits accurate tool retrieval. The authors introduce TOOL-DE (Tool-Document Expansion), a benchmark and framework that systematically enriches tool documentation through LLM-based expansion. Their pipeline generates structured fields (e.g., function description, when to use, limitations, and tags) via multi-stage prompting, validation, and human checking, yielding large-scale, standardized tool profiles. On top of this, they build two dedicated models: Tool-Embed (a dense retriever) and Tool-Rank (a reranker), trained on 50k and 200k examples respectively. Experiments across the TOOL-DE and ToolRet benchmarks show significant improvements in retrieval quality, achieving new state-of-the-art results (e.g., NDCG@10 = 56.44, Recall@10 = 67.81). Analysis further confirms that document expansion improves both training and evaluation by reducing semantic gaps, enhancing discriminability, and stabilizing optimization.", "questions": "See above", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The benchmark is comprehensive. TOOL-DE is built over 35 datasets with a carefully validated expansion process, combining open and closed models (Qwen3, LLaMA-3.1, GPT-4o) and human checks. \n2. Experiments show solid improvements. Both retriever and reranker consistently outperform strong baselines, demonstrating that simple, well-structured enrichment can yield significant improvements.\n3. The paper is well-written and easy to follow.", "weaknesses": "1. The manuscript lacks a clear and comprehensive description of the dataset. While some details are provided in the appendix, it would significantly improve clarity and reproducibility to include a dedicated section in the main text describing the dataset composition (e.g., number and types of tools, instances per tool, data sources, and preprocessing steps).\n2. The training and testing splits is insufficiently explained. The paper shows the proposed pipeline works well when train and test on the same set of tools. Without a detailed account of how the splits are defined, it is difficult to assess whether the proposed approach effectively generalizes to unseen tools. This consideration is particularly important, as in real-world scenarios the set of available tools is dynamic and evolves continuously.\n\nA relevant paper that might be included in the related works: [Planning and Editing What You Retrieve for Enhanced Tool Learning](https://aclanthology.org/2024.findings-naacl.61/) (Huang et al., Findings 2024)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3p2uGWN8db", "reviewer": "Reviewer_qnbA", "summary": "This paper addresses the problem of under-documentation of tool APIs. Specifically, as the authors mentioned, current API documents often lack standardization and critical information. To solve this problem, the authors propose a pipeline for API documentation augmentation. They apply the proposed pipeline on ToolRet to construct a new dataset, ToolDE. And the authors train one embedding model and one reranking model on ToolDE. The evaluation results demonstrate that augmenting documentation is beneficial. And training with standardized documentation leads to better retreival performance.", "questions": "Please see my comments above.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The studied problem is interesting and valuable. Indeed, the quality of API documents often has high variance. Standardization of them is expected to be valuable.\n2. The paper is well written and easy to follow.", "weaknesses": "1. The proposed pipeline for augmenting API documentation involves human annotation, which makes it hard to scale up. \n2. According to Table 1, I find that the improvement of augmenting documentation without training is limited, especially for Qwen3-Embedding series. There is even a performance drop after augmenting the documentation, which makes me doubt the solidity of the motivation of this work.\n3. Following the second point, it would be more valuable for applications if direct augmentation without training could lead to performance improvement. In real-world applications, APIs are often evolving. It is costly to augment new API documentations and train a new embedding model each time. Yet, as I mentioned in the second weakness, I find the direct improvement of augmentation is limited, which significantly limits the contribution of this work.\n4. To demonstrate the generalization of augmenting documentation, the authors should train embedding models other than Qwen series on ToolDE.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WknEvXIaVa", "reviewer": "Reviewer_wGrF", "summary": "This paper tackles a problem in tool retrieval, real tools often have incomplete descriptions, which hurts both retrievers and rerankers. The authors aggregate multiple sources of tool-use data and propose a four-stage, LLM-assisted document expansion pipeline that adds structured fields (e.g., function, when to use, limitations, tags). They then fine-tune a dense retriever and a reranker on these expanded docs.", "questions": "1. What happens when you train with mined hard negatives?\n2. How do you handle deduplication and near-duplicate filtering across splits after expansion?\n3. Can you share the end-to-end expansion cost/throughput and per-stage success/failure rates?\n4. Beyond the reported non-expanded benchmark, can you add another unexpanded corpus or a held-out unexpanded slice to test robustness to documentation style?\n5. You state that rerankers benefit more under expanded views. Can you keep the retrieval top-K fixed and rerank twice, once with original docs and once with expanded docs, to isolate the pure evaluation-time effect of expansion on the reranker?\n6. The ablation results and the final released field set don’t seem fully aligned. Could you reconstruct this section and reconcile the ablation findings with the fields you keep/drop, including the supporting numbers?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. A practical, scalable pipeline with sensible checks.\n2. Consistent gains, plus field-level ablations that give actionable advice.\n3. Clear look at when expansion helps (train vs. eval); rerankers seem to benefit most when expanded views are present.", "weaknesses": "1. Training on your own expanded corpus and evaluating on the expanded view makes strong gains somewhat expected. The non-expanded results help, but an additional external/unexpanded test would make the case stronger.\n2. Only random negatives during training. Adding hard negative mining would be helpful.\n3. Documenting how duplicates/near-duplicates were removed across train/val/test after expansion, and report any impact on scores will be more helpful.\n4. For human validation, authors should consider a random sample across all stages, report agreement, and provide an error taxonomy.\n5. End-to-end throughput and cost (e.g. tokens, $) for generation, judgment, and refinement would help gain more insights.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)