RefTool: Reference-Guided Tool Creation for Knowledge-Intensive Reasoning
Authors: Xiao Liu
Da Yin
Zirui Wu
Yansong Feng
Venue: ICLR 2026 Poster (2026)
Keywords: Tool Creation
Tool-Augmented Reasoning
Primary area: foundation or frontier models, including LLMs

Abstract: Large Language Models (LLMs) can enhance their reasoning capabilities by using external tools. However, many tasks lack predefined tools. Prior works have explored instructing LLMs to generate tools on their own, but such approaches depend heavily on internal knowledge and struggle when tasks fall outside the model’s knowledge scope. To address this limitation, we propose RefTool, a reference-guided framework for automatic tool creation that leverages external materials, such as textbooks and knowledge snippets. RefTool consists of two modules: (1) tool creation, where LLMs generate executable tools from reference content, validate them using illustrative examples, and organize them hierarchically into a toolbox; and (2) tool utilization, where LLMs navigate the toolbox structure to select and apply the appropriate tools to solve problems. Experiments on causality, physics, and chemistry benchmarks demonstrate that RefTool outperforms existing tool-creation and domain-specific reasoning methods by 12.3% on average accuracy, while being cost-efficient and broadly generalizable to non-scientific tasks, e.g., extremely low-resource language translation. Analyses reveal that grounding tool creation in references produces accurate and faithful tools, and that the hierarchical structure facilitates effective tool selection. RefTool enables LLMs to overcome internal knowledge limitations, advancing generalizable reasoning in knowledge-intensive domains.

Published as a conference paper at ICLR 2026
REFTOOL: REFERENCE-GUIDER TOOL CREATION FOR KNOWLEDGE-INTENSIVE REASONING
Xiao Liu Da Yin Zirui Wu' Yansong Feng
1 Wangxuan Institute of Computer Technology, Peking University
2 University of Chicago
3 University of California, Los Angeles
(lxlisa, ziruiwu, fengyansongJjpkk.. wdwp:e/owooeeee.eeuuuee.o edu.cn, da. da.yin97120gail.ccmm
ABSTRACT
Large Language Models (LLMs) enhance their reasoning capabilities by ing external tools. However, many tasks lack predefined tools. Prior works have explored instructing LLMs to generate tools their own, but such approaches depend heavily on internal knowledge and struggle when tasks fall outside the model's knowledge scope. To address this limitation, we propose REFTOOL, a fnference-guider framework for automatic tool creation that leverages external materials, such as textbooks and knowledge snippets. REFTOOL consists of two modules: (1) tool creation, where LLMs generate executable tools from reference content, validate them using illustrative examples, and organize them hierarchi- cally into toolbox; and (2) tool utilization, where L.MMs navigate the toolbox structure to select and apply the appropriate tools to solve problems. Experiments causality, physics, and chemistry benchmarks demonstrate that REFTOOL out- performs existing tool-creation and domain domain-specifie reasoning methods by 12.3% average accuracy, while being cost-efficient and broadly generalizable to non- scientific tasks, e.g., extremely low-resource language translation. Analyses reveal that grounding tool creation in references produces accurate and faithful tools, and that the hierarchical structure facilitates effective tool selection. REFTOOL enables LLMs to overcome internal knowledge limitations, advancing generaliz- able reasoning in knowledge-intenee intensivv domains. Code and data are available at https:://gittuub.oom//xxxxaaaa
INTRODUCTION
Tool Creation
Tool Utilization
Reference Material
Compute the merrag treatment effect (RR) using inverse probabiiity we ghtiig based astimated propensity scores.
Description:
Question
Chapter. 7 Estmatiin 7.6 laverse Prububility W Weighling
What the avemgn trontment effect (ATE) Y?
Function:
Initial Tool
def corpote are tpu(T. T, proganetsy recoret):
Heraachical Tool Selection
Generation
ueighte sreased T progeaxtty..corme
weighte control
D. Ct. proporosty cocree)
Selected Tools
compule aie properity scomes
weighted, cutcome, Loauted
(weggha,, Lecuted
Step Category selection
/ (weipbts, treates)
Step 2. Tool selection within catlegoyy
wwwp:ettttooooooeeooooooeoeeeeee control concrol)
(wwighta_contre..
Tool Verification and Refinement
Solution
Gene ration
weighted Soigctoedoutcooe..o teeated
Solution
Contains tools that successfilly execute and solve example problems
Answer the question with the tools
Example: (Problem, Solution, Answer)
Figure I:
Overview of the REFTOOL framework, which consists of two modules: tool creation (left) and tool utilization (right).
Tools play critical role in enhancing the reasoning capabilities of large language models (LLMs),
particularly in scientific mobblem-solvigg like mathematical reasoning (Lu et al. 2023 Zhang et
"Corresponding author.

---

Published as a conference paper at ICLR 2026
2023). By integrating external tools, LLMs can off-the-shelf modules to complete subtasks and exccute precise computations, thereby improving their performance.
Despite their importance, such tools are not universally available across all scenarios. A prominent line of work attempts to mitigate this limitation by instructing LLMs to generate their own tools based given problems (Qian al. 2023 Cai al., 2023] Wang et al.] 2024b). However, these methods would fall short when models lack the relevant expert knowledge, especially in specialized and novel domains. For example, a treatment variable to an outcome variable, it can hardly generate appropriate tools for such tasks.
LLM is unfamiliar with how to estimate the causal effect from
To address this challenge, propose REFTOOL, a reference-guided framework for automatic tool creation. Unlike existing methods that rely on LLMs' internal knowledge, REFTOOL leverages external reference materials, such textbooks and knowledge snippets, that naturally cover broad range of domains. As shown in Figure REFTOOL consists of two modules: tool creation and tool utilization. During tool creation, the framework employs LLMs to generate executable tools from reference content. In the example, given a segment on Inverse probubility weighting from causal inference textbook, the LLM produces tools like compute_ _etee_aa_______e _p_i according to the content. The generated tools consist of descriptions, functions, alongside illustrative examples teaching models when and how to use the tools. These examples also serve as validation cases, filtering out non- functional incorrect tools while retaining those that successfully solve the example problems. The validated tools are organized into a hierarchical toolbox, mirroring the structure of the reference material, or created by the model if the reference is unstructured.
During inference, REFTOOL guides the LLM to select tools from the toolbox hierarchically and apply tools to solve problems. For input question like what is the average trealment effect from T to Y, the LLM navigates the toolbox hierarchy, selecting the Extimation category and then the compute_ate
tool within the category. Finally, the LLM generates the solution with the help of the selected tool. By grounding tool creation and selection in external references rather than internal knowledge, REFTOOL can construct and deploy tools beyond the model's original capabilities. enabling it to tackle tasks that would otherwise be infeasible.
We evaluate REFTOOL across three -nonlevge-intensive intensive scientific domains: causality, physics, and chemistry. With the help of textbooks, REFTOOL outperforms existing tool creation methods by 13.0% on average, highlighting the value of incorporating external knowledge tool creation. REFTOOL also achieves an average accuracy improvement of 10.2% over domain-spectie reasoning methods (Pang et al. 2025; Ouyang et al., 2024| Tang et al., 2025). Unlike prior works that depend on manually constructed toolsets extensive trial-and-erro
validation data, REFTOOL achieves
greater efficiency in both time and computational cost.
REFTOOL exhibits strong generalization. Akin to human learning knowledge, the generated tools not dataset-specific, but maintain robust performance across diverse datasets in the domain. REFTOOL further proves effective in non-scientific tasks and with unstructured references. In extremely low- resource language translation, it organizes unstructured grammar rules into a hierarchy and transforms them into pseudo-code tools, yielding improved translation performance.
To summarize, we propose REFTOOL a reference-guider framework for tool creation. REFTOOL has the following advantages: (1) By leveraging reference materials, REFTOOL enables LLMs to generate tools beyond their internal knowledge. (2) Experiments on diverse moowledge-intensigg reasoning tasks demonstrate that REFTOOL consistently improves performance over existing baselines. (3)
REFTOOL generates dataset- agnossic tools in cost-efficient and human- numan-free manner, demonstrating the potential for extending the knowledge boundary of LLMs in real-time problem solving.
2 THE REFTOOL FRAMEWORK
REFTOOL operates in two stages: (I) constructing a hierarchical toolbox T' from reference material R. and (2) selecting and applying tools t c T answer the input question g during inference. This section introduces the method with focus on scientific reasoning tasks, and describes how the method is adapted to con-scientific tasks.
Inverse probability weighting common method for estimating causal effects.
2

---

Published as a conference paper at ICLR 2026
Toolbox
Category Estimation
Tool
compute Jpw
Tool my Tool ss. compute_ate, _pp__peea_e_jew
Description: Compute the arerago treamment ellectATE [ using prolubiiity weigting lased estimated propensity scores
Reference Material
Chapler.7 Estimation
Functions
sdl compats.ate..gon T. propenntty, acoevel:
zeggazs, sreated T/ properatty, acaree
7.6 lnverse Probubility W Weighting
veAgat contest
1I peopesnity woorso)
Assuming binory tesemment the tollowing identiiiation equatior: for the ATE: follows from Equation 7.18
7 www.rn/wwwsgaaa.ccmmm towated)
sugghiie_rrattommmcoo -pp.oasfww.ghoo.reeerr.. / up. www.snilleeggaa..oom control]
-(r()- r00]
e(W)
[-e(W)
77399
weighend wttph//nontttteeeeooooo
sie vedgated oatcona, created
that we akabstical estimand the form IPW, wa can get on IPW entimmtoo Replacing expectations by empiredd mumn- and e(W) by proponaty scume mudel E(W), we the Sollowing equavaleet formulations
Example:
Probkfem: Giver treement indicators T+00 observed outcomes = 3, 1.4] estimated propersity treatment effect (ATE) using IPW.
oxtimater" for ATE:
08, computi wergge
RRG - 12g
10, - Dlu
Elon))
86m-
(7.20)
Solution:
def solation(]:
T pa: Gertien (D,, 1, 0, 113
7 pd. Sersent (2, 3, 1, 437
Blan)
peopenasny secres = up (r. 2, 0.8. 0.2, 6. ate scapate iieil, Y, pespensitt..cceee))
where it and Aa are the namber of treatment groap units and control group units, respuctivell.
resurn
xemalt wolotion()
AAnswer:21
Figure 2: Example of a generated tool and its corresponding reference segment.
2.1 THE TOoL. CREATION MODULE
Knowledge Organization The first step in tool creation is to organize the knowledge from reference materials into a structured form. In this work, we adopt a two-level hierarchy for knowledge organization, which also defines how the generated tools are arranged.
Many references, such as textbooks and technical documents, naturally follow a hierarchical orga- nization that supports systematic knowledge acquisition. At the highest level, they are divided into chapters (e.g.. Estimation the causal inference texthook), which are further decomposed into sections (e.g., Inverse Probability Weighting), each addressing a specific technique, theorem, or application within the broader chapter context. For such cases, we directly extract this inherent structure and adopt the chapters the first-level categories.
For unstructured references like knowledge snippets, we ask an LLM to construct the hierarchy based on the content. The model first proposes category names and then assigns reference segments to appropriate categories. After the conversion from unstructured reference material to structured ones, apply the same general method introduced in the following to create tools.
Initial Tool Generation Given reference segment, such as section s; E R, the LLM is instructed to generate executable tools based on its content. Each tool consists of three key components, as illustrated in Figure[] First, description provides a natural language summary of the tool's purpose. Second, a function offers a Python implementation of the tool, including comments that explain its parameters and return values. Finally, an example demonstrates the tool's usage, comprising problem, a piece of solution code where the tool is invoked, and the expected answer. The model prioritizes examples from the reference text when available, otherwise generating an appropriate example by itself. The LLM is asked to generate at most m tools for each section. To ensure proper formatting, the prompt includes numan-written tool example from different domain.
Tool Verification and Refinement Each tool is verified through execution testing and output verification using the geder-ee demonstration example. The solution code should run without errors, and the output should match the expected answer. Failed tools trigger refinement step, where the failure information is provided to the LLM to refine the tool. Finally, the valid tools are organized hierarchically into toolbox.
2.2 THE TOOL UTILIZATION MODULE
Hierarchical Tool Selection During inference, REFTOOL performs hierarchical retrieval to select tools for question q through two phases:
3

---

Published as a conference paper at ICLR 2026
Table 1: Statistics of the reference materials and created tools. "Avg. Lines" indicates the average lines of tool functions.
Domain
Book
# Categories Sections # Tools Avg, Lines
Causality
Introduction Causal Inference (Neal 2020)
55
24
Physics
University Physics (Ling ct al. 201
44
284
515
16
Chemistry Atkins' Physical Chemistry (Atkins et
2023
19
90
158
17
Category Selection: Given the toolbox categories C, the model is instructed to select at most Ye relevant categories e C for the question q.
Tool Selection within Category: For each selected category Ci the model is given access to all tools from the toolbox T associated with that category. including their descriptions, functions, and demonstration examples. It is then prompted to select up to ni, relevant tools t, or none if no tools are deemed applicable.
The selected tools then integrated into the reasoning process. We incorpo- rate the tools with two reasoning paradigms: single-turn Program-of- Oofgram-of-Thoughts (PoT) reasoning (Chen) al. 2023a) and multi-turn ReAct-style agent reasoning (Yao et al., (002)). For both paradigms, the model receives selected tools in the initial prompt and is instructed to invoke them when appropri- ate. When no suitable tools are identified. REFTOOL defaults standard PoT or ReAct reasoning. ensuring graceful degradation for questions outside the reference domain.
Solution Generation
EXPERIMENTS
We conduct experiments on three gewwledge-intensiie scientific domains: causality, physics, and chemistry. This section introduces the experimental setup, presents the performance of REFTOOL, and validates its generalizability to other datasets and non-scientific domains.
3.1 EXPERIMENTAL SETUP
Datasets We employ the following evaluation benchmarks: (1) Causality: QRData-causal LLi
al. 2024a). where each question is accompanied by one or multiple datasheets. Models are asked to ana analyze yze the datasheets and answer causal questions. (2) Physics: TheoremQA-physic (Chen et al.
2023b, covering broad topies of university-level physics. (3) Chemistry: SciBench--cemissty (Wang et al. 2024a), focusing on three sub-datasets (chemmc, quan, and matter) related to physical and quantum chemistry
We maintain consistent evaluation protocols (like answer extraction methods and tolerance rates) with the original benchmarks (see Appendix A. for details) and report accuracy as our primary metric.
Reference Materials Analogous to humans preparing for an exam by reading relevant textbooks, select reference materials that have similar domain of knowledge to the evaluation datasets.
For causality, we choose Introduction to Causal Inference (Neal, 2020), which provides a detailed description main causal inference topics like causal discovery and estimation. For physics, as university physics is a broad domain, we choose the three-volume textbook University Physics (Ling al. 2016), which covers the core concepts of physics like mechanics, thermodynamies, and modern physics. For chemistry, given that the benchmark is in physical chemistry, we choose a famous physical chemistry textbook Atkins' Physical Chemistry (Atkins et al. 2023 Table | (left) provides detailed statistics. Note that none of the evaluation questions originate from these books, and none of these books contain code directly.
We employ GPT-4o (Hurst et al. 2024) for tool creation, and evaluate four prevalent LLMs for tool utilization: Llama-3.1-70B (Dubey et al. 2024). Gemini-1.5 1.5-Pro (Taa et al. 2024), GPT-4 (OpenAl 2023). and GPT-4o.
Implementation Details
WWe omit the other sub-dataset atkins because its question source overlaps with our reference material. QQanttum chemistry is subdomain of physical chemistry, and also introduced in this textbook.
4

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison in causality (QRData), physics (TheoremQA), and chemistry (SciBench). Numbers are in percentages (%), with the best performance for each model in bold.
Accuracy
Method
Llama-3,1 3.1-70B
Gemini-l.5-Prrs
GPT-4
GPT-4o
Average
Causality
LATM
33.5
32.3
27.0
20.9
28.4
Creator
14.9
29.7
39.4
39.8
31.0
TroVE
23.8
33.5
36.4
32.0
PoT
33.1
41.3
34.2
39.8
37.1
PoT + RAG
29.7
36,4
37.5
42.0
36.4
PoT REFTOOL
36.8
43.9
38.7
46.8
41.6
ReAct
30.1
47.6
50.9
46.5
43.8
ReAct + RAG
RAG
32.3
46.8
48.0
49.1
44.1
ReAct + REFTOOL
33,5
48.3
51.3
52.0
46.3
Physics
LATM
38.9
33.3
39.8
30.6
35.7
Creator
40.4
57.0
35.1
40.4
43.2
TroVE
33.3
58.8
35.1
48.2
43.9
Physics Reasoner
48.2
50.9
42.1
33.3
43.6
PoT
48.2
57.9
57.0
52.2
PoT + RAG
44.7
57.0
44.7
57.9
51.1
PoT + REFTOOL
53.5
58.8
49.1
57.9
54.8
Chemistry
LATM
31.4
25.1
45.0
35.7
34.3
Creator
40.1
60.0
46.9
43.3
47.6
TroVE
38.6
65.6
52.7
49.0
StructChem
37.9
50.2
29.7
40.5
39.6
ChemAgent
48.2
65.5
52.5
58.9
56.3
PoT
46.9
62.3
51.8
58.9
55.0
PoT RAG
48.1
63.7
54.1
56.6
55.6
PoT + REFTOOL.
49,5
66.4
53.4
61.3
57.7
During tool creation, we set m = 2 tools per section across all domains. This can be adjusted based each section's length and information density. For tool utilization, employ default category and n,, have validation set, we the default setting for the causality and physics domains. For chemistry, we perform grid search over E [1,2] and ne E [1, 2] on the validation set of SciBench, and choose ne 1 and n4 2. Additional details and prompt templates can be found Appendices A.2 and
configuration of selecting
tool. As QRData and Theorem do not
Baseline Methods We compare against the following baselines: (1) General reasoning methods
(PoT) for single-tur reasoning and ReAct for multi-turn reasoning (2) Retrieval-augmennte generation (RAG) methods using the same reference books employed for tool
including Program-of-Thooeee
creation; (3) General-purposs tool creation methods including LATM (Cai et al. (023), Creator (Qian
et al. 2023), and TroVE (Wang et al. 2024b); (4) Domain-specippe reasoning methods including
Physics Reasoner (Pang al. (2025), 5), StruciChem (Ouyang et 2024). and ChemAgent (Tang et
2025 Detailed descriptions of the baselines are in Appendix A.3
3.2 RESULTS
Toolbox Construction Table (right) demonstrates statistics of tools created. On average, 73% of initially generated tools pass validation directly, with additional 14% tools succeeding after refinement. We assess tool quality through human evaluation in $4.3
'While ReAct demonstrates effectiveness on QRData by allowing error correction through multi-tun aulti-tum interactions (Liu et al. 2024a). our preliminary experiments (Appendix Table show limited benefits for physies and chemistry domains, likely due the simpler code solutions without data analysis and fewer execution error. Consequently, omit ReAct for these domains.

---

Published as a conference paper at ICLR 2026
Main Results The performance comparison in Table demonstrates REFToOL's superior perfor- mance across all domains, achieving the highest average accuracy For each of the three domains, REFTOOL's accuracy (aggregated across all four LLMs) is significantly higher than all baseline methods at the significance level a = 0.05. Notably, REFTOOL surpasses all tool creation methods by an average margin of 13.0%, highlighting the advantage of references in tool creation.
While RAG incorporates the same reference materials, it fails to consistently enhance the performance. This suggests that direct retrieval struggles effectively extract and apply relevant knowledge, while REFTOOL's 's tool format and hierarchical organization enable better utilization of reference materials.
Among domain-specific methods, Physies Reasoner and StructChem perform inferior to PoT, with their complex format requirements leading to suboptimal adaptation on some models. Although ChemAgent approaches REFTOOL's performance, it needs significantly higher computational costs, as discussed in $4.2
Performance on Reasoning Models We also conduct small-scale experiment to evaluate if REFTOOL works for reasoning models. We apply REFTOOL on ol-mini (Jaech et al. 2024), and Appendix Table shows that its average accuracy improves by 4.3% over PoT. This indicates that REFTOOL is also compatible with reasoning models, supplementing their knowledge and skills.
Robustness of Tool Creation We validate whether REFTOOL remains effective when using alter- native LLMs for tool creation. Appendix Table shows that creating tools with Gemini- and -lama-3.1-700-Insttii Instruutttttttttttt also achieves superior performance compared to baseline methods.
3.3
GENERALIZABILITY REFTOOL
Table 3: Performance of REFTOOL on extremely low-resource language translation (%).
Zhuang - Chinese
Chinese > Zhuang
chrU/chrF++
Average
IJama-3. 3-700 Qwen-2.5-72B Llama-3.1-70B
Qwen-2.5-72BB
Whole Grammar Book
32.6 35.2
43.7 42,6
28.0 60.8
34.9 64.2
34.8 / 50.7
Retrieval from Whole Book
33.6 35.3
45.3 /33.1
26.0/ 54.2
54.2
39.2
36.0/ / 49.8
Rule-by-Rule Retrieval
/2.6/43.0
47,4/46.
36.1 66.4
40.6 69.9
41.7 56.5
Rule-by-Rule Retrieval w. Code
36.4/45.J 45.1
56.9 / 55.7
3
47.0/74.3
43.9 60.0
REFTOOL
52.2 2.2/44.0
57.2 54.1
54.2/71.6
52.5/71.1
54.0 61.6
We validate the generalizability of REFTOOL. by examining (1) whether tools created for one dataset can be reused across other datasets in the same domain, and (2) whether the framework remains effective non-scientific domains and when handling unstructured reference materials.
Tool Reusability We conduct experiments another physics dataset SciBench-fund (Wang et al., 2024ah to validate the generalizability tools created. Appendix Table 12 shows that on SciBench- fund, REFTOOL outperforms all zero-shot baseline methods and matches 4-shot Physics Reasoner, using the same tools as in the evaluation TheoremQA. Since REFTOOL is dataset-agnostic, tools developed for one domain can be readily applied to other datasets within that domain.
Applying REFTOOL to Extremely Low-Resource Language Translation Extremely low- resource (XLR) language translation is representative non-scientific knowledge-intensive task, where LLMs, lacking prior knowledge of the XLR language, are tasked with translation using re- sources such dictionaries, parallel sentences, and grammar rules. Grammar rules are erucial for guiding the translation (Tanzer et al. 2024 Team et al.) 2024), and in this experiment, we explore whether REFTOOL can help L.LMs better select and apply these rules.
We experiment on Zhuang-Chiness translation using the ZHUANGRULES dataset (Zhang et al. 2025 which provides 109 grammar rules. The rules presented in unstructured format, making it difficult to identify the relevant ones. To address this, we ask the LLM to organize the rules into a two-level hierarchy. Despite limited prior knowledge of Zhuang, the model leverages its general linguistic knowledge to propose categories such as Numerals and Quantifiers and Word Order and Sentence Structure.
*Chemistry results are averaged over three sub-datasets, with sub-dataset performance in Appendix Table 'Zhuang is language spoken by the Zhuang people of Southern China.

---

Published as a conference paper at ICLR 2026
Table 4: Ablation results (%). (sim) indicates selecting with similarity.
Accuracy
Method
Llama-3. 3-10
Gemini-I..5PPre GPT-4
GPT-4o
Average
Causality
PoT RAG
29.7
36.4
42.0
36.4
PoT + Hierarchical RAG
36,8
38.7
43.5
36.8
39.0
PoT REFTOOL (sim)
30.5
36.1
46.1
39.4
38.0
PoT REFTOOL
36.8
43.9
38.7
46.8
41.6
Physics
PoT RAG
44.7
57.0
44.7
57.9
51.1
PoT + Hierarchical RAG
44.7
64.0
44.7
55.3
52.2
PoT + REFTOOL (sim)
45.6
62.3
43.0
56.1
51.8
PoT REFTOOL
53.5
58.8
49.1
57.9
54.8
We then apply REFTOOL to create tools for each grammar rule. Since the goal is facilitate rule understanding rather than execution, the tools are represented as pseudo Python code. During verification, the LLM assesses whether the tool functions correctly apply to examples, rather than real code execution.
We compare with previous XLR translation methods, including prompting LLMs with the Whole Grammar Book, Retrieval from Whole Book, Rule-by-Rule Retrieval (which examines each rule individually), and Rule-by-Rule Retrieval w. Code (which converts rules into pseudo code). Consistent with the settings of (ZZang et al. (2025), we use GPT-4o to construct and organize the tools, and evaluate performance on two open-source LLMs: Llama-3.1-70BB and Qwen-2.5-72B (Yng et al.
2024). More implementation details are in Appendix A.4
Results in Table showcase that REFTOOL outperforms all baselines on average, with an increase of 10.1% in BLEU and 1.6% in chrF++. By organizing rules hicrarchically, creating tools to facilitate understanding the rules, and verifying and revising the tools for better quality, REFTOOL enhances the translation performance. This highlights the broad applicability of REFTOOL to non-scientific domains and its effectiveness in handling unstructured reference materials.
ANALYSIS
In this section, we further analyze the effectiveness of REFTOOL through: ablation study of key components ($4.I) cost analysis ($4.2), human evaluation of tool quality ($4.3). and case study of how REFTOOL helps LLMs to answer questions (..4).
4.1 ABLATION STUDY
We design two variants of REFTOOL to analyze its key components: code-form tool creation and hierarchical selection. (1) PoT + Hierarchical RAG: Substitutes REFTOOL's 's code-form tools with raw text segments while preserving the hierarchical structure. This maintains the three-step reasoning process of category selection, intra-category text retrieval, and solution generation.
(2) PoT REFTOOL (sim): Retains the tool creation but replaces hierarchical selection with similarity-base retrieval. Tool descriptions are encoded into embeddings, with the most similar tool selected for each problem, mirroring standard RAG approaches but using tools instead of text.
Table shows the ablation results. Due to computational constraints, we focus on causality and physics domains with single-turn reasoning. By comparing PoT REFTOOL with PoT + Hierarchical RAG, as well as PoT + REFTOOL (sim) with PoT RAG, we observe an average 1.9% accuracy gain of tools textual knowledge, confirming that the code form of tools enhances model understanding and application of knowledge.
By comparing PoT REFTOOL with PoT REFTOOL (sim). well PoT + Hierarchical RAG with PoT + RAG, we find that hicrarchical selection outperforms similarity- retrieval by 2.6% average, demonstrating its effectiveness in knowledge retrieval.

---

Published as a conference paper at ICLR 2026
4.2 COST ANALYSIS
Table 5: Cost analysis of representative tool-augmented methods (with GPT-4o as the base model). "Human" indicates that the step is done by humans and the cost is unknown.
Time (min.)
Cost ($)
Domain
Method
Toolbox Construction
Inference
Toolbox Construction
Inference
Physics Reasoner
Human
75
Human
3.5
Physics
PoT REFTOOL
5
2
6.9
1.5
ChemAgent
1233 3
536
79.3
41.3
Chemistry
PoT REFTOOL
3.5
1.4
5 shows that REFTOOL greatly saves cost compared with two representative tool-augmented methods, and the comparison with a broader range of models is in Appendix B.6 Compared with Physics Reasoner which iteratively refines the reasoning process, REFTOOL reduces inference time by 97% and cost by 57%. The improvements divide-and-reery strategy: REFTOOL cuts both toolbox construction time and inference time by 99%. The guidance of references enables REFTOOL achieve great performance without repeatedly trying, offering a scalable solution for complex reasoning tasks.
even more pronounced when compared to ChemAgent'
4.3
HUMAN EVALUATION OF TooL QUALITY
Table 6: Tool quality assessment (%). Example correctness is evaluated only if the function is correct.
Domain
Faithful
Function Correct
Example Correct
Useful
Causality
94
96
Physics
90
94
100
94
Chemistry
96
92
96
90
We conduct human evaluation to assess the quality of created tools along four dimensions. (1) Faithfulness: Whether the tool accurately reflects the source material. (2) Function Correctness:
Whether the tool function meets the tool description and is implemented correctly. (3) Example Correctness: Whether the example solution uses the function properly and returns the right answer. (4) Usefulness: The practical utility of the tool for solving relevant problems without being too narrow. We randomly sample 50 tools for each domain from the toolbox, and ask human expert who has studied corresponding courses to annotate them.
As shown in Table 6 all aspects satisfied by 2 89% tools, indicating that most tools are faithfully derived from the references, correctly implemented, and useful in application. Chemistry tools show slightly lower quality due the domain's complexity. When L.MMs lack foundational knowledge, they may misinterpret nuanced concepts, like mistaking the meaning of a coefficient. We further analyze the alignment between LLM-selected tools and those chosen by domain experts in Appendix
4.4
CASE STUDY
Figure demonstrates case where GPT-4o correctly answers question with REFTOOL. When presented with the causal discovery problem, the model successfully navigates to the relevant category Causal discovery from observational data and selects the appropriate
sorssalddreettionn generating the correct solution code. In contrast, without tool assistance, the model incorrectly uses R- R-squared values infer causal relationships, leading to wrong prediction. The detailed version of the case, along with physics and chemistry cases,is in Appendix D
tool,
5 RELATED WORK
Automatic Tool Creation The automatic creation tools for LLMs aims to overcome the limita- tions of relying solely pre-existing tools. Most works generate tools in code format, while some
8

---

Published as a conference paper at ICLR 2026
Cancal Diraction
Fita lincar model in the causal direction and compute raidials to
Input
test for independencee betwem the input variable and residuds.
Description: The data flow offers contimasss messurenents expression levels multiple phosphouy lated protiins and phospholipid components human immune system colls...
duf cansal, wwrorcioo..ooooooooo fit Y):
1, Y= urray prwshap (-1, 1). sp.arzay (t)
sodel Linrachlerrsszian O (E
restdiill uodel (oddtl.peeteee00
Question: Which cuus-and--ffeee (slatiooship A. pakts473 causes pmeks
llieely?
intependencs sbe flatten (estdraae))
B. pmnck causes 73
[0]3 0.1 Posweiiiieass" "lntwrcupt", inteperdeeneell
C. No causal relat azship exists
zipí
Please
[nodol. insercept... rosidaale intepeshinos)
Program Thoughts (PoT)
PoT + Reffool
O:tmarffeeranatar (tteeaaaa dataly)> y (C (Cpalttntraa, "penst), ('paeit,
Category. Discovery Obseevational Data
'pabta67301311
Selected Tool: Cansal Direction
to2_ dya, e.peediet (datelte))3) Zoo m., (o, y) is canddla, [C (Cpaass078", 'pask
def sointionO: On
"), Crpenicts "pakktattE" []>]
data rad.rasd_com care ('soooscar)))) cav')
peturn 5A" 38 x9(0] xatil slse "g" se 1B(1) > (0)
(cucaal, _directioo, (it(data(s) dataly])
<lse *ge
x, in [(pakes473), 'pock!), ('peok",
"paatted7a1)1>>
Answer: $
The magnitude the R-squared
N. b- e1 e1 Indepppednna ], 72[ raE:Indagendeence return 'A* and not wlae *g* at and not. allw
value does not indicate causality!
(a) Wrong solution when reasoning with PoT
Answer: v
(b) Correct solution when reasoning with PoT RefTool
Figure Example case of GPT-4o with (right) and without (left) REFTOOL.
generate skills or workflows in the format of abstract actions (Wong et al.
2024) or non-executable
text (Wang et al.) (202c)). Existing methods can be broadly categorized into two paradigms: (I)
generating temporary, task-specific tools for individual queries (Qian al. (023), and (2) construct-
ing reusable toolsets based on training or testing data (Cai et al. 2023 Wang et al. 2024b). These
methods have demonstrated success in mathematical reasoning (Qian et al. [023] Cai et al., (023).
visual question answering (Yuan et al., 2024 Wang al.) (024b)) and agent-based tasks (Wang
al. 2025 Zheng al., 2025). Unlike previous works, which primarily rely on LLMs' internal
knowledge, method utilizes external references to create tools, enabling applications beyond the models' inherent knowledge scope.
Tool-Augmented Reasoning Tool-augmented reasoning enhances LLMs' reasoning capabilities by integrating external tools, particularly for tasks requiring specialized knowledge or complex computation. Some studies manually curate a small set of high-quality tools (Gu al.) [2024] Lu
et al. 2025), while others (Qin et al. 2024 Liu et al. 2024b) utilize large-scale APIs from platforms like RapidAPI or API-Bank (Li al. 2023).
However, the large number of tools makes selecting the right one challenging. Prior work often relies on embedding similarity (Qin et al.) 2024 Yuan et al.) 2024), which may fail to capture implicit relationships when the required knowledge not explicitly stated. In contrast, REFTOOL organizes tools within a hierarchical structure that mirrors systematic knowledge organization, enabling effective retrieval. Du al. (2024) adopt related strategy using RapidAPI's categorization for tool selection, whereas REFTOOL proves effective with both inherent structures in reference materials and structures constructed by L.Mss.
CONCLUSION
We present REFTOOL, a framework that enhances LLM reasoning through reference-guided tool creation. Unlike prior approaches that rely solely on models' internal knowledge, REFTOOL generates code-form tools from references such as textbooks and unstructured snippets, validates them through examples, and organizes them into a hierarchical toolbox for effective selection. Experiments across causality, physics, and chemistry domains show consistent improvements over existing tool-creation and domain-specifer reasoning methods, while maintaining computational efficiency. Moreover, REFToOL generalizes beyond scientific domains, showing effectiveness in extremely low-resource language translation. By grounding tool creation and selection in authoritative references, REFTOOL.

---

Published as a conference paper at ICLR 2026
enables LLMs go beyond the limitations of their internal knowledge, yielding accurate and broadly applicable tools. This points to a promising paradigm for extending the knowledge boundaries of LLMs and equipping them to address emerging knowledge-intensiv tasks in real time.
ACKNOWLEDGEMENTS
This work is supported by Beijing Natural Science Foundation (1.253001). We thank Jiuheng Lin and Chen Zhang for suggestions on the experiments. We thank the anonymous reviewers and the area chair for their helpful suggestions.
REFERENCES
Peter William Atkins, Julio De Paula, and James Keeler. Atkins' physical chemistry. Oxford university press, 2023.
Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers. In The Twelfth International Conference on Learning Representations, 2023.
Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen, Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research, 2023a.
Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. Theoremqa: A theorem- driven question answering dataset. In The 2023 Conference on Empirical Methods Natural Language Processing. 2023b.
Yu Du, Fangyun Wei, and Hongyang Zhang. Anytool: Self-reflecivve hierarchical agents for large- scale api calls. In International Conference on Machine Learning, pp. 11812-11829, PMLR, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. ArXiv preprint, Abs/2407.21783, 2024.
Yu Gu, Yiheng Shu, Hao Yu, Xiao Liu, Yuxiao Dong, Jie Tang, Jayanth Srinivasa, Hugo Latapic, and Yu Su. Middleware for lmms: Tools are instrumental for language agents in complex environments. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 7646-7663, 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-40 system card. ArXiv preprint, abs/2410.21126. 2024.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024.
Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. Api-bank: A comprehensive benchmark for tool-augmented Ilms. Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 3102-3116. 2023.
Samual Ling, Jeff Sanny, William Moebs, Gerald Friedman, Stphen D Druger, Alice Kolakowska, David Anderson, Daniel Bowman, Dedra Demaree, Edw Ginsberg, et al. University Physics. OpenStax, 2016.
Xiao Liu, Zirui Wu, Xueqing Wu, Pan Lu, Kai-We Wei Chang, and Yansong Feng. Are lims capable of data-based statistical and causal reasoning? benchmarking advanced quantitative reasoning with data. In Findings of the Association for Computational Linguistics ACL 2024, pp. 9215-9235, 2024a.
10

---

Published as a conference paper at ICLR 2026
Xukun Liu, Zhiyuan Peng, Xiaoyuan Yi, Xing Xie, Lirong Xiang, Yuchen Liu, and Dongkuan Xu. Toolnet: Connecting large language models with massive tools via tool graph. arXiv preprint arXiv:2403.00839, 20244b........
Pan Lu, Baolin Peng. Hao Cheng, Michel Galley, Kai- Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chamelcon: Plug-ande -and-pay compositional reasoning with large language models. Advances in Neural Information Processing Systems, 36:43447-43478 2023.
Pan Lu, Bowen Chen, Sheng Liu, Rahul Thapa, Joseph Boen, and James Zou. Octotools: An agentic framework with extensible tools for complex reasoning. arXiv preprint arXiv:2502. //271, 2025.
Brady Neal. Introduction causal inference, 2020.
OpenAl. Gpt-4 technical report. ArXiv preprint, abs/ 2023.
Siru Ouyang, Zhuosheng Zhang, Bing Yan, Xuan Liu, Yejin Choi, Jiawei Han, and Lianhui Qin. Structured chemistry reasoning with large language models. In International Conference on Machine Learning, pp. 38937-38952. PMLR, 2024.
Xinyu Pang, Ruixin Hong, Zhanke Zhou, Fangrui Lv, Xinwei Yang. Zhilong Liang, Bo Han, and Changshui Zhang. Physies reasoner: Knowledge-augmentt reasoning for solving physics prob- lems with large language models. In Proceedings of the 31st International Conference on Compu- tational Linguistics, pp. 1274--1229, 2025.
Cheng Qian, Chi Han, Yi Fung, Yujia Qin, Zhiyuan Liu, and Heng Ji. Creator: Tool creation for disentangling abstract and concrete reasoning large language models. The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-words apis. In The Twelfth International Conference on Learning Representations, 2024.
Xiangru Tang, Tianyu Hu, Muyang Ye. Yanjun Shao, Xunjian Yin, Siru Ouyang, Wangchunshu Zhou, Pan Lu, Zhuosheng Zhang, Yilun Zhao, al. Chemagent: Self-updating library large language models improves chemical reasoning. arXiv preprint arXiv:2501 200.2550..660000 2025.
Garrett Tanzer, Mirac Suzgun, Eline Visser, Dan Jurafsky, and Luke Melas- Melas-Kyriazii A benchmark for learning to translate a new language from one grammar book. In The Twelfth International Conference on Learning Representations, 2024.
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. ArXiv preprint, abs/2403.05530. 2024.
Xiaoxuan Wang, Ziniu Hu, Pan Lu, Yanqiao Zhu, Jieyu Zhang, Satyen Subramaniam, Arjun R Loomba, Shichang Zhang, Yizhou Sun, and Wei Wang. Scibench: Evaluating college-level scien- tific problem-solving abilities of large language models. In Forty-first International Conference on Machine Learning, 2024a.
Zhiruo Wang, Graham Neubig, and Daniel Fried. Trove: Inducing verifiable and efficient toolboxes for solving programmatic tasks. In International Conference Machine Learning, pp. 51177-51191. PMLR. 2024b.
Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. arXiv preprint arXiv:2409, 2tt.:24499.7422,, 2024c.
Zora Zhiruo Wang, Apurva Gandhi, Graham Neubig, and Daniel Fried. Inducing programmatic skills for agentic tasks. arXiv preprint S5tV::2504.08221, 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of thothhttht prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-2444 24837, 2022.
11

---

Published as a conference paper at ICLR 2026
Lionel Wong, Jiayuan Mao, Pratyusha Sharma, Zachary Siegel, Jiahai Feng, Noa Kornevv, Joshua B Tenenbaum, and Jacob Andreas. Learning adaptive planning representations with natural language guidance. In The Twelfth International Conference Learning Representations, 2024.
An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, 2022.
Lifan Yuan, Yangyi Chen, Xingyao Wang. Yi R Fung, Hao Peng, and Heng Ji. Craft: Customizing Ilms by creating and retrieving from specialized toolsets. In 12th International Conference on Learning Representations, ICLR 2024, 2024.
Beichen Zhang, Kun Zhou, Xilin Wei, Xin Zhao, Jing Sha. Shijin Wang, and Ji-Rong Wen. Evaluat- ing and improving tool-augmented computation-intent inteensvv math reasoning. Advances in Neural Information Processing Systems, 0--233770-35899 2023.
Chen Zhang. Jiuheng Lin, Xiao Liu, Zekai Zhang, and Yansong Feng. Read it in two steps: Translating extremely low-resource languages with code-augmented grammar books. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilchvar (eds.), Proceedings of the 63rd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3977-3997, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 2025 acl long. 202/
doi: 10. 10.18653/v1/2055 1100555l//002...llloooo acl-long.2022 UtLL/
Boyuan Zheng, Michael Y Fatemi, Xiaolong Jin, Zora Zhiruo Wang. Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, et al. Skillweaver: Web agents can self-improve by discovering and honing skills. arXiv preprint (xXXi:25044000775 2025.
A IMPLEMENTATION DETAILS
A. EVALUATION PROTOCOL
We adopt the original benchmarks' evaluation code for consistency. The tolerant rate of numerical questions is 3% for QRData, 4% for Theorem and 5% for SciBench.
A.2 THE REFTOOL FRAMEWORK
By default, each tool's demonstration example included during solution generation. For the causality domain, we omit the example because QRData questions involve data analysis and differ significantly in format from the examples.
The model versions are Llama-3. 1-70B-Instruct and gpt-40-2024-11 11-0. The temperature of models is set to 0. The maximum output tokens are set to 2048 for initial tool generation, refinement, and solution generation, and 512 for hierarchical tool selection. Experiments are conducted on 8 NVIDIA A800 GPUs.
-00pr-0--..------000... 1opt-4-1106-preiiee
A.3 BASELINE METHODS
General reasoning methods: We implement Program-of-Thouughs (PoT) for single-turn reasoning and ReAct for multi turn reasoning. We also experiment with direct reasoning and Chain-of- Thought (Wei 2022) on GPT-4 Appendix B.1 but they are excluded from main comparisons due to inferior performance.
To investigate if LLMs can directly learn from reference materials, we enhance both PoT and ReAct with RAG, using the same reference books employed for tool creation. The books segmented into subsections, and the segment with the highest similarity to the question embedding is retrieved. Reference documents are segmented by subsections. Subsections exceeding 1,000 tokens further divided into 1,000-token chunks.
Reerrevallauugmented generation (RAG) methods:
12

---

Published as a conference paper at ICLR 2026
These segments then processed using the text- 3extt-embedding----aee generate text embeddings. During inference, we compute the embedding for each question and select the text with the highest similarity score to include in the model's prompt.
embedding model to
Ceeneaalppupoos tool creation methods: (1) LATM (Cai et al. 2023 which crafts reusable tools
for each task based on a few demonstrations: (2) Creator (Qian et al. 2023), which dynamically
creates tools for each question; and (3) TroVE (Wang al. 2024b), which
and refines toolbox iteratively during test time. Since LATM requires 6 instances for training and validation, we
randomly sample these from the test set and evaluate on the remaining data, and all other baselines are evaluated in zero-shot setting, For Creator, we provide a tool example from a different domain (math) in the tool creation prompt.
Domain-specific reasoning methods: (1) Physics Reasoner (ang et al. 2025). which man- ually constructs formula set and instructs LLMs retrieve formulas during reasoning; (2) StructChem (Ouyang et al., 2224), which instructs LLMs to generate formulas before reasoning: and (3) ChemAgent (Tang al. (025), which builds library of memories by extensive trial- and-error on validation data. We use GPT-4o for library construction to align with REFTOOL setting. We do not compare with web search methods answers may be searched directly from the Internet.
A.4 THE XLR LANGUAGE TRANSLATION EXPERIMENT
When organizing the rule structure, allow the LLM place rule into multiple categories necessary. This is because some rules cover different aspects simultaneously. For example, the rule In Zhuang, the word "dwg" is copular verb. In simple sentences expressing affirmative judgment, "dwg" is usually omi omi. However, when expressing negation, "dwg" cannot be omi tted, and the negative word "mbouj" must be placed before the copular verb "dwg." (translated from Chinese) is placed under both categories Word Order and Sentence Structure and Negation. To ensure that the rules are fully unstructured in advance, we shuffle the rules before asking the LLM to organize them.
Because the rules are already acompanied with parallel examples, we only ask the LLM create tool function in initial tool generation, and use the parallel examples in tool verification. Since all rules are collected by human experts, the tool verification only to select tools that need further refinement, and we do not filter out any tools after refinement.
In hierarchical selection, we ask the LLM to select at most n. = 4 categories and n, = tools under each category. On average, 3.5 rules are selected for Chinese + Zhuang translation and 3.4 rules for Zhuang + Chinese translation, comparable to the by-ele retrieval methods.
During translation, we provide the word-to-wond dictionary of the source sentence to LLMs in all methods. And each rule is accompanied with 2 parallel examples.
B ADDITIONAL RESULTS
B. PERFORMANCE OF MORE BASELINE METHODS
Table 7: Performance of more baseline methods on GPT-4. Numbers are in percentages (%).
Accuracy Physics
Method
Causality
Chemistry
Direct Reasoning
33.1
14.0
32,8
CoT
41.3
10.5
35.4
PoT
34.2
45.6
51.8
ReAct
50.9
41.2
54.6
Previous works also compare with pure-text baselines like direct reasoning and Chain-of-Thought (CoT) (Wei et al. 2022), but as our preliminary experiment in Table 7 shows that these methods are much inferior to PoT GPT-4, achieves high performance on the causality domain, this results from educated guessing on multiple- choice questions, with none of the numerical questions being answered correctly. This deviates
do not add them into the baselines in the main paper. While CoT
13

---

Published as a conference paper at ICLR 2026
Table Performance comparison with training 1lama-3.1-70B on the reference materials (%)
Accuracy
Method
Causality
Physics
Chemistry
Average
PoT
33.1
48.2
46.9
42.7
PoT (Continued pretrained)
34.6
48.5
PoT (Finetuned on QA pairs)
36.4
50.9
50.6
46.0
PoT RefTool
36.8
53.5
49.5
46.6
from QRData's original goal of conducting data-base quantitative reasoning. Direct reasoning outperforms CoT in physics because, despite the instruction to answer directly, the model still generates intermediate reasoning steps for most questions.
As the code of solving physics and chemistry problems simpler and successful executes in most that necessary in such scenarios, therefore we do implement the
cases, multi-tum reasoning multi-turn settings ReAct and React+REFTOOL. Table 7 also shows that ReAct introduces limited improvement even negative influence these domains.
We also compare Refl with training LLMs based on the content of reference materials, with two methods: continued pretraining and fine-tuning, For continued pretraining. we use the full textbook content. For fine-tuning, we ask GPT-4o to generate question-answer pairs (QA) based on the content of each chapter. GPT-4o is asked generate at most 10 QA pairs each time, and for causality and chemistry we run this process twice to ensure at least 1,000 training instances. We leave 10% of the data for hyperparameter search, and training is performed for 2 epochs with the learning rate le-5.
As shown in Table 8 PoT RefTool achieves the best average performance. Continued pretraining on the textbook text provides only limited improvement, which suggests that L.LMs struggle directly incorporate textbook knowledge for task solving. Fine-tuning on QA pairs narrows the performance gap, although this approach still requires transforming the textbook content through an LLM, and is more issurcceintensive:: Finetuning must be repeated for every target LLM, while the tools created by RefTool can be reused across models.
B.2 SUB-DATASET DATASET PERFORMANCE OF SCIBENCH-CHEMISTRY CHEMISTRY
Table shows the performance sub-datasets of SciBench--hemmitrry While performance varies due to each sub-dataset's small scale, REFTOOL. demonstrates effectiveness in most cases.
B.3 PERFORMANCE REASONING MODELS
Table 10 shows REFTOOL'S performance ol-mini (with the specific version o 2104--2224--1222 where it improves average accuracy by 4.3% over POT. This indi- cates REFTOOL's compatibility with reasoning models, effectively supplementing their knowledge and capabilities.
B.4 PERFORMANCE OF USING DIFFERENT MODELS AS THE TooL CREATION MODEL
To assess the robustness of REFTOOL's 's tool creation module, we experiment with Gemini- 1.5-Pro and Llama-3.1-700 1--ttt-nnttuct as alternative tool creation LLMs. As Table [Tbles]]shows, using these LLMs for tool creation also achieves superior performance compared to baseline methods. This demonstrates REFTOOL's 's robustness to the choice of base model and its compatibility with open-source models, which is particularly important when working with sensitive or proprictary reference materials.
Compared to GPT-4o-created tools, a relatively lower ratio of Gemini- 1.5-Pro and 1lama-3.11-05B Instruct created tools passes the validation. For example, in physics, GPT-4o achieves 82% direct validation success with 8% succeeding after refinement, while Gemini- 1.5-nn .ro-Pro achieves 54% direct success with 24% succeeding after refinement. Although refinement helps recover about one-quarter of tools, approximately 20% still get filtered out, potentially leading to incomplete knowledge coverage and slightly lower overall performance compared to GPT-4o-creeatd tools.
14

---

Published as a conference paper at ICLR 2026
Table Performance of sub-datasets of SciBench-chemistry. Numbers are in percentages (%)
Accuracy Matter
Method
Chemme
Quan
Average
Llama-3.1-70B
LATM
23.4
30.3
31.4
Creator
50.0
34.0
36.4
40.1
TroVE
50.0
44.7
21.2
38,6
StructChem
50.0
21.3
42.4
37.9
ChemAgent
60.5
44.7
39.4
48.2
PoT
65,8
44.7
30.3
46.9
PoT RAG
63.2
44.7
36.4
48.1
PoT REFTOOL
63.2
48.9
36.4
49.5
Demini-1.5-PPP
LATM
28.1
17.0
30.3
25.1
Creator
73.7
63.8
42.4
60.0
TroVE
81.6
63.8
51.5
65.6
StructChem
57.9
38.3
54.5
50.2
ChemAgent
78.9
66.0
51.5
65.5
PoT
78.9
59.6
48.5
62.3
PoT RAG
78.9
63.8
48.5
63.7
PoT REFTOOL
81.6
66.0
51.5
66.4
GPT-4
LATM
53.1
42.6
39.4
45.0
Creator
60.5
46.8
33.3
46.9
TroVE
55.3
31.9
30.3
39.2
StructChem
36.8
19.1
33.3
29.7
ChemAgent
68.4
46.8
42.4
52.5
PoT
68.4
44.7
42.4
51.8
PoT RAG
65.8
51.1
45.5
54.1
PoT REFTOOL
71.1
46.8
42.4
53.4
GPT-4o
LATM
43.8
36.2
27.3
35.7
Creator
55.3
38.3
36.4
43.3
TroVE
71.1
44.7
42.4
52.7
StructChem
55.3
29.8
36.4
40.5
ChemAgent
78.9
55.3
42.4
58.9
78.9
55.3
42.4
58.9
PoT RAG
76.3
51.1
42.4
56.6
PoT REFTOOL
76.3
53.2
54.5
61.3
Table 10: Performance of ol-mini. Numbers in percentages (%), with the best performance for each model shown in bold.
Accuracy Physics
Method
Causality
Chemistry
PoT
44.2
56.1
60.5
PoT REFTOOL
50.2
57.9
65.6
B.5 PERFORMANCE ON ANOTHER PHYSICS DATASET: SCIBENCH-FUND
We evaluated REFTOOL another physics dataset SciBench-fund WWagg al. 2024a) with 71 questions, to test tool generalizability Table 12 shows that REFTOOL outperforms all zero-shot baselines and matches 4-shot Physics Reasoner's performance using the same tools in the evaluation of TheoremQA. This demonstrates REFTooL''s dataset-agnostic nature, where domain-specifie tools can be applied across different datasets.
We excluded two other SeiBench- physies sub-datasets they require advanced thermodynamics and particle dynamics knowledge beyond reference texthook's 's scope.
15

---

Published as a conference paper at ICLR 2026
Table 11:
Performance of REFTOOL using Gemini-1.5-PP and Llama-3.1-70B as the tool creation model. Numbers are in percentages (%), with the best performance for each model shown in bold.
Accuracy Llama-3.1-700 Gemini-
Method
GPT-4
GPT-4o
Average
Physics
Physics Reasoner
48.2
50.9
42.1
33.3
43.4
LATM
38.9
33.3
39.8
30.6
35.7
Creator
40.4
57.0
35.1
40.4
43.2
TroVE
33.3
58.8
35.1
48.2
43.9
PoT
48.2
57.9
45.6
57.0
52.2
PoT RAG
44.7
57.0
44.7
57.9
51.1
PoT+ REFTOOL (GPT-40)
53,5
58.8
57.9
54.8
PoT REFTOOL (GEMINI)
48.2
58.8
47.4
58.8
53.3
PoT + REFTOOL (LLAMA)
49.1
60.5
50.0
57.0
54.2
Chemistry
LATM
31.4
25.1
45.0
35.7
34.3
Creator
40.1
60.0
46.9
43.3
47.6
TroVE
38.6
65.6
39.2
52.7
49.0
StructChem
37.9
50.2
29.7
40.5
39.6
ChemAgent
48.2
65.5
52.5
58.9
56.3
PoT
46,9
62.3
51.8
58.9
55.0
PoT RAG
48.1
63.7
53.7
56.6
55.5
PoT REFTOOL (GPT-40)
49.5
66.4
61.3
57.7
PoT + REFTOOL (GEMINI)
48.3
64.9
52.5
61.6
56.8
PoT REFTOOL (LLAMA)
49.5
62.4
53.2
57.3
55.6
Table 12: Performance another physics dataset: Scibench-fund Numbers are in percentages (%), with the best performance for each model shown in bold.
Accuracy Gemini- 1.aiii---.-P 1,5-Pro
Method
Llama-3.1-70B
GPT-4
GPT-4o
Average
Physics Reasoner
56.3
59.2
63.4
36.6
53.9
Creator
56.3
70.4
53.5
57.7
59.5
PoT
53.5
69.0
59.2
73.2
63.7
PoT+ RAG
54,9
73.2
59.2
73.2
65.1
PoT REFTOOL
57.7
73.2
74.6
67.6
Physics Reasoner (4-shot)
62.0
73.2
63.4
71.8
67.6
B.6 CoST ANALYSIS
Table 13 shows that REFTOOL is highly efficient during inference, compared with all the tool creation and domain-specific reasoning baseline methods. Only LATM costs less than REFTOOL, but its performance is much inferior to REFTOOL. Even when including tool creation costs, REFTOOL remains more efficient than tool-augmented methods. Furthermore, because the tools reusable, the creation cost is amortized and remains fixed regardless of the number of inference instances.
C HUMAN EVALUATION: CONSISTENCY OF TOOL SELECTION WITH HUMANS
We compare human and LLM tool selection by having experts simulate the hicrarchical selection process. For each domain, randomly sample 20 questions where models consistently use tools.
In the tool selection process, given question, annotators are first asked to select at most one category from the given book, and none if no category is relevant the question. If they select category, they are then asked to select one tool within the category if it is the most useful, select two tools only if they are equally useful, and select none if none of the tools useful. For each domain,
16

---

Published as a conference paper at ICLR 2026
Table 13: Cost analysis of tool-augmented and domain-specific methods (with GPT-4o the base model). "Human" indicates that the step is done by humans and the cost is unknown. As Creator and Trove create tools during inference, they do have a seperate toolbox construction cost.
Time (min.)
Cost ($)
Domain
Method
Toolbox Construction
Inference
Toolbox Construction
Inference
LATM
4
0.1
1.2
Creator
59
3.3
Physics
TroVE
22
1.6
Physics Reasoner
Human
75
Human
3.5
PoT REFTOOL
5
2
6.9
1.5
LATM
4
0.1
1.1
Creator
49
2.6
TroVE
52
7.5
Chemistry
StructChem
142
8.7
ChemAgent
1233
536
79.3
41.3
PoT REFTOOL
3
6
3.5
1.4
Table 14: Consistency tool selection with humans (%). Category selection consistency is calculated as the fraction of questions where human and model select the same category. Tool selection consistency is the fraction where their tools overlap, given they both choose tools from the same category.
Domain
Consistency
Llama-3.1-700
Gemini 1.5-Prr
GPT-4
GPT-4o
Category Selection
100
95
100
100
Causality
Tool Selection within Category
94
100
91
94
Category Selection
80
80
75
76
Physics
Tool Selection within Category
53
56
44
69
Category Selection
55
65
72
75
Chemistry
Tool Selection within Category
60
67
40
90
randomly sample 20 questions where most models (at least 3 out of 4) choose to use tools. All human annotators fairly paid.
Consistency Metrics
For category selection, the consistency is computed as:
(human and model select the same category)
Consistency category
[both human and model select category]
And for tool selection, the consistency is computed
Hoverlap exists between tools selected by human and model] Consistency, hool both human and model select tools within the same category
Table 14 shows the consistency between LLMs and human experts. Agreement is higher in category selection than in tool selection, supporting our hierarchical selection step which narrows down the tool search space with high consensus. Across domains, causality shows the strongest consistency in both category and tool selection, owing to more direct questions with keywords like average trealment effect that clearly indicate the relevant knowledge. In contrast, physics and chemistry questions often involve indirect formulations that make harder for models to identify the required knowledge.
Gemini- 1.5-Pro and GPT-4o demonstrate better alignment with human experts, mirroring their superior PoT performance which reflects stronger internal domain knowledge. While Llama-3.1-70B and GPT-4 show weaker consistency with humans in physics and chemistry tool selection, their chosen tools are still valuable relevant knowledge is recalled. In cases where these models select the same category as humans but different tools, we observe 17% accuracy improvement from tool usage compared to the PoT baseline. Appendix D provides concrete example.
17

---

Published as a conference paper at ICLR 2026
Input
Reference Material
Chupte.11 Caroal Dirrovery
Data Desription: livcl shospteriinal
11.25 Faratneirk: anal tlannery
postas and phogialiee composstas hunaa acoans selcss odss..
When we di the in the coeat cirection, rosidants
unthooua dieetton, sariable Wi deptet the resllaal duan dryeadist on X fouarel directonn in Tigue 1130, and we depuct the regiession fiwe we get ue linestty regres or iint-ctues directont in Rigurr
Question Which cuse and pntteneiiin para case press proeicuues paitu77)
line ve gut we incady rumbe 2
PoT RefFool
C mval Eoss stavor viaa. or
Salested Categery Caast Diooney fro Otteratonn
you creser plustig dee 18t arbirzul itbection)
formeal
Dst:
Selected Fook Carak Dosc
Program of Thoughts (PoT)
Solution:
Tool Direction
Bt Incormoed directon ond compate residtaal
coed cant (Die cen')
pakzally anal pisr year
Panation:
cleaat CIATR, Vn
reedil seedd, Liweeeiee pwwpphaaelt poik?
up am arrighs
stain fiie paatutty
dene Trediteee
Flo Jiseur cuprsention cotal
BOMiLA DonicteprereeeD dicilt TD
Clasiige (e and m
ceared goud
(XLA 22 sua wakn A
reaeee palnectt
saing Passiis incresatioe
searal untecioontty
codepecincca o up tpuieiiiinn 0.1. a Thossbold
Crowar:
The magritade of thal aqpansl valus does norindd cate cty
"ieterrrge mocd cesurrighi..
"tedepeniione"" dndepardens
Answen
Wrong solution when reasoning with PoT
(b) Correct solution when reesoning with PoT RefTool
Figure 4: Example case of GPT-4o on a causal problem with (right) and without (left) REFTOOL This is the detailed version of Figure
Reference
Chapter 22 Eletticc Charpss and Fiilds 5.1 Change
Input
Material
Quettios chorpegh itey (p0-2295 ages thery
betawigg and
PoT RafTool
lit perpppaattt wetreeeeeennggn seraarte cheppe the frow gn berooiltone
Selected Categery Chayssood Ridd:
Program Thoughts (PoT)
Selected Teal ieadrobedo
soleices
Tool
Eletrostatic Force
Solution:
Desription Caloa eleetrssddd fome betwen charge sing
Cocoraas lev.
20 all de cotline
beriese glo
Davoda weeetanttaaa www.eetnnnnuoeeeeeoooooo (D) st,
Lrtdeessk 21
ereleaty C2)
chauge
Eesida the fore
(2)
(2la)): distanee betwees the two charges
heve
Sloce bets ippeece ripu, the Sxice la
DesTcRs segsttedo or like atoekeweette foren
deneco diructiv Comadda gi!
conaider the diection
Lapert Dack
(a) Wrong solution when reasoning with PoT.
Dalectove the Focce extog deslonbls Jox
rotaro Sarce
(b) Colrectoolution when with PoT RefTool
Figure 5: Example case of Gemini-1.5-Pro on physical problem with (right) and without (left) REFTOOL.
D CASE STUDY
Figure 4 provides the detailed causality case discussed in $4.4 while Figures 5 and 6|sow physics and chemistry cases. These cases illustrate how REFTOOL helps LLMs solve problems when standard PoT fails. Notably, in Figure 6. while the selected tool doesn't 't directly solve the question, it provides relevant knowledge for the LLM to solve the question in a roundabout way (through the ionization energy of hydrogen), which is different from the expert solution but also leads to the correct answer.
18

---

Published as a conference paper at ICLR 2026
Expert Solution
Reference Chapter Material
spectra
astotsad))
Hlydrooeicc
Dresieaks enttal
The accization energy. I of an deeeent is the minimum en- exy ssganed to nassve dactzon from áz pround state. the stat lowal energy ot one it atoms is tae gas pluse. Becass tie groued stace of tyllogge the saate with n 1, walh exergy R tek, and the stor isstcad when the eloc- tron las leen cuited to Lae levd cortopaeeiing to foo Fig. FA the energy that most sesppiilelss
ayed us lagas (ofk)
000 Ted.e
Dinesticy charpe Do
sicetros soar Ok)
Odiedies redoced mas of peattrestes
Saledaate sodirted lydaacy countin paactonaiaa
j.hR
Calcslate
Fnergy
NLY
Calcuaat the lotiztiation energy of dropeii ston ssag the Rycberg coestest and Flands constar..
Description:
Sipsert alectrornite Kuler 7.1
intias KAA
sed calcclore won.accion aceagy in. 3M4200TT, bed, 02000220000000 24, c*2 20734be10):
Imput
Tarnsatacs:
Question: he postrm bas chorpe and mossegol the decroe Caloditta is di dictooooks the gound-taa polttriin "amm"
RJ (ctoee): The fytherg cocatenn fee tytroger
Eat cossst postzza and doctoo.
0.02028e
(cloen): Speed ot 3ighe It. care (dedesto
Program of Thoughts (PoT)
2.937945x08 mn/e).
Solution:
Reteren: Eleat: The Secizaciee seargy in Jonlee 12).
enllrtion) ckeli
e L. 1000218822-110 clonntiiit shange as
maze
PoT + RefTool
perniriiriyy
buber 15457300-244 24 rebuoed Plante conatent in
Selected Category:
specta
speed
Selected Took Cakxiata Inneation Eneryy
catealate the greend-ase -caaa sowegy of posdeccnics
Solution:
atoca pasttrus has coon scactese,
conisacton ionisation, ce cadculate, sontsatien, mtrige
A.D 100E-p apeiTorc N.aveerd OLe
1: Rugowedd +42g Boler
opeclon,) grownd-
the contantion of kydrogp sbow
valoe, her sugation,
ovcrent the enerpy fcom Dodbes to eiectencncits groudd greeet
1.43317800=19 Elsenctrry dhiige 0 grozzd state searg -ominattim acergy
61
restan ground state J6AE2)
42.7
Wraneeqution
Amswrr:6B
(a) Wrong solution when rea reninn with PoT
(b) solutinn with RefTool
Figure 6: Example case of Llama-3.1 1-7b on a chemical problem with (right) and without (left) REFTOOL.
19

---

Published as a conference paper at ICLR 2026
E THE USE OF LLMS
We use LLMs in polishing the writing, but LLMs do not play a significant role like research idcation or writing the content directly.
F PROMPTS
Figure
Figure demonstrate the prompts of REFTOOL.
Prompts of the general-reasoning baselines shown in Figure 12 and Figure 13 designed with reference to the QRData and SciBench papers. Prompts for ReAct are the same as the QRData paper.
Prompt for Category Generation
Here are rules related to [task). Your task is to identify top-level categories for organizing these rules.
Read through all the rules and determine the major branches of [task] they represent. Ideally there should be about 10-15 top-level categories. One rule can be in multiple categories necessary (at most two).
Output only the names of these categories as JSON list. {segments}
Prompt for Assigning Reference Segments Categories
Here a rule related to [task) and the category names that have been identified:
Categories:
[categories]
Rule: (rule)
Your task is to classify this rule into at most two categories and create a descriptive name for
this rule under each category.
Output in JSON format where:
The keys are the category names that this rule belongs
The value is the name of the rule under this category (a concise, descriptive name) Example output:
{I
"Category 1": "Descriptive rule name for Category
"Category 2": "Descriptive rule name for Category 2"
}}
Note: The rule should be classified into 1-2 categories maximum.
Figure 7: Prompt template for knowledge organization.
Prompt for Initial Tool Generation
Please extract the skills from the following text. The text is a section from the chapter (chapter) of the book (book).
Each skill is python function with comments of parameters and returns, accompanied by description and demonstration example of using the skill.
Please limit the number of skills to 2, and organize the skills in a list of json objects. Please implement the function, and *do not leave it placeholder. Note the indent in code is spaces. All packages used should be imported inside the function. The function should be self-contained.
If the text contains examples, you are encouraged to use the examples in the text, otherwise please design examples by yourself. The answer to the example question is encouraged be numerical.
20

---

Published as a conference paper at ICLR 2026
NOTE THAT THE SKILL PYTHON CODE SHOULD NOT BE SPECIFIC TO/ONLY APPLIED TO THE CHOSEN EXAMPLE! PLEASE GENERATE GENERAL SKILL CODE.
The output should be in *complete" json structure, starting with 'I' and ending with 'J'.
Example output:
[[ "description": "Compute the expected return using the Capital Asset Pricing Model (CAPM)
formula.". ".
"function": """def pepecteedrttun(fff, beta, rm):
V"V"("
Parameters:
rf (float): The risk-free rate.
beta (float): The beta of the portfolio.
rm (float): The return on the market.
Returns:
float: The expected return.
\"V"\"
return rf + beta (rm rf)""".
"example": {
"question": "Suppose stock has the following information. It is listed the London stock
exchange and operates throughout Europe. The yield on UK 10 year treasury is 2.8%. The stock in question will earn 8.6% per historical data. The Beta for the stock is 1.4, i.e., is 140% volatile to the changes in the general stock market. What is the expected rate of return?".
"solution": """def solution():
# Given values.
rf = 0.028 # The yield on UK 10 year treasury beta 1.4 # The stock is 140% volatile to the changes in the general stock market r 0.086 # The stock in question will earn 8.6% as historical data Calculate the expected return.
result expected expected_returm(rf, beta, rm) Return the result.
return result""".
"answer": 0.109
)]]
Text: [text)
Figure 8: Prompt template for initial tool generation.
Prompt for Tool Refinement
Please revise the skill according to the feedback.
The skill is python function with comments of parameters and returns, accompanied by description and demonstration example of using the skill. Please try to keep the original to address feedback. Note the indent in code is 4 spaces. All packages used should be imported inside the function. The function should be self-contained. The answer to the example question encouraged to be numerical.
intent of the skill, and modify the description/funntioneexaag
NOTE THAT THE SKILL PYTHON CODE SHOULD NOT BE SPECIFIC TO/ONLY APPLIED TO THE CHOSEN EXAMPLE! PLEASE GENERATE GENERAL SKILL CODE.
The output should be in *complete* json structure as the original skill, starting with '(" and
21

---

Published as a conference paper at ICLR 2026
ending with "
Original Skill: (skill)
Feedback: [feedback)
Figure 9: Prompt template for tool refinement.
Prompt for Category Selection
You are a data analyst and good at quantitative reasoning. You are required to respond to quantitative question using the provided data.
The question can be found below. Given the table content of the book (book), please select the chapters that you find useful in solving the question.
Please provide an explanation supporting your choice. At the last line of your response, format the number of the chapters with a list, like '[O]'. Limit the number of chapters to most 1. Output '[]" if none of the chapters are useful. The last line should start with '[" and end with 'I'
Question: {question]
Table of Content:
Response:
Prompt for Tool Selection within Category
You data analyst and good at quantitative reasoning. You required to respond to quantitative question.
The question and the list of skills can be found below. Please select the skills that you find useful in solving the question
Please provide an explanation supporting your choice. At the last line of your response, format the number of the skills with a list, like '[O]'. Limit the number of skills to at most Output '[I' if none of the skills useful. The last line should start with and end with 'J'
Question: {question]
List of skills: [tools)
Response:
Figure 10: Prompt template for tool selection.
22

---

Published as a conference paper at ICLR 2026
Prompt for Solution Generation
You are a data analyst and good at quantitative reasoning. You are required to respond to quantitative question below. Please write python code to answer the question. Please encase the Python code within triple backticks. You can use any python library you imported. The returned value of the code is supposed to be the answer. The format of the code should be "'yython
def solution():
import libraries needed
write code to get the answer
# return answer
Question: {question]
Please note that provide you several functions for the above question. If the functions are related to the question, you are encouraged to use the functions to solve the question. The functions will also be provided in execution, just call them. *DO NOT* define the functions again or import the functions.
Functions: [tools)
Response:
Tool Template
Function Description: !description]
Function: (function)
Example Question: coxample_question
Example Solution: )
Figure 11: Prompt template for solution generation. For evaluation of QRData, the data description and ten lines of the shuffled data are also added to the prompt along with the question.
23

---

Published as a conference paper at ICLR 2026
Prompt for PoT
You are a data analyst and good at quantitative reasoning. You are required to respond to quantitative question below. Please write python code to answer the question. Please encase the Python code within triple backticks. You can use any python library you imported. The returned value of the code is supposed to be the answer. The format of the code should be "'yython
def solution():
import libraries needed
write code to get the answer
# return answer
Question: {question]
Response:
Figure 12: Prompt template for PoTT. For evaluation of QRData, the data description and ten lines of the shuffled data also added to the prompt along with the question.
Prompt for CoT
You are a data analyst and good at quantitative reasoning. You are required to respond to quantitative question below. Please provide clear and step-by-step solution to answer the question. Do not write any code in your answer. Conclude the answer by stating "The answer therefore \boxed (boxed [ANSSER]].
Question: {question] }
Response:
Prompt for Direct Reasoning
You are data analyst and good at quantitative reasoning. You are required respond to a quantitative question below. Directly answer by stating "The answer is therefore (bAxee([ANSWER):
Question:
{question]
Response:
Figure 13: Prompt templates for CoT and direct reasoning. For evaluation of QRData, the content of the data (shuffled and truncated the first 3500 tokens) is also added the prompt along with the question. For evaluation of SciBench, the prompt also states "The question will specify the unit of measurement, which should not be included in the answer. Express the final answer as decimal number with three digits after the decimal point."
24


Reviews:
[{"review_id": "AmydTHoUo4", "reviewer": "Reviewer_M5bi", "summary": "The paper proposes a novel framework that enables large language models (LLMs) to create and use their own tools by leveraging external reference materials (e.g. textbooks, knowledge documents). The approach is motivated by the observation that many complex tasks (causal reasoning, physics, chemistry, etc.) require domain knowledge that may not be encoded in the LLM’s parameters. Unlike prior tool-using paradigms, which rely purely on the model’s internal knowledge or a fixed set of APIs, REFTOOL “grounds” the tool creation process in external references.", "questions": "1.\tThe article's innovation seems extremely limited relative to existing work [1], essentially just pre-generating tools in a RAG-like manner. What do the authors believe is the innovative contribution of their research?\n[1] Cai, T., Wang, X., Ma, T., Chen, X., & Zhou, D. Large Language Models as Tool Makers. In The Twelfth International Conference on Learning Representations.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1.\tEnabling LLMs to automatically create executable tools from unstructured text references is more efficient than traditional text retrieval. This structured generation approach (including pseudocode and code templates) is more effective than simple text retrieval. By transforming textbooks into a toolbox, REFTOOL empowers models to solve problems previously beyond their capabilities.\n2.\tREFTOOL achieves significant performance improvements on knowledge-intensive benchmarks like causal reasoning, physics, and chemistry, increasing accuracy by 12.3% compared to existing tool-generation methods.", "weaknesses": "1.\tThe method currently relies on GPT-4 (or similar large models) for tool generation and verification. While understandable (tool synthesis is difficult), this means the initial setup can be expensive and dependent on proprietary models. Without GPT-4 or similar high-performance models, the quality of the generated tools may decline. This introduces an unfairness in experimental baseline comparisons. The article does not deeply explore using open-source LLMs for tool creation; it would be worth knowing how a 70B open-source model performs on this task, even if some performance loss is expected.\n2.\tAlthough overall tool quality is high, the generated tools might be relatively unreliable in some highly complex domains (like chemistry). For instance, chemical reasoning may require extremely complex domain knowledge, and even with textbooks and GPT-4, generating completely correct code isn't guaranteed. The article notes that chemistry tools received relatively low scores in human evaluation. Therefore, for highly specialized or obscure knowledge, the method might require additional fine-tuning (perhaps incorporating symbolic systems or domain expert review).\n3.\tREFTOOL generates hundreds of tools per textbook (e.g., over 500 for physics). Storing and managing such a large toolbox could become overwhelming, especially when considering broader knowledge bases or multiple textbooks. Although hierarchical selection helps, it can become quite complex – for example, if a category still contains dozens of tools, providing all their descriptions to the LLM might exceed its context window limit.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0kxYmj5kfl", "reviewer": "Reviewer_sMaQ", "summary": "REFTOOL introduces a reference-guided framework that enables large language models (LLMs) to automatically create and use tools grounded in external materials (e.g., textbooks, knowledge snippets) rather than relying solely on internal knowledge. This allows LLMs to tackle knowledge-intensive reasoning tasks across domains where pre-defined tools or sufficient prior knowledge are unavailable.\n- LLMs generate executable code tools from reference materials (structured like textbooks or unstructured like text snippets). Each tool includes a natural-language description, Python implementation, and example usage. Tools are validated and refined automatically using example-based execution checks.\n- Tools are organized into a hierarchical toolbox reflecting the reference’s knowledge structure. During inference, LLMs hierarchically select relevant categories and tools to solve input problems. Integrates with reasoning paradigms like Program-of-Thoughts (PoT) and ReAct, allowing graceful fallback when no tools apply.\n- Benchmarked on causality (QRData), physics (TheoremQA), and chemistry (SciBench). Outperforms state-of-the-art tool-creation methods and domain-specific reasoning systems (e.g., Physics Reasoner, ChemAgent) by ~12–13% average accuracy across domains. Achieves dataset-agnostic generalization - tools built from one dataset transfer effectively to others in the same domain. Extends to non-scientific tasks, such as extremely low-resource language translation, yielding a +10 BLEU improvement.\n- Reduces computational cost and inference time by up to 99% compared to domain-specific tool systems. Human evaluation shows ≥90% correctness and usefulness of generated tools.", "questions": "covered in the weaknesses above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper introduces a new perspective on tool creation shifting from internally generated tools (as in Creator, TroVE, etc.) to reference-guided tool generation grounded in external textual materials. This is elegant and underexplored.\n- The approach generalizes to multiple datasets within the domain and also to areas beyond scientific reasoning.\n- The experiments are extensive, spanning multiple scientific domains and extending to non-scientific domains. The evaluation is robust and covers tool reuse across datasets, ablations, alternative base models for tool creation, cost analysis, and human evaluation.\n- The improvements (~12–13%) are consistent and statistically significant.\n- The paper is well-written and logically structured, has illustrative figures and examples.\n- This work contributes a scalable and general paradigm for extending LLMs’ reasoning ability beyond internal knowledge via reference-grounded code tools. It has clear implications for scientific reasoning, educational AI, and self-improving agents.", "weaknesses": "- the validation of generated tools is based on available examples or generation of appropriate examples. It is unclear how one would ensure the correctness of the generated examples and their solution.\n- the choice of GPT-4o for tool creation, the number of tools and categories is unclear - what is the basis for these choices?\n- While causality, physics, and chemistry benchmarks are informative, they are narrow in scope and format (mostly numerical or formula-based). The framework’s generality would be more convincing if tested on applied or multi-modal reasoning tasks.\n- Although the authors claim REFTOOL can extend LLM knowledge boundaries, all experiments use well-bounded domains with relatively small references. It remains unclear whether this approach scales when references are large, diverse, or ambiguous (e.g., thousands of documents).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vc14PAf4EJ", "reviewer": "Reviewer_m8Qh", "summary": "This paper presents REFTOOL, a framework that automatically builds and uses code-based tools for knowledge-intensive reasoning. It works in two stages: (1) tool creation, where an LLM reads reference materials, generates and validates functions, and organizes them into a hierarchical toolbox; and (2) tool use, where the LLM selects and applies these tools to solve new problems. Experiments in causality, physics, and chemistry show that REFTOOL outperforms existing tool-creation and domain-specific methods, and it also generalizes well to a low-resource language translation task.", "questions": "The hierarchical selection is a critical component. Could you provide metrics on the accuracy of the category and tool selection steps themselves, perhaps against an oracle or human judgment? What is the failure rate at these stages?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Principled approach to grounding tool generation in external, authoritative sources. It is a well-motivated and logical solution for specialized domains.\n2. The introduction of a hierarchical toolbox structure that mirrors the organization of the reference material is a strong design choice.\n3. Demonstrated generalization.", "weaknesses": "1. The framework's performance depends on the quality and structure of the reference material. It assumes a well-organized, comprehensive textbook with content that can be easily split into discrete functions. How does the framework cope with noisy, theoretical, or poorly structured references?\n2. The core components are incremental and miss the core baseline, such as fine-tuning the base LLM directly on textbook content. It is unclear whether the explicit, complex, and potentially brittle tool-creation pipeline is truly superior to simply teaching the model the domain knowledge through continued pre-training or fine-tuning.\n3. Human evaluation of tool quality uses a small sample—just 20 tools per domain—making it hard to generalize across hundreds created. While REFTOOL outperforms baselines, the gains are often modest. For example, chemistry scores improve by only 2% over POT + RAG (Table 2), questioning the complexity of the pipeline. Also, Table 5's cost analysis omits the considerable manual and computational effort required for reference curation and initial tool generation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9VVvDyyJU7", "reviewer": "Reviewer_PQMZ", "summary": "This paper introduces REFTOOL, a framework for tool-augmented reasoning designed to enhance the inferential capabilities of  LLMs in knowledge-intensive domains that lack predefined tools. REFTOOL leverages external materials to perform hierarchical tool creation, operating through two primary modules:\n\nTool Creation: The LLM processes reference content to generate executable tools. Each tool is generated with a description, its function code, and an illustrative example. These tools are subsequently validated using their own examples and optimized as necessary. Finally, the validated tools are organized into a hierarchical toolbox that generally mirrors the structure of the reference material.\n\nTool Utilization: During the reasoning process, when a query is encountered, the LLM executes a hierarchical selection procedure. It will select the most relevant category and subsequently selects the most appropriate tool from within that category. The selected tool is then employed to generate the final solution.\n\nExperiments were conducted on causality, physics, and chemistry benchmarks. The results demonstrate that REFTOOL's mean accuracy significantly outperforms existing tool-creation methods, domain-specific models, and RAG-based approaches by an average of 12.3%. The paper also shows that the framework is cost-effective and exhibits high generalizability, having been successfully applied to non-scientific tasks using unstructured reference materials.", "questions": "This paper's core premise is that REFTOOL can overcome an LLM's internal knowledge limitations by leveraging reference materials. However, in specialized domains like chemistry, reference materials often assume a baseline of foundational knowledge rather than explicitly teaching it. For instance, a textbook may freely use representations like SMILES strings or IUPAC nomenclature in its examples and formulas, assuming the reader is already proficient.\n\nDuring Tool Creation: If the base LLM lacks reliable pre-trained knowledge of (e.g.) SMILES, it may fundamentally misinterpret a textbook example and thus generate a semantically flawed tool. The Tool Verification step, which relies on a model-generated example, could also fail to catch this error if the model produces a self-consistent but incorrect validation case, leading to the propagation of a bad tool.\n\nDuring Tool Utilization: Even if a tool is perfectly created (e.g., calculate_property(smiles_string)), it becomes practically useless if the LLM, when given a natural language query (\"calculate the boiling point of ethanol\"), cannot perform the implicit prerequisite task of translating \"ethanol\" into its required SMILES format (\"CCO\"). This adds an additional reasoning burden that the framework was intended to solve, not create.\n\nDoes the REFTOOL architecture possess a specific design or mechanism to address this gap in implicit foundational knowledge?\n\nFor example, does the framework (a) simply rely on the comprehensiveness of the reference material to also teach these fundamentals, or (b) does it include a more active mechanism to detect these dependencies and recursively build a foundational library of prerequisite tools (e.g., a chemical_name_to_smiles converter) that the higher-level scientific tools can then rely upon?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Novel Framework for Knowledge Proceduralization: The paper introduces a novel paradigm that bridges declarative knowledge retrieval with executable tool use. By compiling procedural knowledge from static reference materials into a persistent, validated, and structured toolbox, the framework (REFTOOL) presents a significant conceptual advance over standard Retrieval-Augmented Generation (RAG) and pre-defined tool-use methodologies.\n\n2. Robust Methodological Design and Validation: The framework's two-module architecture is methodologically sound, incorporating critical components for robustness, such as automated tool verification, refinement loops , and a hierarchical selection mechanism. The efficacy of these design choices is rigorously validated through comprehensive ablation studies, which demonstrate the clear superiority of code-based tools over raw text retrieval and hierarchical selection over standard similarity-based retrieval.\n\n3. Rigorous Experiments and Generalizability Validation:  The effectiveness of REFTOOL is substantiated through extensive experiments across three distinct, knowledge-intensive scientific domains, showing consistent and significant performance gains over a wide array of strong baselines. Furthermore, the framework's generalizability is impressively demonstrated by its successful application to a non-scientific task using unstructured reference materials, highlighting its flexibility and broad applicability.", "weaknesses": "1. Dependency on the Capability of the Base Model for Tool Creation: The core function of the REFTOOL framework relies on a specific base model to generate high-quality and validated tools. However, the framework's performance varies significantly under different base models. Supplemental experiments in Appendix B.4 show that other models, such as Gemini-1.5-Pro and Llama-3.1-70B, have much lower initial validation success rates. For example, in the physics domain, the success rate was 54% versus 82%.\n\n2. Insufficient handling of prerequisite foundational knowledge. The design of the framework implicitly assumes that the knowledge in reference materials is entirely comprehensible. However, in specialized fields such as chemistry, textbooks often presuppose that readers have mastered undefined foundational knowledge (e.g., IUPAC nomenclature, SMILES notation). If the tool creation model misinterprets these prerequisite foundational concepts, the model may fail to generate correct tools.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)