ToolQA: A Dataset for LLM Question Answering with External Tools
Authors: Yuchen Zhuang
Yue Yu
Kuan Wang
Haotian Sun
Chao Zhang
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: Large Language Model
External Knowledge
Tool-Augmetned LLMs

Abstract: Large Language Models (LLMs) have demonstrated impressive performance in various NLP tasks, but they still suffer from challenges such as hallucination and weak numerical reasoning. To overcome these challenges, external tools can be used to enhance LLMs' question-answering abilities. However, current evaluation methods do not distinguish between questions that can be answered using LLMs' internal knowledge and those that require external information through tool use. To address this issue, we introduce a new dataset called ToolQA, which is designed to faithfully evaluate LLMs' ability to use external tools for question answering. Our development of ToolQA involved a scalable, automated process for dataset curation, along with 13 specialized tools designed for interaction with external knowledge in order to answer questions. Importantly, we strive to minimize the overlap between our benchmark data and LLMs' pre-training data, enabling a more precise evaluation of LLMs' tool-use reasoning abilities. We conducted an in-depth diagnosis of existing tool-use LLMs to highlight their strengths, weaknesses, and potential improvements. Our findings set a new benchmark for evaluating LLMs and suggest new directions for future advancements. Our data and code are freely available for the broader scientific community on GitHub.

ToolQA: A Dataset for LLM Question Answering with External Tools
Yuchen Zhuang, Yue Yu², Kuan Wang', Haotian Sun, Chao Zhang College of Computing, Georgia Institute of Technology. Atlanta GA tyezhuang, yueyu, kuanwang, haotian. sun, chaozhang)@gatech.c edu
Abstract
Large Language Models (LLMs) have demonstrated impressive performance in various NLP tasks, but they still suffer from challenges such as hallucination and weak numerical reasoning. To overcome these challenges, external tools be used to enhance LLMs' question question-answerring answering abilities. However, current evaluation methods do not distinguish between questions that can be answered using LLMs' internal knowledge and those that require external information through tool use. To address this issue, introduce to faithfully eva LLMss ability to use external tools for question answering. Our development of TooIQA involved a scalable, automated process for dataset curation, along with 13 specialized tools designed for interaction with external knowledge in order answer questions. Importantly, we strive minimize the overlap between our benchmark data and LLMs' vee-trannig data, enabling a more precise evaluation of LLMs' tool-use reasoning abilities. We conducted an in-depth diagnosis of existing tool-use LLMs to highlight their strengths, weaknesses, and potential improvements. Our findings set a new benchmark for evaluating LLMs and suggest new directions for future advancements. Our data and code are freely available for the broader scientific community on GitHub 2
dataset called ToolQA, which is designed
Introduction
Large Language Models (LLMs) have demonstrated superior performance in myriad of NLP tasks [4, 9, 43, 42, 55, 61]. These models have captured vast amounts knowledge from enormous and diverse corpora during pre-training. After instruction fine-tuning [10, 44, 2], they have demonstrated impressive capabilities in information-seeking question answering [65, 26). Despite their remarkable performance, LLMs face several challenges. For example, they susceptible to hallucinations generating plausible yet ungrounded information which can mislead users and affect content integrity [66, 19, 5]. Additionally, they exhibit weaknesses in numerical reasoning, essential skill in numerous real-life applications [ [14, 36, 41, 28, 51, 13]. These limitations highlight the need for techniques that can enhance LLMs' question-answeringg abilities.
Recent research has shown that these issues can be mitigated by augmenting LLMs with external tools, such as retrieval augmentation [58, 17], math tools [56, 76, 32), and code interpreters [13, 63]. For example, a Wolfram math plugin can enhance numerical reasoning [68], and verified database can mitigate hallucinations by providing up-to-date fact-checked knowledge [49]. However, existing evaluation methodologies struggle to distinguish whether the model simply recalling pre-trained information or truly utilizing external tools for problem-solving [37]. This challenge arises, in part, because the external data used for evaluation may have already been exposed to LLMs during the pre-training phase [53]. This exposure can lead to biased evaluation of LLMs' tool-use abilities, as the models could just use their ingrained knowledge and their reasoning abilities, bypassing the use of external tools. As result, these evaluations cannot accurately reflect the true competency of the
"These authors contributed equally to this work. "httpa://ge thub. com/night http:/niigh-chhe//Tootoa chen/ToolQA
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

Kobe Bryant was drafted by Charlotte Hornets with
Question: What did Kobe Bryant start his NBA with??
Retrieve
the 13th pick of the 1996
draft, but his draft rights were immeddaately traded to Los Angeles Lakers...
the montenee
waay
Pro-train
Reasoning with retrieval
Directly input
Using tools or only memorizing?
L.M with implicit knowledge
Los Angeles Lakens
Los Angeles Lakers
Figure 1:
Pre-trained on vast range of corpus, LLMs possess extensive knowledge, which may overlap with evaluation data. This overlap poses significant challenge to current evaluation methods, as it becomes difficult to discern whether the model is merely recalling pre-trained information or genuinely employing external tools for mroblem-solvingg
models. We need fair and explicit way to check if LLMs are really good at problem-solving with tools or if they are just using their memorized information.
To fill this gap, we introduce ToolQA, a question answering (QA) benchmark to evaluate LLMs' ability in using external tools for answering questions. ToolQA comprises data from 8 domains and defines 13 types of tools to acquire information from external reference corpora. Each instance in ToolQA consists of question, an answer, reference corpora, and list of available tools. ToolQA is unique in that all its questions can be answered only by using appropriate tools to obtain information from the reference corpus. This minimizes the possibility of LLMs answering questions by merely recalling their internal knowledge, and allows for faithfully evaluating LLMs' abilities in using tools.
ToolQA is curated with automated three-phase process: (1) The first phase, Reference Data Collection, involves gathering various types of public corpora including text, tables, and graphs from different domains. These corpora have overlap with the LLM pre-training data and will serve as reference corpora for tool-based question answering.
(2) The second phase is Human- guidedd Question Generation with LLMs. In this phase, we generate questions that can only be answered by using tools over the reference corpora. Our approach is lemplate-bused question generation process, which includes noman-guuiden template generation, template validation, and question instantiation with tool attributes. (3) The third phase is Programmatic Answer Generation. This phase produces accurate answers for the generated questions. To ensure answer correctness, we implement operators corresponding to the tools and obtain answers from the reference corpora programmatically. Our three-phase procedure ensures that we generate questions that can only be answered using external knowledge, along with their precise answers. Additionally, the process is highly efficient and requires minimal human labeling efforts.
We conducted experiments using both standard LLMs and tool-augmented LLMs to answer questions in TooIQA Our findings indicate that ChatGPT and thain-of-thoughee prompting [65], which rely solely on their internal knowledge, have low success rates of approximately 5% for easy questions and 2% for hard questions. In contrast, tool-augmented LLMs such as Chameleon [32] and ReAct [76] perform better by leveraging external tools. For easy questions, the best performance achieved by tool-augmented LLMs is 45.15% while for hard questions, the best performance drops to 8.2%.
Our results and error analysis demonstrate that ToolQA is challenging benchmark for existing tool-augmented LLM methods, especially for its hard questions that require more complex reasoning about tool composition.
2 Related Work
2.1 Knowledge-Augmentee LLMs
Several prior works aim to enhance LLMs with explicit external knowledge. Specifically, one line of research focus on retrievul-aunmmented languuge models [58, 3, 17, 27, 30, 80, 34, 72], where they use sparse [54] or dense retrieval [22, 16] to extract relevant knowledge from the corpus. These works mainly focus on leveraging free text, without considering multiple types of tools for task solving. On the other hand, Program-of-Though [6], PAL [13], MathPrompt [15], and

---

Private/Commercill Data
General Knowledge
Out-Dated Information
Professional Abilities
Internal Knowledge
Publicly Available Data
External Knowledge
Most Recent Data
(a) Reference Data Collection
Question
Question
External Knowledge
Human-Guided Question Generation
Programmatic Generation
Data
Q: (Odii())gg to
(Dest)
(Date)
(Date)..divertee ..diver
LAX
SFO
10/15/22
Question
ATL
01/09/22
Templates
CLT
MDW
05/25/22
Terplates:
Did the flight from (Origin) to (Dest): on (Date) get cancelled or diverted? (External Knowledge) Mhat the flight-distance
A: def question_ question_gen(table gen(table row):
Origin cobllerow[""nti
distance fhe-the-------- flight from (arigin) (n----nnn-aa (Internal Knowledge) X prich-praduct- on (Flightlunber)) has the tee-ttte-ttteeee häghest pesoe7-(Not Mentioned)
Dest table now[ FlightDate table_ parte__oww""Fae" rou("=lighttate""
return question,
Figure 2: TooIQA, aiming to faithfully evaluate LLMs' abilities to use external tools, curates data through three phases: (a) Reference Data Collection; (b) Human-Guided Question Generation; and (c) Programmatic Answer Generation.
Code4Strucc [63] apply code-based tools to enhance LLMs' abilities in question answering with a focus tabular and math-related tasks. Several additional works [56, 32, 57] expand the scope of tool utilization by incorporating different types of basic tools (e.g. calculator, calendar, machine translation) solve complex reasoning tasks. To synergize different functional tools together for problem-solving, LLMs must have advanced planning and memory capabilities. In terms of planning, current methods either enable LLMs autonomously break down complex tasks into intermediate reasoning steps [45, 24, 47, 50, 65, 75, 31, 76). or encourage LLMs to self-reflect the previous decisions with environmental feedback [39, 76, 35. 8]. Memory capabilities, on the other hand, provide LLMs with opportunities to learn and adapt based on past experiences, whether successes or failures [62]. In addition, several works have extended this line of learning paradigm other modalities [73, 69] and other domains [20]. Concurrent to our work, there are also several studies [70, 37] that investigate the parametric and nonparametric knowledge from LLMs. detailed comparison between existing tool-use LLMs can be found in Appendix A.
2.2 Benchmarks Tool-Augmented LLMs
Earlier tool-augmented LLMs primarily assess single tool usage based downstream task perfor- mance across existing benchmarks. For example, there are works that study how text retrievers augment LLMs' performance on open-domain question-answering [21, 74], fact-checking [60], and timely information benchmarks [7, 23, 78, 12]. Besides, the mathematical reasoning abili- evaluated using computation-intensis QA datasets [11, 33]. However, these evaluation benchmarks may not faithfully reflect the extent to which models leverage external tools, as some questions could still be correctly answered solely using the internal knowledge of the LLMs. ToolQA attempts to mitigate these issues by selecting data from out-of-scope sources that have been memorized by LLMs. Concurrent with work, there are several recent benchmarks for evaluating LLMs' ability in using multiple tools for solving challenging tasks, including API-Bank [29], APIBench [48], and ToolBench [52, 71]. They mainly focus on constructing high-qualin tool chains for LLM fine-tuning and evaluating API call trace accuracy against a fixed ground truth trace. contrast, ToolQA is unique in that it focuses on open-ended use of tools for question-answeringg rather than benchmarking the intermediate process of tool use. Specifically, ToolQA creates tool-based question-answer pairs and assesses whether LLMs arrive at the correct answer, regardless of the tool chains used.
ties of external calculators and Python interpreters

---

3
ToolQA Dataset
3.1 Dataset Details
We curate the ToolQA benchmark evaluate LLMs' capability in leveraging external tools for question answering. ToolQA consists of data from 8 distinct domains, each instance being tuple (question, answer, reference corpora, and tools). The reference corpora are external knowledge sources that can be queried, which can be a text corpus, tabular database, or graph. To enable obtaining information from the reference corpora, we have developed 13 tools for text retrieval, database operations, code interpretation, mathematical computations, and more. The questions are designed to simulate real-world information-seeking inquiries. However, they cannot be answered directly with L.MSs internal knowledge, but instead require LLMSs to obtain information from the reference corpora via tool use. Table shows the detailed statistics of TooIQA
To reduce human efforts in generating faithful question-answe pairs to evaluate LLMs' tool-use capabilities, we propose an automatic three- phree-phase process (Figure 2): (1) We first select data from public sources that are unmemorized by LLMs during Reference Data Collection; (2) We adopt Human- Human-Guided Question Generation steer LLMs to generate valid questions according to pre- defined templates; (3) We produce accurate answers for the generated questions with Programmalic Answer Generation. We detail the three-phase generation process the following.
3.2 Reference Data and Tools
To evaluate LLMs' ability in using external tools for question answering, it is crucial to ensure that they cannot directly answer the questions with their internal knowledge. To this end, we collect reference corpora that meet the following criteria (Figure 2(a)): 1) The reference corpora should ideally not overlap with the LLM's pre-training data; 2) The reference corpora should contain context-sensitive facts for generating questions that cannot be directly answered solely based on LLMs' internal knowledge and reasoning abilities; 3) LLMs should be able to obtain all the necessary information from the reference corpora to correctly answer the questions.
Based on these criteria, we define 6 contextual dimensions: temporal, spatial, social, scientific, mathematical, and personal. We collect reference corpora that can yield context-specific questions along one or more the 6 dimensions. Specifically: 1) Along the temporal dimension, we collect the Fli Flights and Coffee corpora, which contain the latest information that is out of the temporal scope of the LLM's pre-training data. 2) Along the spatial dimension, we collect Yelp and Airbub, which two non-text corpora that can yield questions with spatial contexts. 3) Along the mathematical dimension, we collect the questions from GSMBK that ChatGPT cannot answer correctly with its own mathematical reasoning ability; 4) SciREX emphasizes detailed model performances from the scientific domain [18], where GPT family models can easily hallucinate [42]. 5) To incorporate personal data and avoid privacy issues, we synthesize the personal Agenda corpus with ChatGPT with virtual names and events. 6) In addition, we also select data from the most recent DBLP database and create graphs between authors and papers, where social relational knowledge cannot be understood by LLMs currently. Further details can be found in Appendix B.
To obtain information from these reference corpora, we design 13 tools that are available to the L.LMs (Table 2). These tools are designed as follows:
Table 1: Dataset Statistics of ToolQA
External Knowledge
Easy
Hard
Context
Topic
Format
Size
# Templates
Questions # Templates
Questions
Flight
Tabular Database
4078318
10
100
10
100
Temporal
Coffee
Tabular Database
5746
100
130
Yelp
Tabular Database
150346
11
100
10
100
Spatial
Airbub
Tabular Database
102599
10
100
10
100
Mathematical
GSM8K
Professional Ability
100
Social
DBLP
Graph
553320
10
100
10
100
Scientific
SciREX
Pure-Text Corpus
438
100
100
Personal
Agenda
Pure-Text Corpus
10000
100
5
100
SUM
800
730

---

Table 2: Different tools in ToolQA
Tool Types
Tools
Tools
Text Tools
2
Agenda Retriever, SciREX Retriever
Dutabuse Tools
3
Databuse Loader, Data Filter, Get Value
Math Tools
1
WolframAlpha Calculator
Graph Tools
4
Graph Loader, Neighbour Checker, Node Checker, Edge Checker
Code Tools
2
Python Interpreter. SQL Interpreter Finish
System Tools
1
Text: AgendaRetriever and SciREXRetreiver are text retrieval tools. They can retrieve relevant information to given query from the (synthesized) personal agenda corpus and scientific corpus. Database: Dutabase Loader loads data from the local tabular Database. Data Filter can filter the database according to a set of conditions, each of which composed of column name, relation, and pre-determined value (e.g.. "Date=2022-10- 11-1""... Get Value returns all the values under a certain column in the database.
Math: Calculator is a mathematical tool that treats the input string as a formula and calculates the corresponding result. We the Wolfram API portal the calculator 3, which can perform both simple computations (e.g., add, subtraction, multiplication) and complicated operations (e.g., averaging. finding maximum values).
Graph: Graph Loader loads the graph from local files for future operations. Neighbour Checker
lists all the neighbors of the query node in the graph. Node Checker and Edge Checker return the detailed attribute information of the query node and edge, respectively.
Code: The SQL. Interpreter and the Python Interpreter are responsible for interpreting and executing SQL commands and Python code, respectively. They can receive and transform data from other tools, serving as bridges between different tools and the LLM.
System: Finish parses the feedback from execution and returns the answer to finish the task.
3.3 Human-Guided Question Generation
The question generation phase aims to generate questions that can be answered by using the available tools over the reference corpora. There are two straightforward strategies to generate questions: 1) letting human experts come up with questions about reference corpora, or 2) relying solely on LLMs generate questions about the reference corpora. However, both strategies have their drawbacks. While human experts can produce high-quality questions, the entire process is labor- intor-ittensiee, time- consuming, and hard scale. Depending solely on LLMs may generate unanswerable questions or hallucinate information that does not exist in the reference data. Besides, some of the LLM-generated questions are too easy and can be directly answered with only LLMs' internal knowledge.
To address these challenges, we propose a cuman-guideed LLM generation approach that uses question templates to bridge human guidance and automatic LLM generation [67, 79]. We first ask ChatGPT to generate candidate question templates from reference data, using prompts such as "Generate diverse and challenging template questions that users may have based the given information." ". To obtain diverse questions, we generate around 50 template questions for each external data source. We then perform manual validation to select the templates that cannot be answered with LLMs' internal knowledge but become answerable with the reference corpora. We go through all question templates and eliminate those that meet either of the following conditions: (1) Template questions that vanilla ChatGPT can answer based on its internal knowledge with a success rate of over 50% (e.g.. "What is the distance between LAX and SFO?", where distance information can be memorized by ChatGPT); (2) Template questions posing queries about information not present in the external data (e.g.. "What the average price from LAX to SFO?", where price information is missing from the flight data). After examining all the templates, we selected 1 117 most representative and diverse question templates for the entire ToolQA dataset.
After the high-quality question templates are manually selected, we sample values from the reference data to automatically fill into the templates generate concrete questions. For example, given the template "Did the flight from {Origin} to {Dest) on [Date} } get canceled or diverted?", we can sample the values "LAX", "MDW". "01/09/2" 22" from the reference Flight tabular data and fill into the template to form a question: "Did the flight from LAX to MDW on 01/09/2 get canceled or diverted?"
inttps:/ppodduta wolframalpha.commm com/ cww.comm.pppp.... api
5

---

Depending on the difficulty of the questions, we classify them into two classes easy and hard. Easy questions primarily focus on extracting a single piece of information from external knowledge. thus requiring fewer tools to involve in the solution. Conversely, hard questions require complex operations (e.g., average) and reasoning (e.g,, comparison) over multiple information pieces drawn from the reference corpora, requiring more tools and complex reasoning among them. We provide a comprehensive list of both easy and hard question templates in Appendix C and D.
3.4 Programmatic Answer Generation
Our final step is to create accurate answers for the generated questions. To guarantee the validity of these responses, we implement 1) operators, which are functions corresponding to the predefined tools; and 2) tool chains, which are schemas for composing different operators for different question templates. For each question, as we know the true arguments filled into the question template, we can run the tool chains with the corresponding arguments to programmatically extract answers from the reference data. This process enables automatic generation correct answers to questions, even for those questions that involve multi-step reasoning. Figure 2(c) demonstrates this generation process. When answering a generated question with sampled values "Did the flight from LAX to MDW on 01/09/ get canceled or diverted?", write Python codes to implement the operators over the reference data, including database loader, data filter, and get-value function. Then, the programmatic pipeline runs a tool chain of these operators to automatically generate the correct answer (details in Appendix E).
4 Experiments
4.1 Baselines
We evaluate the performance of the following methods on ToolQA, covering both standard LLMs and tool-augmented LLMs: (1) LLaMA-2 [61] and Falcon [1] are ofate-of-the-ae art open-soureed large language models. We directly feed the questions into two versions (13B and 70B) of LLaMA-2 and Falcon (40B) obtain the predictions; (2) ChatGPT [43]: We directly feed the questions into OpenAl's ChatGPT model (gpt-3.5-4 b-u and obtain its response as the final answer. (3) CoT [65, 26]: We chain-of-thoughh prompting for ChatGPT, adding the prompt "Let's think step by step:" after the question to leverage LLMS' reasoning ability for question answering. (4) Chameleon [32] is recent method that uses LLMs controller use multiple tools for solving subtasks and has shown promising results in reasoning and QA tasks. When running Chameleon on ToolQA, we the tool pool to our defined tools in $ 3.1. (5) ReAct [76] integrates reasoning with tool use by prompting LLMs to generate interleaved verbal reasoning traces and tool calls. This integration has been shown effective in enhancing LLMs' problem-solving capabilities. We instantiate two versions of ReAct using gpt-3. 3.t-3.5-t0 5oturbo and text-davinci-O00
Different from the existing works that mainly provide task-level few-shot exemplars, provide tool-level demonstrations. We used 8 demonstrations about how to use tools for QA, ensuring that each tool in the pool is covered at least once by the demonstrations. Such tool-level demonstrations provide a concise tutorial to the LLMS for tool use, covering all tool uses with the LLM context limit. Details about the demonstrations and our prompts are included in Appendix F. To assess the performance of methods on the ToolQA benchmark, we normalize both the ground-truth answers and the model predictions to ensure uniformity in format. Success rates are then computed based on the exact match between these normalized answers. We use series of rules for normalization (I) We normalize different time string formats, (e.g., converting "I "18:06" and "1006.0" " "1806".); (2) For price-related questions, we normalize the units by removing price units (e.g.. USD, $); (3) We remove all the punctuations from both the model predictions and ground normalize the article usage (e.g., an, the) via removing all articles from both the model predictions and ground- -ruth answers; (5) We normalize the white spaces by trimming multiple spaces into single space. As most of the predictions and answers are numerical values or entities, these normalization rules address most of the false negative cases during matching. We evaluate the model's ability against the generated question-answer pairs in an open-ended manner, focusing on whether the model can arrive at the correct answer, regardless of the used tool chains.
answers; (4) We
4.2 Results
Internal Knowledge vs. External Knowledge. From the results in Tables 3 and 4, the vanilla open-sourced L.LMS and ChatGPT underperform their tool-augmented counterparts on both easy and hard questions. This is expected, vanilla LLMs lack access to external information for question

---

Table 3: Success rates on easy questions.
LLM Category
Models
Flight
Coffee
Agenda
Yelp
DBLP
SciREX
GSM8K Airbnb Average
LLaMA-2 (13B)
0.0
2,0
0.0
5.0
1.0
0.0
9.0
1.0
2.3
Open- Sourn-d LLMs
Falcon (40B)
1.0
1.0
2.0
8.0
1.0
0.0
8.0
5.0
3.3
L.aMA-2 (70B)
2.0
6.0
5.0
15.0
0.0
9.0
4.0
5.1
ChaiGPT
2.0
0.0
0.0
15.0
0.0
2.0
26.0
0.0
5.6
Closed-Soureee LLMS
CuT
1.0
1,0
0.0
9.0
0.0
0.0
30,0
0.0
5.1
Chameleon
30.0
9.0
4.0
3.0
27.0
4.0
10.6
Tool-Augmented LLMS
ReAct (GPT:3)
61.0
90.0
29.0
77.0
28.0
32,0
25.0
43.1
ReAct (GPT-3.5)
48.0
81.0
24.0
64.0
23.0
2.0
23.0
29.0
36.8
Table 4: Success rate on hard questions.
LLM Category
Models
Flight
Coffee
Agenda
Yelp Airbnb DBLP SeiREX Average
LLaMA-2 (13B)
1.0
0.0
0.0
4.0
1.0
5.0
1.0
1.7
Open- Seen-Soureed LLMs
Falcon (40B)
1.0
0.0
0.0
4.0
1.0
6.0
1.0
1.9
LLaMA-2 (70B)
1.0
0.0
0.0
4.0
1.0
4.0
3.0
1.9
ChatGPT
2.0
2.3
1.0
0.0
2.0
4.0
3,0
2.0
Closed-Sourced LLMs
CoT
0.0
0.8
0.0
1.0
0.0
3.0
5.0
1.4
Chamelcon
3.0
0.0
0.0
0.0
8.0
0.0
1.9
Tool-Augmented LLMs
ReAct (GPT-3)
3.0
10.8
0.0
3.0
0.0
19.0
(GPT-33))
5.0
17.7
7.0
8.0
7.0
5.0
8.2
answering. Additionally, the vanilla LLMs show near-zero performance different tasks, indicating that there is little overlap between the benchmark data and the LLMs' internal knowledge.
Comparing Different Tool-Use LLMs. Tables 3 and 4 show the results of different methods on the easy and hard questions. ChatGPT and CoT achieve very poor success rates (< 10) on both easy and hard questions across different tasks. This is expected as the questions in ToolQA cannot be answered solely based on LLMs' internal knowledge and reasoning. Chameleon achieves slightly better performance, with 10.6% and 1.9% success rates easy and hard questions, respectively. This is because Chameleon incorporates tool descriptions and integrates human-induced orderings of these tools in its context, enabling it to comprehend and compose different tools for QA. However, Chamelcon cannot take feedback from the execution trace, thus often suffering from infeasible actions omitted arguments in its generated plans. ReAct is the west-performing model. It can use observations in the execution trace to generate its next action, allowing it to iteratively refine its tool use chain and obtain better success rates.
Easy vs. Hard Questions. Comparing Tables and 4, we observe that all the baselines perform much worse on hard questions. The best method achieves an average success rate of 43.13% on easy questions, while that number drops to 8.24% on hard questions. As mentioned in $ 3, the hard questions in ToolQA require more tool calls and more complicated compositions. Current tool- augmented LLMS struggle with answering such hard questions, which requires further development of techniques to improve their ability to reason about the task and generate plans for tool use.
GPT-3 vs. GPT3.5. Comparing the different versions of ReAct, we observe that the ReAct (GPT-3) outperforms ReAct (GPT-3.5) on easy questions, yet it shows inferior performance on hard questions. Our hypothesis is that for easy questions, it more important learn and follow the format of the tool calls in the context, which GPT-3 is stronger at. For hard questions, the better reasoning and code understanding abilities of GPT-3.5 enables come up with "innovative" solutions that never appear in the context, leading to higher success rates. An example can be referred to in $ 5.3.
Result Analysis and Discussion
We analyze the drawbacks and possible improvements of existing tool-augmented LLMs, taking the best pertorerformed ReAct (GPT-3.5) model on the hard questions of ToolQA as an example.
5.1 Main Error Type I: Argument Errors
By performing comprehensive error analysis, we found that the most common error type when asking LLMs use tools for QA is argument error LLMs calling the tools with wrong arguments.
*GPT-4 was not included in the evaluation we have access its APL.
7

---

150-
# Wrong Calls
103
1000-
88
coffee-
55
dbip
39
126
22
arbrbb
11
111
112
yelp
Loudb
Fiterbi
sot
Pytten
GatValue
Calcalster
RatrievaBerrxx
Nhighleighhous
CheckEage
agends-
Hights coffee
dbip
abrbbb
yelp
achrex agends
(a) Incorrect tool calls of ReAct ToolQA
(b) Confusion matrix of questions from dif- ferent resources in ToolQA
Figure 3: Analysis of incorrect tool calls and incorrect data sources made by ReAct TooIQA
For ReAct, this error type makes 44.56% and 28.23% out of the 377 and 436 error cases on easy and hard questions respectively, as shown in Figure 3(a). Interestingly, ReAct shows different argument error patterns on easy and hard questions. On easy questions, it tends to make more mistakes on database-related tools. For example, the model commits a total of 120 errors when calling LoadDB, FilterDB, and GetValue tools for easy questions, while this number reduces to 95 for hard questions. On the other hand, when dealing with code-related tools (e.g., SQLInterpreter and SythonInterppetent
ReAct makes nearly 10x more errors for hard questions than for easy ones. This phenomenon is likely because the solution logic for hard questions is often more complex and cannot be fully inferred from the context alone. Consequently, the LLMs tend to rely on their understanding of code and programming concepts tackle these intricate questions. In contrast, for easy questions, the LLMs tend to follow the patterns provided in the context, attempting to combine different database operations to arrive at solution.
5.2 Main Error Type II: Incorrect Data Source
We have conducted investigation into the data sources preferred by LLMs when answering questions. We found that LLMs also have difficulties in identifying the proper reference corpora answer the questions. This behavior is graphically represented as a confusion matrix in Figure 3(b). Upon examining the figure, it is apparent that for target reference corpora like Flight, Coffee, Airbnb, and Yelp that contain temporal information, LLMs are more likely to query the Agenda corpus for answering questions. Similarly, given that the SciREX knowledge corpora and DBLP graph are both in the scientific domain, LLMS tend to be confused about which source query when answering scientific questions.
5.3
Main Error Type III: Innovation and Hallucination
For in-context tool-augeented LLMs, it is typical to include descriptions and use-case examples of each tool in the prompt. However, as the problem complexity increases with the number of tools, it becomes challenging to encompass all possible instances of compositional tool use as few-shot exemplars. Consequently, it is vital for LLMs to uncover logical relationships among different tools, which have never been encompassed in the human-provided exemplars, to solve challenging tasks process we refer as "innovation."" " However, these innovative behaviors are a double-edged sword as they are often accompanied by hallucinations. Figure 4 illustrates this phenomenon with case study, where LLMs answer hard questions with reference Coffee data. Given the context length constraint, the few-shot exemplar only showcases the basic usage of database operations and the SQL interpreter. For the hard question in Figure 4(a), ReAct (GPT-3) strictly follows the operations displayed in the context, leading to failure. On the contrary, ReAct (GPT-3.5) innovatively identifies the SQL interpreter as a possible alternative to database operations, especially when the latter fails repeatedly. However, such innovations can oftentimes lead to hallucinations. As shown in Figure 4(b), when answering another hard question from the Coffee data, ReAct (GPT-3.5) opts to hallucinate certain observations (highlighted in yellow) that are non-existent in the feedback from tool execution.
5.4 Other Error Types
We manually go through and count all the errors made by the ReAct (GPT-3.5) model and show the errors on both easy and hard questions in Figure In addition to the aforementioned 3 main error types, there are 4 error types that frequently occur:
8

---

Coffoe-Bard
(GP2-3.5)
Coffee-Bard
Queation: that arszege coffes
Queation: Eox much the colfee peice
price lcom 2019-10-3; to 2021-07-237
calcalate the eccroge coffee psice the filtered dasa,
Tooght
change fron 2006-11-27 2016-06-233
Cozsect Answee:
Cocrect Answer:
Pelct (GPT-3)
Calculate (egeeeeeeeooose
Pwllct (GPT-3.5)
This sa cueasion zelated to coffee. We need to lead the cofffee darabaace,
thought 1:
Thought Action detvialue faverct oset doservation the value guerying 15.
adlectation.
Thought need no rodify the
#iltaring query to the connect formst by addinng guotes around the date. Actson 8: Pelterte tawllbbeeeteeeeee Obeacvation 6: Ene Hileezing guary:
Action 1: cotten
Observation 1: We zave auccesstully leaced the cottes catabaze...
Please
Eatae1202 202500gm02" incorract. Please
thought 2: We need to filces inforration celated so quecy. Action 2: Filteris (dltocts/eetto0001 (uutew22199-10 sacec-2021-07-
godley coodi siond
Thought Oopa, made mistate sgu Las correct it. Action 5: EQEEnsaEpzate [sstace ANcICicael 2908 codfne, soEfow data RSEXS 2014 9019-10-31 A2D Dask
KONGTd Thought
succesorally
observation 21
euccesstully
GetValue [closse)
detaia filte
Ooservaticn 5: RVE(CIORR)
the
coffae 2005- 2005-11-2 Ieos
Action 3: (close]
of osffes 2016-05-03.
Observation 3: E01.5, 104,6, 103,65, 193,65, 18900 (43 valuee)
Zhought 6: afses colculatioon, ve inos Lhst Lie avezage eaffoe peice from
Action wwloouttee1l..0 116.0-96:21 23:25
2019-12-31 so 2021-07-23 to 123.9. Action 6: (1200))
Contast, Xoosted
Thought Atter calculation,
Observation Anoxer 000300
that the cottee peice changed Action 9: Ficiab 23.75)
Obssrvation 9: Mnavez INCORRSCE
(a) Innovation
(b) Hallucination
Figure 4: An example of innovation and hallucination when answering hard questions Coffee data. Actions and observations shrouded in pink are incorrect, whereas those in green are correct. Terms highlighted in yellow signify hallucinations produced by ReAct (GPT-3.5).
44.56%
Argument Error
48.23%
Argument Error
9.02%
Infeasible Actions
9.15%
Infeasible Actions
4.77% Hallucination
7.48% Hallucination
18.04%
Incorrect Data Source
7.48%
Incorrect Dala Source
10.88%
Too Long Context
16.63%
Too Long Context
6.90% Mis-understanding
0.83%
Mis-understanding
5.84%
Low-Quality Retrieval
10.19%
Low-Quality Retrieval
(a) Easy questions.
(b) Hard questions.
Figure 5: Error analysis of ReAct on ToolQA
Infeasible Actions:
The execution of tool calls are infeasible in the environment, often involving new tools that do not exist in the pre-defined tool pool.
Too Long Context:
The encoding of interaction history, observations, and tool-use plans exceed the length limitation of GPT family models, resulting runtime errors;
Mis-understanding: The LLMs cannot understand the observations obtained from external interaction and fail to determine the next steps or generate answers;
Low-Quality Retrieval: This error occurs when the retrieval model fails to extract the relevant information from text corpora, indicating insufficient external knowledge for LLMs to answer questions accurately.
Comparing these error types easy and hard questions, we find that the overall distribution is similar, though there is a slightly higher rate hallucination and long-com long-context errors when answering hard questions. This can be attributed to the complexity of hard questions, which often require composing more tools for question answering.
5.5 Potential Misuse of ToolQA
We summarize the potential misuse of the ToolQA dataset and Tool-augmented L.MMs follows:
If L.MMs can be trained or prompted to use external tools, they could be prompted to use tools
that extract personal information, propagate malware, or provide misleading information;
There is potential for future LLMs to over-rely on these external tools, sacrificing their intrinsic reasoning abilities. This can make them less versatile in situations where tool use not feasible;

---

Table 5: Performance of tool-augmented open-sourced LLM.
Methods
Coffce-Easy
Coffee-Hard
LLaMA-2 (13B)
2.0
0.0
ReAct (LLaMA-2, 13B)
31.0
6.2
ReAct (GPT-3.5)
81.0
17.7
As LLMs are prompted to interact with more external systems, the security risks can increase. Malicious actors might find ways exploit the interactions between L.MMs and the external tools they leverage.
5.6 Comparing Open-Source LLMs with ChatGPT
Table 5 shows the comparison of vanilla LLaMA-2, tool-augmented LLaMA-2, and tool-augmented ChatGPT. There is significant performance gap between vanilla LLaMA-2 and coll-augmented
LLaMA-2, which is consistent with what we have observed on closed-source LLMs in Section 4.2. In terms of the ability to use external tools for answering the questions, we found that LLaMA- 2 is indeed lagging behind ChatGPT. The prompts tailored for tool-augmented LLMs tend to be complicated and lengthy, containing tool descriptions, few-shot examples, and interaction history with the environment. Such long contexts make it difficult for ILLMA-22 to understand complex instructions hidden inside.
6 Conclusion and Recommendation
We have developed TooIQA, dataset that assesses the ability of Large Language Models (LLMs) in using external tools for solving complex problems. ToolQA is curated by an automated three-phase process, including reference data collection, template- baseee question generation, and programmatic answer generation. This pipeline general and can be expanded to incorporate external knowledge corpora in different domains. We tested both standard LLMs and lool-augmented LLMs ToolQA Our experiments showed that even the strongest model achieved limited performance on the hard questions of TooIQA. Our analysis found that current oool-augmented LLMs tend to make errors such as incorrect tool calls and using incorrect data sources. These issues could be potentially addressed by fine-tuning using a collection of tool-use corpora with open-source LLMs. In the future, we plan collect high-quality tool-use sequences to fine-tune open-source LLMs, and subsequently evaluating their performance ToolQA
important to note that the reported performance of closed-source LLMs, such as ChatGPT, based on specific versions of these models. As these closed-soures models undergo further development and updates, the results may change accordingly. We advocate two strategies to mitigate the issue: (1) We plan continuously include the performance of newly released models in repository, with the collective efforts of both our team and peer researchers of the community. We intend to integrate results from different versions of such product APIs and document the identifiers of each version (e.g., 0314, 0613, etc.) to promote reproducibility. (2) The reported performance of closed-source LLMSs should be regarded primarily optional. Instead, we encourage the community to use the performance of open-source L.LMs as the primary baseline when employing the TooIQA benchmark. This will promote reprodueibility and consistency of the evaluation on ToolQA over time.
reference, and making comparisons with them
Acknowledgment and Disclosure of Funding
This work was supported in part by NSF (IIS2008334, IIS-2106961, CAREER IIS IIS-2145 144438),, ONR (MURI 100001417-11 100001-17-112656,) IDEaS Cyberinfrastruucure Resources, and Microsoft Accelerate Foun- dation Models Research Program.
References
[1] E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, M. Debbah, E. Goffinet, D. Heslow, J. Launay, Q. Malartic, B. Noune, B. Pannier, and G. Penedo. Falcon-4OB: an open large language model with state-of-the-art performance. 2023.
10

---

[2] Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204. 00062,, 2022.
[3] S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. B. Van Den Driess- che, J.-B. Lespiau, B. Damoc, A. Clark, et al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages 2206-2240, PMLR, 2022.
[4] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[5] S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Li, S. Lundherg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.1277 12712, 2023.
[6] W. Chen, X. Ma, X. Wang, and W. W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks, 2022.
[7] W. Chen, X. Wang, and W. Y. Wang. A dataset for answering time-sensitive questions. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
[8] X. Chen, M. Lin, N. Schärli, and D. Zhou. Teaching large language models to self-debug. arXiv preprint arXiv:2304.055222 2023.
[9] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204. 02311, 2022.
[10] H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, E. Li, X. Wang, M. De- hghani, S. Brahma, et al. Scaling instruction-finetue language models. arXiv preprint
MIXiv:210.111416 2022.
[11] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14.14 2021.
[12] B. Dhingra, J. R. Cole, J. M. Eisenschlos, D. Gillick, J. Eisenstein, and W. W. Cohen. Time- aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10:257-273, 2022.
[13] L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang. J. Callan, and G. Neubig. Pal: Program- aided language models. arXiv preprint arXiv:2211.10435, 2022.
[14] D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset. NeurlPS, 2021.
[15] S. Imani, L. Du, and H. Shrivastava. Mathprompter: Mathematical reasoning using large language models. arXiv preprint arXiv:2303. 05398, 2023.
[16] G. Izacard, M. Caron, L. Hosseini, S. Riedel, P. Bojanowski, A. Joulin, and E. Grave. To- wards unsupervised dense information retrieval with contrastive learning. arXiv preprint 1120v:211 2021.
[17] G. Izacard, P. Lewis, M. Lomeli, L. Hosseini, Petroni, T. Schick, Dwivedi- Yu, Joulin, S. Riedel, and E. Grave. Few-shot learning with retrieval augmented language models. arXiv preprint arXiv:2208. 03299, 2022.
[18) S. Jain, M. van Zuylen, H. Hajishirzi, and I. Beltagy. SciREX: A challenge dataset for document- level information extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pages 7506-7516, Online, July 2020. Association for Computational Linguistics.
[19] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, J. Bang, A. Madotto, and P. Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
[20] Q Jin, Y. Yang, Q. Chen, and Z. Lu. Genegpt: Augmenting large language models with domain tools for improved access to biomedical information. ArXiv, 2023,
11

---

[21] M. Joshi, E. Choi, D. Weld, and L. Zettlemoyer. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 1601-1611, Vancouver, Canada, July 2017. Association for Computational Linguisties.
[22] V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781, 2020.
[23] J. Kasai, K. Sakaguchi, Y. Takahashi,
R. L. Bras, A. Asai, X. Yu, D. Radev, N. A. Smith. Y. Choi, and K. Inui. Realtime qa: What's 's the answer right now? arXiv preprint arXlv:2207.13332 2022.
[24] T. Khot, H. Trivedi, M. Finlayson, Y. Fu, K. Richardson, P. Clark, and A. Sabharwal. Decom- posed prompting: modular approach for solving complex tasks. The Eleventh International Conference Learning Representations, 2022.
[25] G. Kim, P. Baldi, and S. McAleer. Language models can solve computer tasks, 2023.
[26] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large language models are zero-shot reasoners. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022.
[27] Lewis, Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, et al. Retrieval-augmented generation for knowledge- inttes-v-ntenivee nlp tasks. Advances in Neural Information Processing Systems, 33:9459-9999 9474,, 2020.
[28] A. Lewkowyez, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone,
C. Anil, I. Schlag, T. Gutman-Solo, et al. Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206. 14858, 2022.
[29] M. Li, F. Song, B. Yu, H. Yu, Z. Li, F. Huang, and Y. Li. Api-bank: A benchmark for tool-augmented Ims,, 2023.
[30] B. Lin, K. Tan, C. Miller, Tian, and X. Ren. Unsupervised cross-task generalization via retrieval augmentation. In Advances in Neural Information Processing Systems, 2022.
[31] B. Liu, Y. Jiang, X. Zhang, Q. Liu, S. Zhang. J. Biswas, and P. Stone. Llm+p: Empowering large language models with optimal planning proficiency, 2023.
[32] P. Lu, B. Peng, H. Cheng, M. Galley, K.-W Chang, Y. N. Wu, S.-C. Zhu, and J. Gao. Chameleon: Plug-and-play compositional reasoning with large language models. arXiv preprint arXiv:2304.09842 2023.
[33] P. Lu, L. Qiu, K.-W. Chang, Y. N. Wu, S.-C. Zhu, T. Rajpurohit, P. Clark, and A. Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. arXiv preprint arXiv:2209 14610, 2022.
[34] S. Lu, N. Duan, H. Han, D. Guo, S.-w. Hwang, and A. Svyatkovskiy. Reace: A retrieval- augmented code completion framework. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 6227-6240, 2022.
[35] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXivv2303. 2023.
[36] A. Madaan and A. Yazdanbakhsh. Text and patterns: For effective chain of thought, it takes two to tango. arXiv preprint arXiv:2209. 200.::22000.0886.. 2022.
[37] A. Mallen, A. Asai, V. Zhong, R. Das, H. Hajishirzi, and D. Khashabi. When not to trust language models: Investigating effectiveness and limitations of parametric and non-parametric memories. arXiv preprint arXiv:2212.1083
2022.
[38] S. Mishra, M. Finlayson, P. Lu, L. Tang, S. Welleck, C. Baral, T. Rajpurohit, O. Tafjord,
A. Sabharwal, P. Clark, et al. Lila: A unified benchmark for mathematical reasoning. arXiv preprint 271.1:2210.17517, 2022.
[39] S. Mishra, D. Khashabi, C. Baral, and H. Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pages 3470-3487, Dublin, Ireland, May 2022, Association for Computational Linguistics.
12

---

[40] R. Nakano, J. Hilton, S. Balaji. J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, et al. Webgpt: Browser-assister question-answennig with human feedback. arXiv preprint arXiv:2/ 2021.
[41] R. Nogueira, Z.. Jiang, and J. Lin. Investigating the limitations of transformers with simple arithmetic tasks, arXiv preprint arXiv:2102.13011 2021.
[42] OpenAl. Gpt-4 technical report. arXiv, 2023.
[43] OpenAl. Introducing chatgpt, 2023.
[44] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal,
K. Slama, A. Ray, al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 11227730-277442 2022.
[45] B. Paranjape, S. Lundberg, S. Singh, H. Hajishirzi, L. Zettlemoyer, and M. T. Ríbeiro. Art: Automatic multi-step reasoning and tool-use for large language models. arXiv preprint vrXiv:2303.00014 2023.
[46] A. Parisi, Y. Zhao, and N. Fiedel. Talm: Tool augmented language models. arXiv preprint arXiv:2205. 22::: 2022.
[47] P. Patel, S. Mishra, M. Parmar, and C. Baral. Is question decomposition unit all we need? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4553-4569, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics.
[48] S. G. Patil, T. Zhang, X. Wang, and J. E. Gonzalez. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305. /53344 2023.
[49] B. Peng, M. Galley, P. He, H. Cheng, Y. Xie, Y. Hu, Q. Huang, L. Liden, Z. Yu, W. Chen, et al. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint :rv: 12813, 2023.
[50] O. Press, M. Zhang, S. Min, L. Schmidt, N. A. Smith, and M. Lewis. Measuring and narrowing the compositionality gap in language models, 2023.
[51] J. Qian, H. Wang. Z. Li, S. Li, and X. Yan. Limitations of language models in arithmetic and symbolic induction. arXiv preprint arXiv:2208. 0505/, 2022.
[52] Y. Qin, S. Hu, Y. Lin, W. Chen, N. Ding, G. Cui, Z. Zeng, Y. Huang, C. Xiao, C. Han, Y. R. Fung, Y. Su, H. Wang, C. Qian, R. Tian, K. Zhu, Liang, X. Shen, B. Xu, Z. Zhang, Y. Ye,
B. Li, Z. Tang, J. Yi, Y. Zhu, Z. Dai, L. Yan, X. Cong. Y. Lu, W. Zhao, Y. Huang, J. Yan,
X. Han, X. Sun, D. Li, J. Phang, C. Yang, T. Wu, H. Ji, Z. Liu, and M. Sun. Tool learning with foundation models, 2023.
[53] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
[54] S. Robertson, H. Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends@ Information Retrieval, 3(4):333-388) 2009.
[55] T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Iliè, D. Hesslow, R. Castagné, A. S. Luccioni, Yvon, M. Gallé, al. Bloom: 176b-parameter open-access multilingual language model. arXiv preprini 1..0::22/.005000 2022.
[56) T. Schick, J. Dwivedi- Yu, R. Dessi, R. Raileanu, M. Lomeli, I.. Zettlemoyer, N. Cancedda, and Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.0476. 2023.
[57] Y. Shen, K. Song, X. Tan, D. Li. W. Lu, and Y. Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303. 17580, 2023.
[58] W. Shi, S. Min, M. Yasunaga, M. Seo, R. James, M. Lewis, L. Zettlemoyer, and W.-t. Yih. Replug: aetrieval-augmenter black-box language models. arXiv preprint 2rXiv:2301.12653 2023.
[59] H. Sun, Y. Zhuang, I.. Kong, B. Dai, and C. Zhang. Adaplanner: Adaptive planning from feedback with language models, 2023.
13

---

[60] J. Thome, A. Vlachos, C. Christodoollpoolls, and A. Mittal. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809-819, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
[61] H. Touvron, I.. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L.. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, Edunov, and T. Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023.
[62] W. Wang, L. Dong, H. Cheng, X. Liu, X. Yan, J. Gao, and F. Wei. Augmenting language models with long-term memory, 2023.
[63] X. Wang, S. Li, and H. Ji. Code4struct: Code generation for few-shot structured prediction from natural language. arXiv preprint arXiv:22 /2810, 2022.
[64] Z. Wang, S. Cai, A. Liu, X. Ma, and Y. Liang. Deseribe, explain, plan and select: Interactive planning with large language models enables open-world multi-tase agents, 2023.
[65] Wei, X. Wang, Schuurmans, M. Bosma, B. Ichter, Xia, E. Chi, Q. Le, and D. Zhou, Chain- of of-thought prompting elicits reasoning in large language models. arXiv, page 2201. 1990v6,, 2022.
[66] L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh, et al. Ethical and social risks of harm from language models. arXiv preprint 1xXiv:2//2.04330
2021.
[67] S. Wiegreffe. J. Hessel, S. Swayamdipta, M. Riedl, and Y. Choi. Reframing human-ai collab- oration for generating free-text explanations. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lunguage Technologies, pages 632-658, 2022.
[68] S. Wolfram. WolframlAlpha as the Way to Bring Computational Knowledge Superpowers to ChatGPT. Stephen Wolfram Writings, 2023.
[69] C. Wu, S. Yin, W. Qi, X. Wang, Z. Tang, and N. Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671, 2023.
[70] J. Xie, K. Zhang. J. Chen, R. Lou, and Y. Su. Adaptive chameleon or stubborn sloth: Unraveling the behavior of large language models in knowledge conflicts. arXiv preprint a3X1002305.13300, 2023.
[71] Q. Xu, F. Hong, B. Li, C. Hu, Z. Chen, and J. Zhang. On the tool manipulation capability of open-source large language models. arXiv preprint arXiv:2305. /6504, 2023.
[72] R. Xu, Y. Yu, J. C. Ho, and C. Yang. supervisd scientific document classification via retrieval-augmennte
multi-stage training. arXiv preprint arXiv.2.306.0719. 2023.
[73] Z. Yang, L. Li, J. Wang, K. Lin, E. Azarnasab, F. Ahmed, Z. Liu, C. Liu, M. Zeng, and L. Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint 2303 2023.
[74] Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning. HotpotQA: dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369-2380, Brussels, Belgium, Oct.-Nov. 2018. Association for Computational Linguistics.
[75] S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305. 10601, 2023.
[76] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, 2023.
14

---

[77] J. Zhang. Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt, 2023.
[78] M. Zhang and E. Choi. SituatedQA: Incorporating extra-linguistic contexts into QA. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7371-7387, Online and Punta Cana, Dominican Republic, Nov. 2021. Association for Computational Linguistics.
[79] R. Zhang, Y. Yu, P. Shetty, L. Song, and C. Zhang. Prboost: Prompt-based rule discovery and boosting for interactive
learning. arXiv preprint Stti:::22030097333 2022.
[80] Y. Zhuang, Y. Li, J. Zhang, Y. Yu, Y. Mou, X. Chen, L.. Song, and C. Zhang. ReSel: N- ary relation extraction from scientific text and tables by learning to retrieve and select. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pages 730-744, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics.
15

---

A
Additional Related Works
Methods
Tool Numbers
Tool Categories
# Tool/Task
Reasoning
Instruction Type
Task
Single-Tool Methoos
CoT [65]
Generation
Prompting
QA
1ila [38]
math/cade code
Generation
Prompting
MathQA
Program-of- Thennggh [6]
Generation
Prompting
TabQA
Code44truct [63]
Prompting
Extraction
PAL [13]
code
Generation
Prompting
MathQA
MathPrompt [15]
code
Generation
Prompting PR FT PR & FT
MathQA
ToolFormer [56]
Basic
Generation
GraphToolFormerr [77]
Graph
Human Info
Graph
Talm [46]
QA
Multi-Tool Methods
WehGPT [40]
10
Weh Operation
>
Feedhack
Fine-tuning
QA VQA
HuggingGPT [57)
>10
Vision
>
Human Info
Prompting Prompting Prompting
Chameleon [32]
>10
ende, nlp.
Human Info
ScienceQA, TabQA
GeneGPT [20)
38
NCBI APIs
>
Generation
Gene Tasks
ART [45]
code/mattivetrieeer
Human Feedhack
Prompting PR & FT
BigBench
ReAct [76]
retriever
>
Feedhack
QA. AlfWorld. WehShop
MM RMMcR [73]
>10
vision
51
Feedhack
Prompting
CV tasks
Visual CharGP
>10
vision
Feedback
Prompting
CV tasks
Table 6: A comparison of methods that leverage LLMs for Tool-use.
We list the state-of-the-arr related works in lool-augmented LLMs in Table 6. All of them can be categorized into two groups: (I) single-tool methods, that focus making single API call perfect the solution; (2) multi-tool methods, that emphasize more on studying how to compose different tools together to solve challenging problem. ToolQA is more suitable for the evaluation of the second category to test the inherent logical reasoning behind different tools. Additionally, there exist other notable contributions [64, 25, 59] within the realm of prcissonnmaking that specifically emphasize the planning capabilities of expansive language models. These endeavors can be regarded methods affiliated with tools, wherein the actions within generated plans are analogous to distinct tools utilized for specific purposes.
B
Data Sources
B.1 Different Data Source Introduction
Flight Status (2022-2023) contains almost all flight information of airlines between 2022 and 2023, which is too contemporary for LLMs' internal knowledge.
Daily Coffee Price (2000-2022) contains the daily price of coffee, ranging from 2000 to 2022, where the information too contemporary and detailed for LLMs' internal knowledge.
Yelp Business Data² a subset of Yelp's business data across metropolitan areas in the USA and Canada, where the information is too detailed for LLMs' internal knowledge.
Airbnb Open Data is subset of Airbnb activities in New York, where the information is too detailed for LLMs' internal knowledge.
DBLP Citation Network (V14) constructs the graph based on the records after 2020. The author-author and paper-paper relations formulated as two separate graphs
GSM8K is dataset of 8.5K high-quality linguistically diverse grade school math word problems. We sample the questions from the error cases made by ChatGPT on the original dataset to make sure that the questions cannot be easily handled with its internal knowledge.
SIiREX challenging dataset for document-level information extraction based on collection of full-length machine-learning scientific papers.
hhttps://wwwkkag
Combined_Flights 22res_22222_22222222 csv
"nttps://www.ket kaggle.
dataset_busiinese
Phttpe:/www.aminerr.oo amwaaamiin......aae
"onttpa///githll
16

---

Agenda is our own synthetic dataset to model the real- personal agenda data. To avoid the privacy issue, we first create names, events, and dates with ChatGPT and then randomly compose them to form 10000 different records. To create a pure-text personal agenda corpus, we feed each of the records into ChatGPT, containing generated agenda for virtual characters. More Details can be seen in Appendix B.2.
B.2 Generation Details of Agenda Dataset
As mentioned in $ 3.2, personal or private data serves as significant external knowledge source. There exist applications that have been designed with plugins and external tools specifically querying this type of data, such as AI personal assistants on daily agenda. Nevertheless, we recognize that this data often intersects with sensitive areas, and hence, privacy concerns are paramount. these issues, we automatically synthesize a personal agenda corpus. This not only ensures that the large language models (LLMs) have not been previously exposed to the data but also eliminates any possibility of them inadvertently memorizing the information within their internal knowledge.
address
In the synthetically generated personal agenda corpus, each entry follows the pattern: "NAME performs EVENT at TIME on DATE", incorporating key elements such as names, events, dates, and time slots. To begin, we employ ChatGPT to virtually generate these elements. More precisely, we create 100 unique names, 10000 distinctive events each associated with corresponding time slots within a day. and span all possible dates from 01/01/2022 through 1/222/ Following this, we commence the random assembly of these generated elements to formulate personal agenda entries. For every event- time pair generated, we randomly select from the pool of 100 names and possible dates construct each record. This process yields a total of 9,494 unique personal agenda entries. To transform this corpus into an accessible external database for model querying, we transcribe each record into comprehensible natural language description. Prompts designed for agenda data generation are listed in Appendix F.2.
C Easy Question Templates
C.1 Flights
design the following 10 templates:
What was the departure time of the (CARRIER) (NUMBER) flight from (ORIGIN) to (DEST) on (ORIGIN}?
Was the flight (CARRIER) (NUMBER) from {ORIGIN} to (DEST) cancelled on {ORIGIN)?
What is the flight number of the {AIRLINE) flight from (ORIGIN) to (DEST) on (ORIGIN)?
How long was the different between the CRS-recorded departure time and actual departure time of the [CARRIER] (NUMBER) flight from (ORIGIN) to (DEST) on (ORICIN}?
How long did (CARRIER) [NUMBER) delay when arrival on (DEST)?
How many extra minutes did the (AARREER)(UMBERR flight take from {ORIGIN) to (DEST) on {ORIGIN}?
What was the local arrival time of the (CARRIER) [NUMBER] flight from (ORIGIN} to (DEST) (ORIGIN}?
What was the CRS-recorded arrival time of the (CARRIER) [NUMBER] flight from (ORICIN) to (DEST) on [ORIGIN)?
How long was the flight (CARRIER) (NUMBER) from (ORIGIN) to (DEST) on {ORIGIN)?
How many minutes did the (CARRTER)
(ONEEEERRR(UUBBRRRRR (NUMBER) flight take to taxi in (DATE}?
C.2 Coffee
We design the following 8 templates:
What was the daily coffee price opening on (DATE)?
What was the lowest coffee price on (DATE)?
What was the highest coffee price on {DATE)?
What was the daily coffee price closing on (DATE)?
What was the trading volume of coffee (DATE)?
17

---

What was the percentage change in coffee price on (DATE), based on the difference between the opening and closing prices?
Was (DATE} bearish or bullish day for coffee price?
What was the range of coffee price on (DATE), based on the difference between the high and low prices?
C.3 Yelp
design the following 11 templates for the Yelp dataset:
What is the address of {NAME} in the area of postal code (ODSTAL-CODE)?
What city (NAME} located (STATE)?
What state {NAME} located in?
What is the postal code of [NAME} in the area with postal code (POSTAL-CODE), (CITY),
{STATE}?
What is the star rating of [NAME} in the with postal code (POSTAL-CODE), (CITY). {STATE}?
How many reviews does { receive in the area with postal code (POSTAL-CODE), (CITY), {STATE}, received?
Is (NAME} still open in the area with postal code (POSTAL-CODE), (CITY), {STATE}?
Does (NAME} require appointment in the area with postal code (POSTAL-CODE), {CITY), {STATE}?
What are the hours of operation for [NAME} in the area with postal code (POSTAL-CODE), (CITY), (STATE}?
What categories does [NAME] belong to, in the area with postal code (POSTAL-CODE), (CITY). {STATE}?
What are the coordinates of (NAME) in the area with postal code (POSTAL-CODE), (CITY), {STATE}?
C.4 Airbnb
We design the following 10 templates for easy questions on Airbnb dataset:
What is the host's name for (NAME) in (NEIGHBOURHOOD}?
How many days are (NAME) (id: (ID}) available during year (365 days)?
What is the room type of (NAME) (id: (ID}) in (MEIGHBOURHOOD)?
What is the price of (NAME) (id: [ID}) in (NEIGHBOURHOOD)?
What is the minimum number of nights for (NAME) (id: (ID)) in (NEIGHBOURHOOD)
When did [NAME} (id: {ID}) in [NEIGHBOURHOOD constructed?
How many reviews does (NAME) (id: {ID}} in (MEIGHBOURHOOD) have?
What is the last review date for (NAME} (id: {ID}) in (NEIGHBOURHOOD)?
What is the review rate number for [NAME} (id: (ID}) in (MEIGHBOURHOOD)}
What the average number of reviews per month for [NAME) (id: {ID)) (NEIGHBOURHOOD)
C.5 SeiREX
We design the following 1 templates for easy questions on SciREX dataset:
What is the corresponding (METRIC) score of the (METHOD) method on (DATASET) dataset for (TASK) task?
C.6 Agenda
We design the following 5 templates for easy questions on Agenda dataset:
What did (NAME) do from (START-TIME) to (END-TIME) on (DATE)?
Where did (EVENT) that (NAME) attended take place on (DATE)?
When did (NAME) attend (EVENT) on (DATE)?
How long did (NAME) attend (EVENT) on {DATE}?
Who attended [EVENT) between (START-TIME) and (END-TIME) (DATE) in (LOCATION)?
18

---

C.7 DBLP
We design the following 10 templates for easy questions on DBLP dataset:
Who are the authors of (TITLE}?
What organization is (AUTHOR) from?
How many pages is (TITLE)?
How many papers did (TITLE) cite in the DBLP citation network?
How many papers did papers in the DBLP citation network cite (TITLE}?
How many collaborators does (AUTHOR) have in the DBLP citation network?
How many papers did (AUTHOR) and {AUTHOR] write together in the DBLP citation network?
What papers did {AUTHOR) write in the DBLP citation network?
How many papers did {AUTHOR) write in the DBLP citation network?
What venue did (AUTHOR) and (AUTHOR) collaborate most in the DBLP citation network?
C.8 GSM8K
The questions randomly sampled from the ChatGPT errors in GSM8K dataset without following some templates. Thus, we cannot offer any question templates for GSM8K.
D
Hard Question Templates
D.1 Flights
What percentage of the flights from {ORIGIN} were delayed on (FLIGHTDATE)?
What is the average delay time of all the flights that departed from {DRIGIN) [FLIGHTDATE]?
How many flights were diverted (FLIGHTDATE)?
How many flights with a distance greater than 500 miles on (FLIGHTDATE)?
What is the average airtime of the flights from (ORIGIN) to (DEST) host by (AIRLINE)?
How many flights from (ORIGIN} to (DEST) host by (AIRLINE)?
What is the average flight time of (CARRIER) [NUMBER)
What is the fastest flight from (ORIGIN) to (DEST) on (FLIGHTDATE)?
What is the average speed of (CARRIER) (NUMBER) from (ORIGIN) to (DEST)?
What is the total number of flights operated by (AIRLINE) on (FLIGHTDATE)?
D.2
Coffee
What was the highest coffee price from
ISTART-DATE to LEND-DATE]?
What was the lowest coffee price from
ISTART-DATE to EEND BNDD-DAEE>?
What was the average coffee price from (START-DATE to (END-DATER -DATE)?
How much did the coffee price change from (START-DATE) to (END-DATE)?
What was percentage change in coffee price on (DATE) compared to the previous day?
On which date from EITART-DATES to (END-DATE) was the difference between the highest and lowest coffee prices the greatest?
What was the average daily volume of coffee traded from (START-DATE) to (END-DATE)?
On which date from (START-DATE) to (END-DATE) did the coffee price have the highest increase compared to the previous day?
How many times from (START-DAYT to the previous day?
JEND-DATE) the coffee price increase compared
What was the percentage increase coffee price from (START-DATT to (END-DATE)?
What was the coffee price range from (START-DATE) to (END-DATE)?
19

---

D.3 Yelp
We design the following 10 templates for hard questions in Yelp Dataset.
How many (CATEGORY) businesses are there (CITY), {STATE)?
How many businesses are there (POSTALCODE) area of (CITY), {STATE]?
Which (CATEGORY) business has the highest star rating in {CITY), (STATEJ?
Which (CATEGORY) business has the highest review count in (CITY), (STATE)?" ?"
What is the average review counts of businesses within 5-mile radius from (NAME}?
Which is the nearest (CATEGORY) business (NAME)?
Can you recommend (CATEGORY) business with the highest star rating within a 5-mile
radius of (ADDRESS)?
How many businesses not open currently in (CITY)?
What is the average star rating of (CATEGORY) businesses in (CITY)?
Which region has most bussinesses in (CITY), (STATE)?
D.4 Airbnb
We design the following 10 templates for hard questions on Airbnb dataset.
What is the total price at least if you want to stay at (NAME) in (MEIGHBOURHOOD) for [NUMBER]
nights?
How many airbnbs are there in (NEICHBOURHOOD)
What is the average price of airbnbs in [NEIGHBOURHOOD)?
What is the average review rates within 5 miles from (NAME} (NEIGHBOURHOOD)
How much proporion of airbnbs (MEIGHBOURHOOD) have flexible cancellation policy?
How much does it cost per night to stay at the most expensive entire home/apt in
(MEIGHBOURHOOD)
How many airbnbs are there (NEIGHBOURHOOD) that have review rate higher than Can you recommend me a hotel room with the lowest price in (NEIGHBOURHOOD)?
Can you recommend me a private room with the highest review rate that can host at least people in (NEIGHBOURHOOD))
Can you recommend shared room with the lowest price within 10 miles from (LONGITUDE) longitude and (LATITUDE) latitude?
D.5 SciREX
We design the following templates for hard questions on SciREX dataset:
What is the corresponding (METRIC) score of the {METHOD) method on (DATASET) dataset for (TASK) task?
On which dataset does the (METHOD) method achieve the highest (METRIC) score for (TASK) task?
Which method achieves highest [METRIC} score on (DATASET) dataset for (TASK) task?
On what metrics is the (METHOD) method evaluated on (DATASET) dataset for (TASK) task?
Which datasets is (METHOD) method evaluated on for (TASK) task?
D.6 Agenda
We design the following templates for hard questions on Agenda dataset:
How many events happen on (DATE) in the agenda table?
Who is unavailable between (START-TIME) and (END-TIME) -TIME) (DATE) in the agenda table?
When should I schedule meeting with (NAME) from 9:00 AM 6:00 PM on (DATE) in the agenda table?
What events does (NAME) have on (DATE) in the agenda table?
How many dates in the agenda table have (NAME) scheduled?
20

---

D.7 DBLP
We design the following 10 templates for hard questions on DBLP dataset:
What keywords does (AUTHOR) focus most in the DBLP citation network?
How many people does (AUTHOR-1) need to know at least to know (AUTHOR-2) the DBLP citation network?
How many common collaborators does (AUTHOR-1) have with (AUTHOR-2)?
Which is the most cited paper written by (AUTHOR) in the DBLP citation network?
Which collaborator does (AUTHOR) have the most citations with in the DBLP citation net- work?
Which venue does (AUTHOR) publish the most papers in the DBLP citation network?
How many accumulated citations do papers collaborated by {AUTHOR-1] 1) and (AUTHOR-22 have in the DBLP citation network?
How many papers in do (AUTHOR) and his/her collaborators have in the DBLP citation network?
Who collaborated with (AUTHOR) most the DBLP citation network?
What institutions participated in the study of {TITLE) in the DBLP citation network?
E
Code Examples of Programmatic Answer Generation
Below is an example of programmatic answer generation. The example code answering the question of "What percentage of the flights from (ORIGIN) were delayed on (FLIGHTDATE) ?". More details of the programmatic answers can be seen in the public code.
def solution (data, flightdate, origin):
num_total (data. loc (data (data ("Figghtttttt ==1ightdaate & (data("Drigin"] origin)])
num_cancelled (deta [neo__attaa"Fiiggheeeee
flightdate) & (data("Origin) ==origin) "Cata("Cacelled"] ==True)])
num, num__ancees can_caeelleed >0:
question =WWhat percentage of the flights from 0 were delayed on "f?"mtormat (origin, flightdate)
answer ="{ 1f)" format caum_cane /num_total +100))"7"
Additional Implementation Details
F.1
Implementation Details
All experiments are conducted on CPU: Intel(R) Core(TM) 17-5930K CPU @ 3.50GHz and GPU:
NVIDIA GeForce RTX A5000 GPUs using python 3.8, Huggingface 4.6.0 and Pytorch 1.10. We keep the parameter top_p = 1.0 and temperature = 1.0 for calling ChatGPT APIs [43] for the question generation part.
F.2 Prompts
F.211
Prompts for Agenda Data Generation
The prompts used for virtual name generation:
<Agenda_Name Gen>> Prompt
an assistant ansver questions.
Can you list 100 English Names?
The prompts used for virtual events generation:
21

---

AAgenda_Evente _Gen> Prompt
You are an AI assistant for text generation.
Generate 100 detailed agenda events, including the event, start time, end time, and location. Please make the events diverse possible and make aure these events can happen real life. Make sure the location detailed name that may exist in real life. Make sure the dates are selected from 2022/01/01 to 2023/01/01.
Example:
Doctor's appointment 9:00 11:00 AM Medical Center
Yoga class 10:30 AM 11:30 AM Yoga Studio Downtown
Generate 100 more detailed agendas that do not conflict with the previous ones.
The prompts used convert the agenda records into natural language descriptions:
<Agenda_Gems Prompt
Please use natural language to describe the event in the agenda with the following
information:
Name: NAME
Date: DATE
Event: EVENT
Start Time: START-TIME
End Time: END-TIME
Location: LOCATION
F.2.2 Prompts for Methods
The prompts used ReAct [76]:
<ReAct> Prompt
Question: How many extra minutes did the DL1575 flight take from ATL MC0 on 2022-01-12?
Thought 1: This is a question related to flights. need to load the flights database.
Action 1: LoadDB[ [flights]
Observation 1: We have successfully loaded the flights database, including the FlightDate, Airline, Origin, Dest, Cancelled, Diverted, CRSDepTime, DepTime, DepDelayMinntes,, DepDelay, ArrTime, ArrDelayMinutes, AirTime, CRSElapsedTime, ActualElapseeTTime Distance, Year, Quarter, Month, DayOfWeek, MarketinggAirllin__eetttt DayofMonth, DOT DOT_ID_ Mr__DD.Mrrketing.Aiilin.. IATA_ Codee
following columns:
Operated_or Branded_Code_ Sawpeedddoooe_SSeeeeaerreee Share_Partners, MTTA_cooeeMMrketinn...... Marketing aperating_Airlinng DOT Nor__D_.perrating_AiZll. Airline, Baaatt/Wuueee_aaaettiiggggg Tail_Number:
Batpgght__ummbe__peraattttt OriginAirportID, OrigimAirrportSqqDD, OriginCi OriginCityName, OriginState, OriganStatoFips, OriginStatellame, OriginWac, DestAirportID, DestAlrportSeqID, DestCityMard tastCityMarkttID, DestCityName, DestState, DestStateFips,
DestStateName, DestWac, DepDel15, DeparturebelayGroups DepTimeBlk, Taxi0ut, WheelsOff, WheelsOn, TaxiIn, CRSArTTime, ArrDelay, ArrDel15, ArrivalDelayGroops, ArrTimeBlk, DistanceGroup, DivAirportLanding:
Thought need filter the information related the query- Action 2: FilterDB (wwightttuubbbbtteee Marketting_Arrlnneeeee Origin=ATL, Dest=MCO]
FlightDate
Observation 2: We have successfully filtered the data (1 row). Thought 3: We then need to know the departure delayed time.
Action GetValue [DepDelay]
Observation 3: -7.0
Thought 4: We then need to know the arrival delayed time.
Action 4: GetValue [ArrDelay]
Thought 4: -17.0
Observation
compute the extra minutes, we need to subtract the departure delayed time from the arrival delayed time.
Action 5: Calculate
Observation 5: -10
22

---

Thought 6: After calculation, know that the answer is -10.
Action 6: Finish[-10]
Question:
Michael had 58 golf balls. On tuesday, he lost 23 golf balls. On vednesday, he lost 2 more. How many golf balls did he have at the end of wednesday?
write python program solve this problem. Action 1: PythonInterpreterf. [# solution in Python: def solution():
Thought Maybe
# Michael had 58 golf balls. On tuesday, he lost 23 golf balls. On wednesday, # he lost
How many golf balls did he have at the end of wednesday? solf_balls_initial. 58
tuedday 23
golf_balls_ _eft golf_ bawls_alllllaaaeaaa biillsinntiall golf_ loss_tuesday
golf_ loaall___oot_wwdeeddaa
result goll_balls__efff
return result]
Observation 1: 33
Thought 2: After execution of Python Code, know that the answer is 33.
Action 2: Finish(33)
Question: What is the corresponding Mean_ IoU score of the FRRN method on Cityscapee dataset for Semantic_Segmenaatiie
task?
Thought 1: The question is asking
performance information about IoU the FRRN method on Cityscapes dataset for Semantic_Sege Serriiittmmettiionnn task, we need to retrieve some useful information from the scirex database.
Action 1: RetrieveScirex [Mean_ IoU score of the FRRN method on Cityscapes dataset for Semantic_ Sernnttic_Seeeeetttiiin task]
Observation 1: The other stream undergoes sequence pooling operations obtain robust features for recognition The two streams are coupled at the full image resolution using residuals Without additional processing steps and without pretraining
approach achieves intersection over union score of 71.8 the Cityscapes dataset
Thought the approach achieves an intersection over union score 71.8 X on the Cityscapes dataset, the answer should be 71.8%.
Action 2: Finish[71.
Question:
paper Learning the Principle of Least Action with Reinforcement Learning. published?
When
Thought 1: The question is asking some basic information of a dblp paper. We need to load the DBLP graph.
Action 1: LoadGraph [dblpp
Observation 1: DBLP graph loaded.
Thought 2: The question is asking the published date of a paper, we need to check the node from the PaperNet in DBLP graph. Action 2: NodeCheck (PaperNNt, Learning the Principle of Least Action with Reinforcement Learning.]
Observation 2: (ttile': 'Learning the Principle of Least Action with Reinforcement Learning.', ; 'year': 2021, 'venue;; ('raw": "AAAI Spring Symposium MLPS'), 'n_citation'; 0, 'keywords'; [,, 'doc_type"; 'Conference', 'page_start'; " "page_end' ">
Thought 3: The published date of the paper is 2021.
Action 3: Finish[2021)
Question: How many collaborators does Chao Zhang have in the DBLP graph? Thought The question is asking the number of collaborators of person, need load the DBLP graph.
Action 1: LoadGraph [dblp]
Observation 1: DBLP graph is loaded.
Thought 2: The question is asking the collaborators of person, need check the neighboring nodes from the AuthorNet DELP graph.
Action 2: NeighbourCheck (Authortee,, Chao Zhang]
Observation 2: ['YHHUI YUAN?, 'Rao Fu', 'Lang Huang', "Weihong Lin', "X Chen', > Jingdong Wang']
23

---

Thought 3: The number of collaborators of Chao Zhang is 6.
Action 3: [in
Question:
How many papers does Chao Zhang and Weihong Lin have in common in the DBLP graph?
Thought 1: The question is asking the number of common papers of two persons, need load the DBLP graph.
Action 1: [db1ppppp[hbppp)
Observation 1: DBLP graph is loaded.
Thought 2: The question is asking the common papers of two persons, we need to check the edges between them from the Paperlet in DBLP graph.
Action 2: EdgeCheck
Chao Zhang, Weihong Lin]
Observation 2: ('ueight': 1, 'papers': ['HRFormer: High-Resolution Vision
Transformer for Dense Predict. '], 'n, citation': [95]}
Thought 3: The number of common papers of Chao Zhang and Weihong Lin is 1.
Question: Where did Stephen's Opera performance take place?
Thought 1: The question is aaking the location of Stephen's Opera performance from agenda.
Action 1: (Steeeea Opera performance]
Observation 1: January 29, 2022, there will be an opera performance the Lyric Opera House, featuring Stephen. The show will start at 7:00 PM 9:00 PM. It promises to be a wonderful evening of beautiful music and powerful performances in stunning venue. Come and experience the magic of opera at its finest!
at
Thought 2: The event happened Lyric Opera.
Action 2: Finish [Lyric Opera]
Question: What the trading volume of coffee 2000-01-14? Thought Maybe
write a SQL query to solve this problem.
Action 1: SQLInterpreter (SELECT Volume coffee. coffee_ data WHERE Date
22000-01-1';
Observation 1: Volume: 10115
Thought 2: The volume of coffee 2000-01-14 is 10115.
Action 2: [10115]
The prompts used in Chameleon [32]:
<Chameleon> Prompt
You need to act as a policy model, that given a question and a modular set, determines the sequence of modules that can be executed sequentially can solve the question.
modules are defined as follows:
Calculate [formula) This module calculates given formula and returns the result. It takes in mathematical formula and returns the calculated result. Normally, we only consider using "Calculate" when question involves mathematical computations.
Retrievedgenda [keyword]] This module retrieves agenda related specific keyword and returns it. takes in keyword and agenda. Normally, we only consider using "RetrieveAgenda" when the question about specific actions or tasks related to a topic.
the corresponding
RetrieveScirex (keyRord]: This module retrieves paragraphs from machine learning papers related the specified keyword returns them. It takes keyword and returns the relevant paragraphs. Normally, we only consider using "RetrieveScirex" when the question involves underatanding specific concepts in machine learning.
LoadDB [DBName]: module loads database specified by the database name and returns the loaded database. It takes in a database name and returns the corresponding database. The DBName can be one of the following: flights/ coffee/ airbbb/eebbbbpp.....eb Normally, only consider using "LoadDB" when the

---

question requires data from specific structured dataset.
[1lterD[[columm name, relation, value]: This module filters database by specified column name, relation, and value, and then returns the filtered database. It takes in column name, relation, and value, and returns the filtered database. Normally, only consider using "FilterDB" when the question requires specific subset data from structured dataset.
GetValue [columm_[[ : This module returns the value of a specified column in database. It takes in column name and returns its value. Normally, only consider using "GetValue" when the question requires specific piece of data from structured dataset.
LoadGraph [GraphName]] This module loads a graph specified by the graph name and returns the loaded graph. It takes in graph name and returns the corresponding graph. Normally, we only consider using "LoadGraph" when the question involves understanding or navigating specific graph structures.
NeighbourCheck (GraphName, Node]: This module lists the neighbors of specified node in graph and returns the neighbors. It takes in graph and node, returns the node's neighbors. Normally, only consider using "NeighbourCheck" when the question involves understanding relationships graph structure.
(GreChhhhcGGraaaaaaaaaa Node]: : This module returns the detailed attribute information of a specified node graph. takes graph name a node, and returns the node's attributes. Normally, we only consider using "NodeCheck" when the question requires information about a specific entity in graph.
(aggeCheckGGraplaaaaa Node1, Node2] This module returns the detailed attribute information of the edge between two specified nodes in a graph. It takes in graph name and two nodes, and returns the attributes of the edge between them. Normally, only consider using "EdgeCheck" when the question involves understanding the relationship between two entities in graph.
SQLInterpreter This module interprets a SQL query and returns the result. It takes in SQL query and returns the result of the query. Normally, only consider using "SQLInterpreter" when the question requires data manipulation extraction from structured dataset.
PythonnItterpreterrPPyye (Python) This module interprets Python code and returns the result. It takes in Python code and returns the result of the code execution. Normally, we only consider using "PythonInterpreterr" when question requires complex computations or custom data manipulation.
Finish [answer]: This module returns the final answer and finishes the task. This module is the final module in the sequence that encapaulates the result of all previous modules.
Below are some examples that map the problem to the modules.
Question: many extra minutes did the DL1575 flight take ATL MCO 2022-01-12?
Modules: ("LosdD8[fllghht]]] "FilterDB(Flight. (FiitrDDBFFLighttNWWbeerr Marieeirraaateiiinn.naa Marketiing Airline-1575, FlighcDatee=2022-01-22 Origin=ATL, Dest=MCo]", ", "GetValue [DepDelay] "GetValue [ArrDelay (Ca))
"Fimish[-10]"]
Question:
Michael had 58 golf balls. On tuesday, he lost 23 golf balls. On wednesday, he lost 2 more. How many golf balls did he have at the end of wednesday?
Modules: ("PychonInterpreeeer solution Python: Anne solution () An Michael had 58 golf balls. On tueaday, he lost 23 golf balls. On wednesday, he loat 2 more. How many golf balls did he have at the end of wednesday?
25

---

helf_balleiimttial = 58hn golf_balls_lost_tuee = 23
golf_ solf_balls_lose seee_dddda 2 golf_balls_left
solf_balls_init result golf_ haaf_balls__eft11 return result]", "Finish [33]
solf_balls_lott__ee_e_
golf_
Question: What is the corresponding Mean_IoU score of the FRRN method Cityscapes dataset for Semantic_Segeentation task?
Modules: ("ScirexRetrieve [Mean_ IoU score of the FRRN method on Cityscapes dataset for Ciibantic_Sementatttt task]", (71.ish(71.8%)1
Question:
When was the paper Learning the Principle Least Action with Reinforcement Learning. published?
Modules: [[ood [dblp]", (Papeerrcc(aapreeeee Learning the Principle of Least Action with Reinforcement Learning. 1". "Fimish [2021] "]
Question:
How many collaborators does Chao Zhang have in the DBLP graph?
Modules:
[dbplp "NeighbourChheck (AuthoroCCCeCCCCttttcaAaCC [AuthorNet, Chao Zhang]", ", (F]
Question: How many papers does Chao Zhang Weihong Lin have common in the DBLP graph?
Modules: [dblpp "EdgeCheck (Paperrlie,, Chao Zhang, Weihong Lin] "Pimish[1]"]
Question:
Where did Stephen's Opera performance take place?
Modules: I"AgendaRetrieve (Stephen's Opera performance]", "Finish Opera]
Question: What was the trading volume of coffee on 2000-01-14?
Modules: ["sQLInterpreter (SELECT Volume FROM coffee coffee_dateeeeocoeeeeeeeeeeeeoae WHERE Date "2000-01-14''", "1111s][
Now, you need to act as policy model, that given question and a modular set, determines the sequence of modules that can be executed sequentially can solve the question.
Key Information of ToolQA
G.1 Dataset Documentations
The dataset is provided in jsonl format. Each task corresponds to two files: easy and hard (e.g., "flight-easy.jsonl"" and "figgh--hard,jonnl" etc.). Each data point contains the following fields:
qid: the unique identifier for the question-answe answer pair;
question: the question query;
answer: the corresponding ground-truth answer to question.
G.2
Intended Uses
ToolQA is intended for researchers in machine learning and related fields to innovate novel methods for tool-augmented large language models (LLMs). We also aim to help developers to test their plugins our dataset.
G.3 Hosting and Maintenance Plan
TooIQA codebase is hosted and version-tracked via GitHub. It will be permanently available under the link https: / //gi htgttthbb... hto.b.cooooo..aoooooooooa..a.eee niihtt chanttt-hh//TooQ... The download link of all the datasets can be found in the GitHub repository.
26

---

ToolQA is community-drinn and open-source initiative. We are committed and have resources to maintain and actively develop ToolQA in the future. We plan grow ToolQA to include more tasks, tools, and more baseline methods. We welcome external contributors.
G.4 Licensing
We license our work using Apache 2.0 All the datasets will be publicly released through the aforementioned GitHub link.
G.S Statement
The authors will bear all responsibility in case of violation of rights.
Phttps://ww.appche http://wwwwpppeeeeere///eeeeee/s http://iiioneee/I//SSSSL-..
27


Reviews:
[{"review_id": "AasciK6TeI", "reviewer": "Reviewer_sogC", "summary": "", "questions": "", "limitations": "Yes, the error analysis is good.\nMore on data quality studies and use of instruction paradigm to mitigate errors will be useful.", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "- focus on a timely problem to distinguish between questions that can be answered using LLMs’ internal knowledge and those that require external information through tool use.\n- introduce a new dataset called ToolQA to evaluate LLMs’ ability to use external tools for question answering.\n- semi-automated process for dataset curation, along with several specialized tools designed for interaction with external knowledge in order to answer questions. \n- diagnosis of existing tool-use LLMs to highlight their strengths, weaknesses, and potential improvements. \n- data publicly available.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "WBahszdx4E", "reviewer": "Reviewer_H5AJ", "summary": "", "questions": "", "limitations": "None", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "This paper explores LLMs' ability of external tool using more fairly and explicitly.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "0cWRyXWn12", "reviewer": "Reviewer_MPGH", "summary": "", "questions": "", "limitations": "Limited Tool Diversity: The current ToolQA dataset includes 13 specialized tools, which may not fully represent the vast array of potential external knowledge sources and tools that exist in real-world applications.\nLess-scalable answer generation: It seems that the answers have to be carefully given so that the tools can be used properly, which may involve non-trivial amount of human work, leading to less scalability. Also, the current programmatic answer generation process assumes that each question has a fixed and pre-determined answer, which may not fully capture the complexities of real-world question answering scenarios with ambiguous or context-dependent answers.", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "The dataset excels in several major aspects as below:\n\n- Large-scale dataset: ToolQA's development involves a three-phase automated dataset curation process, ensuring efficiency and minimizing the need for extensive human labeling efforts. The benchmark data in ToolQA is meticulously crafted to have minimal overlap with LLMs' pre-training data, ensuring a more accurate evaluation of LLMs' abilities. ToolQA comprises data from 8 different domains.\n- Evaluation: The paper addresses the existing challenge in evaluation methods, which fail to distinguish between LLMs' internal knowledge and their genuine use of external tools for problem-solving. ToolQA includes 13 specialized tools that allow interaction with external knowledge, providing a comprehensive evaluation of LLMs' reasoning abilities when using these tools. Also, the benchmark data in ToolQA is meticulously crafted to have minimal overlap with LLMs' pre-training data, ensuring a more accurate evaluation of LLMs' abilities.\n- Insightful Performance Analysis: Experiments on ToolQA reveal the substantial performance gap between standard LLMs and tool-augmented LLMs, highlighting the dataset's effectiveness as a challenging benchmark.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "fRiFpJk6Al", "reviewer": "Reviewer_HHeV", "summary": "", "questions": "", "limitations": "Any discussion of potential misuse of the ToolQA dataest? Like LLMs can be fooled/prompted to use malicious tools.", "rating": 9, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "The paper studies an important research problem: the evaluation of the tool-use ability of LLMs.\n\nToolQA creation process is scalable and automated, the 13 specialized tools are diverse and comprehensive. \n\nEvaluation results of LLMs on the ToolQA dataset show the proposed benchmark is effective. The proposed ToolQA benchmark is challenging for state-of-the-art LLMs and can faithfully evaluate the LLMs' ability to use external tools for question answering. \n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "c55aeRze7n", "reviewer": "Reviewer_3WCN", "summary": "", "questions": "", "limitations": "The reviewer did not find that the authors have adequately addressed the limitations and potential negative societal impact of their work.\n\nThe most critical limitation of this work the reviewer found is that all the baselines considered in this study are based on black-box product APIs. While different approaches are considered, using only the same black-box models makes the reviewer feel how scientifically meaningful the resulting findings are.\n\nAlso, the code repository still misses Chameleon code.", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "- This study is well-motivated and addresses a few critical problems in the existing LLMs, considering a risk that the external data used for evaluation may have already been exposed to the models during the pre-training.\n- To discuss how well LLMs can use external tools to provide correct answers, the authors designed experiment pipeline such that LLMs would not correctly answer the questions if the models do not use the right tool and right data sources.\n- The result analysis is very detailed and provides interesting findings, especially how a method failed to correctly answer questions (incorrect tool calls, incorrect data sources, too long context, etc).", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: ToolQA is a dataset for evaluating tools in question answering.  This will be a timely addition to the Benchmarks track and will help push forward research in tool use, an area which is rapidly growing.
Decision: Accept (Poster)