MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
Authors: Yue Huang
Jiawen Shi
Yuan Li
Chenrui Fan
Siyuan Wu
Qihui Zhang
Yixin Liu
Pan Zhou
Yao Wan
Neil Zhenqiang Gong
Lichao Sun
Venue: ICLR 2024 poster (2024)
Keywords: Tool usage
large language model
benchmark
dataset
Primary area: datasets and benchmarks

Abstract: Large language models (LLMs) have garnered significant attention due to their impressive natural language processing (NLP) capabilities. Recently, many studies have focused on the tool utilization ability of LLMs. They primarily investigated how LLMs effectively collaborate with given specific tools. However, in scenarios where LLMs serve as intelligent agents, as seen in applications like AutoGPT and MetaGPT, LLMs are expected to engage in intricate decision-making processes that involve deciding whether to employ a tool and selecting the most suitable tool(s) from a collection of available tools to fulfill user requests. Therefore, in this paper, we introduce MetaTool, a benchmark designed to evaluate whether LLMs have tool usage awareness and can correctly choose tools. Specifically, we create a dataset called ToolE within the benchmark. This dataset contains various types of user queries in the form of prompts that trigger LLMs to use tools, including both single-tool and multi-tool scenarios. Subsequently, we set the tasks for both tool usage awareness and tool selection. We define four subtasks from different perspectives in tool selection, including tool selection with similar choices, tool selection in specific scenarios, tool selection with possible reliability issues, and multi-tool selection. We conduct experiments involving eight popular LLMs and find that the majority of them still struggle to effectively select tools, highlighting the existing gaps between LLMs and genuine intelligent agents. However, through the error analysis, we found there is still significant room for improvement. Finally, we conclude with insights for tool developers -- we strongly recommend that tool developers choose an appropriate rewrite model for generating new descriptions based on the downstream LLM the tool will apply to.

Published as a conference paper at ICLR 2024
METATOOL BENCHMARK FOR LARGE LANGUAGE MODELS: DECIDING WHETHER TO USE TOOLS AND WHICH TO USE
Yue Huang' Jiawen Shi?, Yuan Li³, Chenrui Fan", Siyuan Wu', Qihui Zhanga Yixin Liu?, Pan Zhou?, Yao Wan Neil Zhenqiang Gong', Lichao Sun!
Lehigh University'
Huazhong University of Science and Technology? University of Cambridge
Duke University*
ABSTRACT
Large language models (LLMs) have garnered significant attention due their impressive natural language processing (NLP) capabilities. Recently, many studies have focused on the tool utilization ability of LLMs. They primarily investigated how LLMs effectively collaborate with given specific tools. However, in scenarios where LLMS serve as intelligent agents, as seen in applications like AutoGPT and MetaGPT, LLMs expected to engage in intricate decision-making processes that involve deciding whether to employ a tool and selecting the most suitable tool(s) from collection of available tools to fulfill user requests. Therefore, in this paper, we introduce METATOOL, a benchmark designed to evaluate whether LLMs create
have tool usage awareness and can correctly choose tools. Specifically, dataset called TOOLE within the benchmark. This dataset contains various types of user queries in the form of prompts that trigger LLMs to use tools, including both single-tool and multi-tool scenarios. Subsequently. we set the tasks for both tool usage awareness and tool selection. We define four subtasks from different perspectives in tool selection, including tool selection with similar choices, tool selection in specific scenarios, tool selection with possible reliability issues, and multi-tool selection. We conduct experiments involving eight popular LLMs and find that the majority of them still struggle to effectively select tools, highlighting the existing gaps between LLMs and genuine intelligent agents. However, through the error analysis, we found there is still significant room for improvement. Finally, conclude with insights for tool developers we strongly recommend that tool developers choose an appropriate rewrite model for generating new descriptions based on the downstream LIM the tool will apply to. Our TooLE dataset is available at URL and code is in Github.
INTRODUCTION
Toolwempowered large language models (LLMs) (Qin al., 2023a;b; Patil et al., 2023: Ruan et al., 2023; Cao et al., 2023; Zhou et al., 2023a) have recently attracted widespread attention. An important milestone for L.MSs marching toward intelligent agents (Park et al., 2023; Li al., 2023e) the flexible use of tools (e.g., APIs (Qin et al., 2023b; Rapid, 2023) and plugins (OpenAI, 2023d)) to fulfill users' requirements. By utilizing tools, LLMs can obtain real-time data, such as getting the latest weather forecast (GPTStore, 2023); enhance interactions with users, like helping users book flight tickets (Deng et al., 2023); and better deal with uncertain questions by querying knowledge bases (Li et al., 2023c; Hu et al., 2023) Internet (Lazaridou et al., 2022). Moreover, LLMs also leverage specific tools to process multimodal information, thereby acquiring the same capabilities as multimodal models (Zhang al., 2023; Yan et al., 2023; Yuan et al., 2023b). The capacity to use
"Lichao Sun and Yue Huang are co-corresponding authors: lis221 @ @lehigh.edu, howichwong swwichhwng@gmmil.com gmail.com Visiting Students LAIR Lab, Lehigh University.

---

Published as a conference paper at ICLR 2024
Table 1: Comparison of previous work and METATOOL.
Dimeussoo
APIRank
GPT4Too
APIBench (Paril ot 2023)
ToolLLM
ToolBennch (Ku 2023)
ToolQ
Meta (Oun)
(L 3023d)
(Yang ot al., 2003))
(Qin et 20236
Zhuing al. (2023)
Evaluation Rango
(0:3)
(0:34)
02
Number of Tosks
Reliability Test
X
X
V
Multi-Tou Rool
X
Different Secnarios
X
X
X
tools enables L.MSs to break through their own limitations, acquire external information, and thereby make more accurate and effective responses, providing users with better service.
Previous research has focused on how to enhance the ability of LLMs to use tools, including training models with instruction related tool usage (Qin et al., 2023b; Tang et al., 2023; Schick et al., 2023), or augmenting the model's ' problem-solee solving capabilities for domain-speciff tasks through external APIs (Yang et al., 2023c). typical process of employing LLMs to use tools is illustrated in Figure 1. Initially, users input question (i.e., query) that triggers the tool usage. Based on prior research (Yang et al., 2023c; Qin al., 2023b), under the ReAct (Yao al., 2022) prompt approach, the process of using tools can be divided into four stages: Firstly, LLMs consider whether to employ a tool (0) and if so, which tools to select (2). The tool selection process involves directly having LLMs choose from provided tool list (Yang al., 2023c) or selecting via retriever (Qin et al., 2023b). Next, LLMS configure the users' input as tool parameters (3), then handle the results from the tool ()) and finally return the outcomes to the user.
With the emergence of more and more LLMs like open-source Llama2 (Tou- vron et al., 2023), Vieuna (Chiang al., 2023), and closed-source ones like ChatGPT (OpenAI, 2023a) and GPT-4 (OpenAI, 2023b). designing a comprehensive benchmark to measure the tool-related capability of these models has become crucial. Current studies have proposed several bench- marks (Xu al., 2023; Qin et al., 2023b; Li et al., 2023d) about tool us- age for LLMs, with the main contributions being limited to the stages 3 and ④ However, the awareness of tool usage (O) and tool selection (Ø) ability are also important for LLMs when they're acting as intelligent agents including AutoGPT (Significant-Graviaas 2023), MetaGPT (geekan, 2023) and Baby AGI (babyagi, 2023), or in the multi-agent environment where LLMS need to tools solve collaborative tasks (Shen al., 2023; Qian al., 2023; Park al., 2023; Cai et al., 2023). As result, it is necessary to establish a benchmark to evaluate LLMs' tool usage consciousness and tool selection ability.
Thought: Do
tool?
Can you recommand some popular attractions New York City?
Tool name use. Action Input: The Input tool.
Observation: The results from the tool.
Query
Use tooks
Users
Return results
LLMs
Return results
Tool database
Figure 1:
Tool usage pipeline of LLMs. METATOOL includ- ing awareness of tool usage (0) and tool selection
The difficulty in establishing such a benchmark is reflected in two aspects. The first one is the dataset: previous research proposed datasets (Qin al., 2023b; Xu al., 2023) lacked diverse user inputs, making it hard to cover various real-world scenarios. Additionally, there is an issue of overlapping in the dataset, meaning that a user's needs can be addressed by more than one tool, which makes challenging to conduct evaluations since user inputs can correspond to multiple tools. The second aspect is the task setting: the benchmark should include different tasks to evaluate LLMs from different perspectives, such reliability, the performance under different scenarios in daily life. To address these issues, we propose METATOOL, a benchmark designed to evaluate the awareness tool usage and tool selection capability of LLMs. As demonstrated in Table 1, METATOOL distinguishes itself from previous research efforts and is structured into three primary components:
TooLE dataset. We introduce TOOLE, a comprehensive dataset that encompasses a wide range of 21,127 user queries, with both single-tool and multi-tool queries. Different from the previous single-method generation (Yang et al., 2023c; Qin et al., 2023b), these queries are generated using various prompting methods, including emotional generation, keyword generation, direct
2

---

Published as a conference paper at ICLR 2024
Tool selection with similor choloes
Tool solection in specific scenarios
Useea Outtee C Vore il Woreetb Larab-20
Good dayt I'm hoppng fndd scmm
Taant Eall the afferonoc of
notharm wngg Foee?
arracc soringg brerjeane and ciseournt to totley. Son you guido me the right direction?
Shoudl wolk solva the queey?
I'm feeling realty down today, con you summaatce thea You Tutas vidao for me? [Link You Tube videot
cesado
Could you recommend some popular shoopang mails maticts in New York (ity that wide nange products including International brands, food options, uniqque shopping experiences?
selection with possiible reliabiltty
Multi-tzol
Musdi
Arcooer 3:
A
Noroo
FI Seer
reetion
What ane the franding a tapica on Twitter U.S.A?
Roual
03) Diverse Queries
(bi Taal Usage Motivation and Toal Selection
(CI Evalation Results
Figure 2: METATOOL benchmark architecture. contains the dataset ToooLE with diverse queries related to different tools (a), and based on it, we conduct the evaluation of the awareness of tool usage and tool selection (b) and finally obtain the results of eight prominent LLMs (c).
diverse generation, and detailed generation. Moreover, to address the challenge of overlapping tool functionalityy, we undertake tool merging and decomposition. Evaluation awareness of tool usage and tool selection. We construct a test set to evaluate the awareness of tool usage based on TOOLE and existing instruction datasets. Moreover, we formulate four distinct tasks to evaluate the tool selection ability of LLMs. These tasks are thoughtfully designed to assess semantic comprehension, adaptability, reliability, and inferential capability, namely tool selection with similar choices, tool selection in specific scenarios, tool selection with possible reliability issues, and multi-tool selection.
Empirical analysis on results. We rigorously evaluate the performance of eight well-known LLMs. We have observed that most LLMs struggle to recognize their capability boundaries and lack a good awareness of tool usage. Regarding tool selection, we find that while I.LMs possess basic tool selection capabilities, the tool selection of most L.MMs remains unreliable, with noticeable variations in performance across different daily scenarios. Moreover, the error analysis indicates there is still room for improvement in tool selection. Finally, by analysis the tool description, we gained two insights for tool developers.
2
METATOOL DESIGN
2.1
PRELIMINARY & REQUIRED ABILITIES
In this section, we first introduced the composition of the TOOLE dataset, outlining how we generated user queries related to tools. Subsequently, we explained how we set up the evaluation tasks, including tool usage awareness and tool selection. The evaluation mainly requires LLMs with the following properties and abilities: (1) Less hallucination and sycophancy. The awareness of tool usage can reflect the truthfulness about whether an LLM has a clear understanding of its capabilities (e.g.. realizing its capability limitation about what problems it cannot solve well and using tools for assistance), thereby helping to mitigate issues of hallucination (Ji et al., 2023; Sun et al., 2024) and sycophancy (Wei al., 2023). (2) Recommendation and retrieval. Moreover, existing research has tentatively explored the potential of LLMS in applications like LLM--ased recommendation systems (e.g., tool recommendation for users) (Gao al., 2023; Wang et al., 2023e; Dai al.. 2023). In LLM-as-agent scenarios. LLMs usually need to select the specific tool according to the text description (Park et al., 2023; Shen et al., 2023; Ruan et al., 2023), actually is a kind of information retrieval (Sun et al., 2023), making the ability of tool selection crucial. (3) Task-level abilities. In METATOOL, we set four tasks as shown in Table 12. Incorporating similar tools for selection (i.e., Task 1) requires high-level semantic comprehension for LLMs, and tool selection in specific scenarios tests the flexibility of LLMS when using tools in different scenarios (c.g., finance (Wu et al., 2023a) and biomedical domain (Zhang et al., 2023; Wang et al., 2023c)). Task 3 aims to explore the internal hallucination and reliability extent of LLMs when using tools and Task 4 is designed to evaluate the inference ability (e.g., order of using multiple tools) (Creswell et al., 2022) of LLMs.
3

---

Published as a conference paper at ICLR 2024
Direct Diverse Generstion
Generation
Emational Generation
Keyword Generation
Mergence
Decompositin
Tool-Description
Singlo-Tool Quary Generation
Overlapped Issue Solving
Multi-Tool Query Generation
Human Checking
Figure 3: The process of dataset generation.
2.2 TOoLE DATASET
In this section, we introduce the TOOLE dataset with 21. diverse user queries related to tool usage. Each entry within the dataset comprises user request (i.e., query) along with its corresponding tool name and tool description. These queries serve as triggers that prompt LLMs to utilize specific tools. The step-by-step process employed for generating the dataset is shown in Figure 3.
2.2.1 DATASET GENERATION
Tool description. Tool description is important for LLMs to use them (Hsich et al., 2023). We retrieve tool names and descriptions from OpenAl's plugin list (OpenAI, 2023d). The reason for selecting Open Al plugins as the data source for our tools is that these tools have been installed in ChatGPT (OpenAI, 2023a) and GPT-4 (OpenAI, 2023b), and they have been widely used, making them more practical. We obtained names and descriptions for a total of 390 tools across different domains. We show more details about tool descriptions in Appendix A.5.
Single-tool queries generation. Next, describe how we generated queries. Inspired by prior studies (Qin et al., 2023b; Yang et al., 2023c), our approach revolves around incorporating a tool's description into prompt while implementing specific constraints to guide the generation of user queries by ChatGPT//PTT-4 We adopt four distinct techniques for query creation: direct diverse generation, emotional generation, keyword generation, and details generation. We show the data examples generated by different prompt ways in Table 8 of the Appendix A.5. (1) Direct diverse generation. We introduced conditional criteria within the prompt to encourage ChatGPT/GPT-4 to produce variety of query types, encompassing distinct tones (such as requests or orders) and levels of detail. (2) Emotional generation. Building on prior research (Li et al., 2023a; Goodside, 2023), which highlights the influence of emotion within prompts on model performance, we augmented the prompt with constraints to guide ChatGPT generating content in different emotions. Here we used four distinct emotions- happiness, excitement, anger, and depression. (3) Keyword generation. Direct generation occasionally fell short in capturing specific description details, such as tools limited to particular regions, we devised the generation way through key keywords. This way involved ChatGPT extracting keywords from the tool's description and then we incorporated both the extracted keyword and the tool's 's description within the prompt, tasking ChatGPT with generating queries focused on the given keyword. (4) Details generation. To add more details to the queries, we instructed ChatGPT to add details augment the original queries generated by direct diverse generation methods.
Overlapped issue. Overlapped issue refers to query that can be solved by multiple tools. If left unaddressed, this overlap could potentially influence the computation of final metrics. For instance, given query g, the corresponding tool in our dataset is ta yet an alternate tool to could also feasibly address the same query q. single-label scenario, the accuracy of tool selection becomes compromised. To address this, we merge the group of tools with similar functions as a single tool. Meanwhile, if a tool can function for multiple purposes across the groups of tools. the corresponding generated queries cannot be simply merged into any one of them. So decompositions are needed for the queries of these tools before merging. After decomposition and merging, each query in our dataset has only ground truth label. The decomposition and merging operation follows three steps, and details about this can be found in Appendix A. We also show the efficiency of our operation in Appendix A.2 through the silhouette coefficient.
Multi-tool queries generation. Unlike single-tool queries, we generate multi-tool queries after addressing the overlapped issue because it is challenging to map the original labels to new labels in lubti-label (i.c.. multi-tool) situations. Here, we only consider queries related to two tools. We observe that if we obtain combinations of two tools by iterating through all the tools (i.e., C2 iterations,
4

---

Published as a conference paper at ICLR 2024
where is the size of the toolset), there would be many tool combinations that are not practical (i.e., rarely encountered in daily life, such as the combination of fortune-telling tools and currency exchange tools). Therefore, we select the top 15 most popular tools from the toolset, and for each pair of tools, we generate queries. We determine the popularity of a tool based on the number of tools it is merged with, shown in Appendix A.5. The multi-tool queries we generate can be divided into two types: The first category pertains to situations where tools are employed in parallel, indicating that the utilization of each tool operates independently of the others. The second category deals with cases where tools are used causally, signifying that the deployment of one tool may be contingent upon the outcomes of preceding tool. Detailed prompt templates can be found in Appendix D.2. Similar to single-tool queries, we also manually verified multi-tool queries to ensure the combination of tools was reasonable and the query of the tool corresponded to the tool description.
Human checking, We conducted manual verification of all queries in TooLE, including the removal of non-compliant queries and tools, as well as the handling of queries corresponding to special categories of tools. Detailed guidelines for human validation are provided in Appendix A.3.
2.3 TASK FORMULATION
We seek to address two research questions in this paper: (1) To what extent can L.Ms be conscious of their limitations and ask for assistance from external tools? (2) How effectively can LLMs select the tools when they ask for assistance? To answer these questions, we design two tasks based on the TooLE dataset to evaluate the capacity of LLMs regarding tool usage.
2.3.1 AWARENESS OF TOOL USAGE
In this part (i.e., Thought (Q)), we aim to investigate the awareness of tool usage of LLMs; that is, whether LLMs can resort to external tools when they encounter problems they cannot solve. To this end, we need to construct the test set with both positive and negative samples. Positive samples are the queries that can not be solved by L.LMs themselves and need tool usage, whereas negative samples are queries that can be directly solved by LLMs and therefore do not necessitate tool usage. For positive samples, we selected a subset of samples from TOoLE and conducted manual validation to confirm whether they would trigger LLMs to use the tool (the process of which is detailed in the Appendix B). As for negative samples, we select three recent instruction datasets, including instructions about downstream tasks (Wang et al., 2022), common-sens sense questions (Talmor et al., 2019), and high-quality instructions used in LIMA (Zhou et al., 2023b). Similarly, we conducted manual verification to ensure that these requests can be resolved by LLMs' intrinsic capabilities. Specifically, we use the prompt with a query to inquire the LLMs whether need to employ tool or not, and the output of LLMs should be either "yes" or "no".
2.3.2 TOOL SELECTION
Preliminary. We propose four subtasks evaluate LLMs in tool selection "(i.e,, Action phase (+)). Generally, the prompt comprises a query q E Q (i.e., the user's input) and tool list L, (Le C T) containing n potential tool candidates. In the single-tool tasks (Sub-task 1~33), we designate the corresponding tool for query as t E T. In the multi-tool task (Sub-task 4), this corresponds to S, C (S.. > 1). Consequently, we obtain JAction C (L, U Ø) as the outcome of the tool selection process, where FAction represents the selected tool(s).
Sub-task 1: tool selection with similar choices.
The task is designed to challenge LLMs to select the correct tool from a tool list containing similar tools, thereby testing their thorough understanding of tool functionality. Given a query q with its label t, we task LLMS with selecting a tool from the specified tool list L, containing n candidates. To construct L, we first obtain the embedding of t's description, denoted as E(t), where E(.) represents the embedding function (here, we utilize the text-embedding -ieediingdda--O0 ada-002 model (OpenAI, 2023c) to generate embeddings). Denote the most similar tools of I. as top-(n - 1)., which are selected based on the cosine similarity of their embeddings: top-(n- 1); arg top-krerr,te sim (t(t), F(('). Consequently, I,+ (1) U top-(n 1)r-
Sub-task 2: tool selection in specific scenarios. The objective of this task is simulate how LLMs perform using tools when they act as controllers of a system (Shen et al., 2023) faced with
Wee separate the prompt of Thought (1) and Action (2) avoid the influence taken from different tool lists.
5

---

Published as a conference paper at ICLR 2024
different scenarios. As LLMs are widely applied across various domains like biomedical domain (?) and educational domain (Kasneci et al., 2023), in scenarios where the system caters to diverse demographies professions (c.g., software engineer (Qian et al., 2023)), its set of tools also varies. This task allows us to explore the performance disparities of L.LMs in selecting different kinds of tools, essentially highlighting form of bias inherent to L.MSs (Ferrara, 2023). In such cases, this task examines how effectively LLMs utilize the tools. Given query g with its label t, we specify the tool list L containing n candidates according its corresponding scenarios, This task consists scenario, and the second one is the
of two types of scenarios: the first one is the popularity-relate group-related scenario. For the popularity related scenario, we have selected the 5, 10, and 15 most popular tools based on the number of tools it's merged with (refer to Table 9 in the Appendix A.5 for details.) to construct the tool list. As for the group-related scenario, we chose six usual occupations or identities and manually curated a tool list consisting of the 10 most relevant tools for each of them (see Table 10 in the Appendix A.5 for details).
Sub-task 3: tool selection with possible reliability issues. The reliability of LLMs' tool selection is of utmost importance. However, issues like hallucination (Ji et al., 2023) and sycophancy (Wei et al., 2023) within LLMs' responses that will negatively impact their selection of tools. Therefore, introduce sub-task 3. In this task, given a query g and its corresponding tool t, we need to construct the tool list L, and ensure / Lt, This aims to assess whether L.MS can answer questions honestly and avoid issues like choosing non-existent tools or selecting unrelated tools. It should be noted that this task closely resembles real-world scenarios. as not all existing tools capable of addressing user queries are present in the tool list controlled by LLMs. To be specific, we obtain the embedding of t's description Et) and get the top-k, similar tools about as the way in Task 1. Then we randomly sample n tools from the rest tool set T to construct Lt, denoted as L, = {f1, l2, ..., fn } where t E (1 S i S n) and T" T ((t) U top-k,). Overall, we remove the ground truth tool of query q and l's most similar k tools to keep the tools in L, not related to / as much as possible.
Sub-task 4: multi-tool selection. In addition to testing the selection of single tools, like previous research (Qin et al., 2023b), we set up a task for multi-tool selection which may evaluate the inference ability and more complex semantic comprehension in the tool selection. We tested whether LLMs would correctly choose the specified tools by inputting multi-tool queries. Specifically, given query q with its related tool set St (|Sy) > 1), we construct the tool list L, containing n tool candidates (n > [S,)). Like the candidate selection way in sub-task 3, we obtain each tool t's embedding E(t) where t E S,, and get the most & similar tools of t,, t2, .... f/s,). denoted as top-k,, top-kez top- We randomly select (n (S.) tools from T = Utop-k, Utop-ks, U... U...Ukop-kk Finally, these (n - (S,)) tools and the tools E S₁ consist of the tool list L, The reason we do not include the most similar tool in L, like sub-task 3 rather than task 1 is that the multi-tool selection task itself is inherently challenging, and we do not want to further increase the difficulty.
3 EXPERIMENTS
3.1 EXPERIMENTAL SETUP
Model selection. We have chosen eight models that are currently excelling and popular in this field. These models include ChatGPT (OpenAI, 2023a), ChatGLM2 (6B) (THUDM, 2023). Llama2 (7b-chat, 13b-chat) (Touvron et al., 2023), Vicuna (7b, 13b, 33b) (Chiang et al., 2023), Baichuan2 (13b) (Baichuan, 2023) and Koala (13b) (Geng et al., 2023).
Prompt template and test samples. Due to the large scale of TOOLE. we sample from it as our test set (more details are shown Appendix C). For a better understanding of the importance of tool usage and to tell LLMS when need to use tools, we add the reasons for tool usage in the prompt template of Thought (Q) part. We show the detailed prompt template in Appendix D.2. We also conducted few-shot learning experiments for the first three tasks and details of the experimental design can be found in Appendix C.4.
Metries. For the awareness of tool usage evaluation, we use accuracy, recall, precision, and Fl score as the metrics. For tool selection, we propose the Correct Selection Rate (CSR) to calculate the percentage of correct selection action. Denote the output results for all queries as Y = {21 32,...) ... for a specific output y. we use A(y) to denote the tool(s) that the model chooses from the tool list. The CSR is computed as follows:

---

Published as a conference paper at ICLR 2024
Table The results for the awareness of tool usage test. We use accuracy (Acc.), precision (Pre.), recall (Rec.), and FI score (F1) evaluation metrics. And F1 A is the percentage change of F1 Score between zore-shot and shve-shot, as calculated by Fl ==
Flz=0
Metrics
ChatGPT
ChatGLM2
Llama2-7b
Ilama2-13b
Vicuna-7h
Vicuna-13b Vicuna-33b
Koala-13h
Acc.
74.85
54.56
53.88
51.94
52.43
61.17
65.05
54.95
Pre.
69.63
61.90
52.71
80.00
74.51
82.49
79.92
55.53
0
Rec.
88.16
40.39
84.85
6.21
7.38
28.35
40.19
52.62
F1
77.81
48.88
65.03
11.53
13.43
42.2
53.49
$4.04
Acc.
79.71
56.02
50.39
$7.86
56.12
55.15
61.55
$4.66
Pre.
73.98
64.32
50.55
63.31
56.49
52.92
57.85
52.61
Rec.
91.65
51.46
99.03
47.57
54.95
96.89
85.83
97.86
F1
81.87
57.17
66.93
54.32
55.71
68 .5.45
69.12
68.43
F1 À
4.06
8.92
1.90 f
42.79
42.28
26.25
15.63 f
14.39 1
for Task
CSR =
Σ DEY
A(v)
Task
(
[
Task
3.2 RESULTS ANALYSIS
Through the experiment results, we have gained the following conclusions:
Even under the few-shot prompts, the majority of LLMs still perform poorly in tool usage awareness. In Table 2, we observe that under the zero-shot prompt, only ChatGPT has both accuracy and Fl exceeding 70%, while the performance of other models relatively poor, with the F1 score of llama2- 13b being only 11.53%. Under the five-shot prompt, some models show significant improvement Fl scores, for example, llama2- 13b increased by 42. 42,79% and vicuna- 7b by 42.28%. This indicates that though few-shot learning generally improves the performance of L.I.Ms in tool usage awareness, they still lack sufficient tool usage awareness.
When selecting similar tools, there is significant performance disparity among existing LLMs, and the improvement brought by few-shot prompts limited. Table 3 shows that under the zero-shot prompts, the best-performing LLM is Vicuna-7b. with nearly 30% difference compared to the worst- perforrero Llama2- 13b. The gap between the best-performing ChatGPT and the worst-performing 1ama2-13bb still exceeds 20% under 5-shot prompts. Additionally, the maximum improvement brought by 5-shot prompts does not exceed 7%. Moreover, the performance of Vicuna- 7b even declined by 10% under the five-shot condition, suggesting potential bias in its O-shot performance, which reflects either a lack of robustness or over-sensitivity of the model.
LLMs still face serious challenges in dealing with reliability issues, for instance, reducing hallucination. As seen from Table 3, although few-shot prompts improve the performance of all LLMs, the CSR of most LLMs remains below 20%. We find that LLMs sometimes fabricate non-existent tools, a severe hallucination issue that has significant impact on LLM-based agents. Additionally, the potential sycophancy of LLMs may lead them avoid returning "none" answer, instead choosing irrelevant tools to respond to users.
Table 3: The CSR (%) for tool selection with similar choices and with possible reliability issues. A the percentage change of CSR between zore-shot and five-shot, calculated by CSR CSR
Metric
ChatGLM2 ChatGPT Llama2-7b Llama2-13b Vicuna-7b Vicuna-13b Vicuna-33b Koala-13b
0
54.17
69.05
45.95
44.06
73.46
58.23
53,96
56.34
Similar
== 5
57.44
72.94
51.12
49.85
63.67
63.15
60.54
60.85
3.27
3.89
5.17 +
5.79 f
-9.79
4.92 t
6.58 1
4.51
0
6.63
50.35
0.90
2.31
1.50
2.51
2.81
1.70
Reliability
x 5
15.68
78.49
2.51
5.93
1.81
3.42
3.11
5.83
9.05
28.14 1
1.61 +
3.62 +
0.31 1
0.91 1
0.30 1
4.13

---

Published as a conference paper at ICLR 2024
LLMs perform poorly in processing long texts. From Figure 4, all LLMs decreases as the length of the tool list increases, especially the range from top 5 to top 10. This indicates that LLMs still need improvement in understanding long texts. LLMs exhibit imbalances and biases in tool selection across different scenarios. For example, in Figure 5, L.MMs generally have higher CSR in tool selections related to the elderly and artists & designers, while their CSR is lowest for tools related to students. This means that developers still need to enhance the generalization capabilities of LLMs. the same time, for downstream applications, is best to choose suitable LLMs based on different applied fields.
that the CSR of almost
Top
Tap 10
Top 15
0.8
0.6
0.4
0.2
0.0
chatgim2
chatgpt
Ilama2-7b
Ilama2-33b
koala-
vicuna-7b
vicuna-13b
vicuna-33b
Model
Figure 4: The CSR results (%) of top (n=5,10,15) tool in different scenarios.
llama2
Softwre or gineer
Bderr
Financee staff
0.75
0.75
0.75
0.25
0.25
0.25
0.00
0.00
0.00
Artistskdhiirress
Students
1.00
1.00
1.00
0.75
0.75
0.75
0.25
0.25
0.25
0.00
Figure 5: The CSR result (%) of tool selection in specific scenarios.
Table 4: Multi-tool selection results. We evaluate LLMs' performance based on two kinds of prompt templates: one is telling LLMs to choose zero, one, or two tools (i.e., multi-choice), while another is foreing LLMs choose tools (i.e., one-choice). We consider the different kinds of CSR (%) for the former one: the LLM selects two correct tools (2/2 CSR), selects only one tool and it's correct (1/1 CSR), and selects two but only one correct (1/2 CSR).
Metric
ChatGPT ChatGLM2 Llama2-7b Llama2-13b Vicuna-7b Vicuna-13b Vicuna-33b Koala-13b
2/2 (CSR)
88.28
20.20
35.69
81.49
44.06
83.70
48.69
39.03
Multi-choices 1/1 (CSR)
3.03
36.57
21.98
0.00
25.55
1.01
48.49
1.61
(CSR)
1.01
13.94
8.87
11.07
7.04
8.05
0.20
20.32
One-choice
CSR
88.53
23.34
$7.34
77.87
64.34
78.47
91.15
25.10
There are significant performance differences among LLMs in multi-tool selection. As shown in Table 4, ChatGPT, the top-performing model, outperforms ChatGLM2, the worst-performing model, by nearly 70%, highlighting the variability in the capabilities of different language models for this task. Furthermore, the most common error made by the models is omitting tool selection, such as in the case of Vicuna-33b, which only selected
tool in 48.49% of cases. Moreover, several LLMs overly rely on the explicitly specified number of tools they should select in the prompts. As shown in Table when explicitly instructed return tools, Vicuna-33b' correct selection rate increased to over 90%, and Vicuna-76 7b also improved by over 20%. This indicates that these
8

---

Published as a conference paper at ICLR 2024
Table Error analysis results. The Top@ metric quantifies the proportion of incorrect choices by the model that are ranked within the Top @ positions of the similarity-ranker ranked list.
Top@k
ChatGPT
ChatGLM2
Llama2-7b Llama2-13b
Vicuna-7b
Vicuna-13b
Vicuna-33b
Koala-13b
Top@1
18.44
19.89
14.37
15.12
15.03
15,91
16.62
15.04
Top@3
34.29
36.48
34.08
34.94
34.43
35.94
35.09
35.34
Top@5
47.26
43.63
51.41
49.81
50.55
48.15
48.28
49.62
L.Mss still possess good multi-tool selection capabilities but require prior knowledge, which makes challenging to apply in L.L.-based agents.
Error analysis. We further investi- gate the reasons for errors in LLM's tool selection. We employ the Top@k k: metric to analyze failure cases in tool selection with similar choices, as shown Table 5. suggests that, despite being incorrect, the choices made by the model often retain a de- gree of similarity the correct tool. In general, all LLMs have nearly 50% chance of choosing a tool from the Top@5 most similar to the correct tool, and more than a 15% chance choosing the most similar one (i.e., Top@) I). This indicates that there is still significant room for improvement in tool selection with LLMs.
1974 Lanes Tee
1.44
-6.12
7.83
2.58
-1.37
.5.02
0.60
100
-13.85
-2.46
-..7
-1.23
14.99
-1311
-10
ChiGggg
Uameb-70 Lemu0-336
Wore-7
Awazge
Figure 6: Percentage change on the description rewritten by Llama2-70b and GPT-4.
Insights for Tool Developer. We also investigated the relationship between tool descriptions and CSR. We calculated CSR for the queries corresponding
Original tool Pit Line
and visualized them in Figure 7. There are two categories of tools: those that have been decomposed and merged (i.e., new tools) and those that have not been merged or decomposed (i.e., original tools). From the figure, we can draw the conclusion: The more detailed the description, the more ef- ficient tool selection. As shown by the fitted line, as the length of the description increases, the CSR continuously increases, indicat- ing that detailed descriptions can help LLMs better understand the functionality of tools, thus improving the accuracy of tool selection. Additionally, as shown in Figure 6, built upon the original descrip- tion by having two proficient LLMs rewrite it and then observed the performance changes eight LLMs on the new descriptions. Differ- ent rewritten LLMs yielded varying benefits for different groups. For instance, descriptions rewritten by Llama2-70b resulted a 7.83% improvement for llama2-13b but did not significantly enhance the performance of the Vicuna series models. In contrast, descriptions rewritten by GPT-4 caused sharp decline in the performance of ChatGLM and Llama2 series, while significantly boosting the Vicuna series, possibly due to the Vicuna series' training corpus being largely sourced from ShareGPT (ShareGPT, 2023). Therefore, we strongly recommend that tool developers choose an appropriate rewrite model for generating new descriptions based on the downstream LLM the tool will apply to.
0.6
0.4
0.2
40
Description Length
The CSR of tool se- lection and description length.
Figure 7:
CONCLUSION
In this paper, we introduce METATOOL. a benchmark for evaluating LLMs based on their tool usage awareness and tool selection capabilities. We propose TooLE within the benchmark, which contains diverse queries to trigger LLMs to use tools. We found that most LLMs lack good tool usage awareness and exhibit significant gap from real intelligent agents in tool selection.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENT
Lichao Sun and Yue Huang are supported by the National Science Foundation Grants CRI1-2246067 and Microsoft Accelerate Foundation Models Research Award.
REFERENCES
Nomic AL Nomic ai, 2023. https://atlas.nmiic.tt
Abid Ali Awan. The 10 best chatgpt plugins for data science, 2023. https//www..daaccnp.com/bbbgg the- 10-best-chat-gae mongins-for-daaa
babyagi. Babyagi, 2023. https:/github.com wwt..//oooociccaaiaaaaaaaaaaaaaa
Baichuan. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309. /0305, 2023.
URL https://axxiv.or//bb//230.. 10305.
NATALY BIRCH and ANDRIAN VALEANU. 16 best ai tools for web designers, 2023. https:
Adrian Bridgwater. Auto-tech series octoml: Large language model (Ilm) automa- tion for developers, 2023. http://wwww.oottrrreeeeey./o///a Auto-tech-serie OctoML Loooo-LLrre-LLanugggeWWeee Model- LLM-MMooo-----MMMMM LLLMMaaaaattio---ooe for-developers.
www.--evellpr---eeee Network/
Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers, arXiv preprint arXiv:2305.177 /7/26, 2023.
Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S Yu, and Lichao Sun. A comprehen- sive survey of ui-generated content (aige): A history of generative ai from gan to chatgpt. arXiv preprint arXiv:2303. 04226, 2023.
Feilong Chen, Minglun Han, Haozhi Zhao, Qingyang Zhang, Jing Shi, Shuang Xu, and Bo Xu. X-IIm: Bootstrapping advanced large language models by treating multi-modalities as foreign languages. ArXiv, abs/2305.04 1660 2023. URL
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* * chatgpt quality, March 2023. URL hitps: //mmsys.rg/bblog220333-3 00-003--30uu0 30-vicuna/.
Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprint arXiv:2205.09712, 2022.
Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongxiang Sun, Xiao Zhang, and Jun Xu. Uncovering chatgpt's capabilities in recommender systems. arXiv preprins arXiv:2305.02 2023.
Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang. Huan Sun, and Yu Su. Mind2web: Towards generalist agent for the web. arXiv preprint arXiv:2306.0607 06070, 2023.
Empresaria. House manager job description, 2023. http://www.geyyottuullee..c.kk
looking-for-al for-a-job/carerraadiic//// adwice/house--eeeeaaeeeeeeaaeaea jonaass--naaaa-----aaaa -oddescriptionnn.
Emilio Ferrara. Should chatgpt be biased? challenges and risks of bias in large language models.
arXiv preprint 2rX11:2304.03733. 2023.
Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. Chat- rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint arXiv:2303.14524 2023.
geekan. Metagpt, 2023. http://giinbb.oomgeeeaa//eeedrr
10

---

Published as a conference paper at ICLR 2024
Xinyang Geng, Arna Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song. Koala: A dialogue model for academic research. Blog post, April 2023. URL
https://bair.brrkke bwwpb//www....culo//////////0/00
Riley Goodside. The past and future of prompt engineering, 2023. https//exccangg..elee.oo///aa//a
videos/the-past-add past--ad---------- of (ooompt-nniineerngg2222-22------
GPTStore.
Overview of ai/chatgpt plugin weather, 2023.
https///ppsttor..iilllgggs
weatter--iienntscood--rep---
Level Up Coding Youssef Hosni. 9 helpful chatgpt plugins for data scientists, 2023. https://levelup. htwp://eddd..ooo/eeeeeeei-hheeee plppptt--pui------e for- scientists-32eceebddeee
Cheng-Yo Yu Hsich, Si-An Chen, Chun-Liang Li, Yasuhisa Fujüi, Alexander Ratner, Chen-Yi Yu Lee, Ranjay Krishna, and Tomas Plister. Tool documentation enables zero-shot tool-usage with large language models. arXiv preprint vrXiv:2308.00677 2023.
Chenxu Hu, Jie Fu, Chenzhuang Du, Simian Luo, Junbo Zhao, and Hang Zhao. Chatdb: Augmenting Ilms with databases as their symbolic memory. arXiv preprint arXiv:2306.0390) 2023.
Ziwei Ji, Naycon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
Vartika Kashyap. 26 best designer tools for web and graphic design professionals, 2023. https: hww.:ppoofhubb.cm//rrrieeeeeeeep
Enkelejda Kasneci, Kathrin SeBler, Stefan Küccheman, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, al. Chatgpt for good? opportunities and challenges of large language models for education. Learning and individual differences, 103:102274 2023.
Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. Internet- augmented language models through few-shot prompting for open-domain question answering. arXiv preprint / 15, 2022.
Cheng Li, Jindong Wang, Kaijie Zhu, Yixuan Zhang, Wenxin Hou, Jianxun Lian, and Xing Xie. Emotionprompt: Leveraging psychology for large language models enhancement via emotional stimulus. arXiv preprint arXiv:2307. 1760, 2023a.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tris- tan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training large language-and- vision assistant for biomedicine in one day. ArXiv, abs/2306.00890, 2023b. URL hitps: //api.semanticccholaarrrog// wwt.:/arrrrrrrcooooo0229999999
Jinyang Li, Binyuan Hui, Ge Qu, Binhua Li, Jiaxi Yang, Bowen Li, Bailin Wang, Bowen Qin, Rongyu Cao. Ruiying Geng, et al. Can Ilm already serve as database interface? a big bench for large-scale database grounded text-to-sqls. arXiv preprint arXiv:2305.03// 2023c.
Minghao Li, Feifan Song, Bowen Yu, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. Api-bank: A benchmark for tool-augmented llms. arXiv preprint arXiv:2304.0822 2023d.
Yuan Li, Yixuan Zhang, and Lichao Sun. Metaagents: Simulating interactions of human behaviors for llm-based task-oriented coordination via collaborative generative agents, 2023e.
Gemingtian Liu, Xinyu Ma, Yu Zhang, Boyan Su, and Pinan Liu. Gpt4: The indispensable helper for neurosurgeons in the new era. Annals of Biomedical Engineering, 51:21 15, 2023. URL http::/ppp...mccccccc/oooo/////o
2558887338.
Chenyang Lyu, Minghao Wu, Longyue Wang, Xinting Huang, Bingshuai Liu, Zefeng Du, Shuming Shi, and Zhaopeng Tu. Macaw-Ilm: Multi-modal language modeling with image, audio, video, and text integration. ArXiv, 0bb/2306609099 2023. URL 359165461.

---

Published as a conference paper at ICLR 2024
Teresa Mears. 10 essential tech tools for older adults, 2015. http://moongyuunnee....oommonnny retiremen//ariclee/0111.3 1/11/1111/1/////ee/iiii teeeentill-tehhhtee tech-toohh- for o-l-oolderrddd adults.
Ofer Mendelevitch. Large language models for code generation - part 2, 2023. https://vectara.com/
large-language-models mww.es--oool-------e-e----- for--ood--eneraation--o-ee generation-part-aa
Manuel Odendahl. Lims will fundamentally change software engineering, 2023. https://dew.to/wesen//
OpenAl. Introducing chatgpt, 2023a. URL httts://ppemai.ccm//bbogchhhggu
OpenAl. Gpt-4 technical report. 2023b. URL http:://arrivooggg//////330211//
OpenAL.
nevyaandd-mppoeeeeeeeelldinnnnmd 2023c.
http://openai.coo/bblgg/
aawraaaaiippove-eeeeeeeeee model. model.
OpenAl. Openai plugin, 2023d.
pluggig
Joon Sung Park, Joseph C O'Brien Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442, 2023.
Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.1533: 2023.
Zhi Qi, Yi Yu, Meiqi Tu, Junyi Tan, and Yongfeng Huang. Foodgpt: A large language model in food testing domain with incremental pre-training and knowledge graph prompt. ArXiv, a0s/2308.10173, 2023. URL
Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arXiv preprint arXiv:2307.07924, 2023.
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang. Chaojun Xiao, Chi Han, et al. Tool learning with foundation models. arXiv preprins arXiv:23004008335 2023a.
Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16788 2023b.
Salvatore Raieli. Fingpt: open-source Ilm for finance, 2023. fingpt-open-source source-llm llm-llffooo for-for-ffie co--fannne-eee10d00ne 1000ff40.
Rapid. Rapid api, 2023. https//rapidapi.com//
Peter J Rousseeuw. Silhouettes: graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53-65, 1987.
Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Xingyu Zeng, and Rui Zhao. Tptu: Task planning and tool usage of large language madel-based ai agents. arXiv preprint arXiv:2308.08 034277 2023.
Timo Schick, Jane Dwivedi- Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2.302.04730 2023.
National Science and Technology Council.
Emerging technologies to support an ag- wtw.://www....ccooeoooooooooeeee
ing population, 2019. https:/trummpwhiteheee wwwh-----SSppoot---aeeee-e---- Support-Aging-2019.o
Emerging-
ShareGPT. Sharegpt, 2023. https://shareppt.com//
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint 23X1 2023.
12

---

Published as a conference paper at ICLR 2024
Significant Significant-Gravitat Autogpt, 2023. https//gthhbbcoommigiifftttttttt Gravitass/ttoo--oooooooooo GPT.
Peter HA Sneath, Robert R Sokal, et al. Numerical laxonomy. The principles and practice of numerical classification. 1973.
Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao. Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang. Zhikun Zhang, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao Liu, Heng Ji, Hongyi Wang. Huan Zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang. Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang, John Mitchell, Kai Shu, Kaidi Xu, Kai- Chang, Lifang He, Lifu Huang, Michael Backes, Neil Zhenqiang Gong, Philip S. Yu, Pin- Yu Chen, Quanquan Gu, Ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tianming Liu, Tianyi Zhou, William Wang. Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, and Yue Zhao. Trustlim: Trustworthiness in large language models, 2024.
Weiwei Sun, Lingyong Yan, Xinyu Ma, Pengjie Ren, Dawei Yin, and Zhaochun Ren. Is chat- gpt good at search? investigating large language models as re-ranking agent. arXiv preprint arXiv:2304.005442 2023.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question
answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume (Long and Short Papers). pp. 4149-4155 Minneapolis, Minnesota, June 2019, Association for Computational Linguistics. doi: 10.19653/ URL https: Tacclanthlooyy.rr//N9999
Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, and Le Sun. Toolalpaca: General- ized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv:2306.050: 2023.
ODSC Team. nancial large
Have you met fingpt?
new open-source fi-
have-you- language
model,
2023.
httpss//opendataxcinnee.ccm
THUDM. Chatglm2, 2023. http://gttbbbcc//HHHDD//CaaaLLL
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Ilama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008
Nicolas Vidal. How ai and 1mms are streamlining financial services, 2023. https:/wwww.forbes.cou com/ witess/rreeseeccccoonc////////// ai--nd----n---------- -aa-llll-----eramlinneee fnnanial-seervice///sh- 2ea8b923017a.
Leandro von Werra and Loubna Ben Allal. Starcoder: A state-of-the-art Ilm for code, 2023. https:
Guangyu Wang, Guoxing Yang. Zongxin Du, Longjun Fan, and Xiaohu Li. Clinicalgpt: Large language models finetuned with diverse medical data and comprehensive evaluation. ArXiv, abs/2306.09968, 2023a. URL
Hao Wang, Chi-Liang Liu, Nuwa Xi, Zewen Qiang. Sendong Zhao, Bing Qin, and Ting Liu. Huatuo: Tuning Ilama model with chinese medical knowledge. ArXiv, abs/2304.06975. 2023b. URL
Haochun Wang, Chi Liu, Nuwa Xi. Zewen Qiang, Sendong Zhao, Bing Qin, and Ting Liu. Huatuo: Tuning llama model with chinese medical knowledge. arXiv preprint vrXiv:2304.0697. 2023c.
13

---

Published as a conference paper at ICLR 2024
Wen Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu, Jie Zhou, Y. Qiao, and Jifeng Dai. Visionllm: Large language model is also an open-ended decoder for vision-centric tasks. ArXiv, abs/2305 2023d. URL http:/pp...ematttccccoaaa.rorroo 258762579.
2023d.
Xiaolei Wang, Xinyu Tang, Wayne Xin Zhao, Jingyuan Wang, and Ji-Rong Wen. Rethinking the evaluation for conversational recommendation in the era large language models. arXiv preprint arXiv:2305..3112, 2023e.
Yizhong Wang, Yeganch Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannanch Hajishirzi. Self-instruct: Aligning language model with self generated instructions, 2022.
Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. Simple synthetic data reduces sycophancy in large language models. arXiv preprint arXiv:2308.0394 2023.
Matt Welsh. The future of software development with Ilms is here: Announcing
fixie's developer preview and 17m in seed funding, 2023. https://blog.fixx fixie..iiiiiiiiii
lww------eer--anouueeeeeeeo----- wwwpiinnng-ixii---eeee dereedeeeeooo-rrvveeeee 17m-cf6fca0c4041.
Stephen Wolfram Writings, Chatgpt gets its "wolfram superpowers"!, 2023, https://writings. wtl-iiss-oooor------rooeeee
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhan- jan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint 2 2023a.
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhan- jan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint 2303::2403./754 2023b.
Daniel Xiao. Ai-based literature review tools, 2023.
Qiantong Xu, Fenglu Hong, Bo Li, Changran Hu, Zhengyu Chen, and Jian Zhang. On the tool manipulation capability of open-source large language models. arXiv preprint arXiv:2305.16504 2023.
Zhiling Yan, Kai Zhang, Rong Zhou, Lifang He, Xiang Li, and Lichao Sun. Multimodal chatgpt for medical applications: an experimental study of gpt-4v. arXiv preprint arXiv:2310.1906) 2023.
Hongyang Yang, Xiao-Yan Liu, and Chris Wang. Fingpt: Open-source financial large language mod- wtwpoo////ooossD/591199999
els. ArXiv, abs/2306. 2023a. URL
Hongyang Yang, Xiao Yang Liu, and Christina Dan Wang. Fingpt: Open-source financial large
language models. arXiv preprint arXiv:2306.0603/ 2023b..
Rui Yang. Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. Gpt4tools: Teaching large language model to use tools via self-instruction. arXiv preprint arXiv:2305.1877 2023c.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022.
Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, and Songfang Huang. How well do large language models perform arithmetic tasks? arXiv preprint arXiv:2304.02015, 2023a. URL https//aariioorrabb2234000022.20
Zhengqing Yuan, Zhaoxu Li, and Lichao Sun. Tinygpt-v: Efficient multimodal large language model via small backbones. arXiv preprint #rXiv:2312./6822 2023b.
Jennifer Zhang. Unleashing the potential of 1lms: a new era for financial services, 2023. https: //www www. wwwwwwiwwi/ullsshiiigttteeeeeen
llms----nww-eee financial-servicss/.
14

---

Published as a conference paper at ICLR 2024
Kai Zhang, Jun Yu, Zhiling Yan, Yixin Liu, Eashan Adhikarla, Sunyang Fu, Xun Chen, Chen Chen, Yuyin Zhou, Xiang Li, et al. Biomedgpt: unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks. arXiv preprint arXiv:2305.17700 2023.
Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, et al. A comprehensive survey on pretrained foundation models: A history from bert to chatgpt. arXiv preprint arXiv:2302.09419 2023a.
Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. arXiv preprint arXiv:230.//226, 2023b,
Yuchen Zhuang, Yue Yu, Kuan Wang. Haotian Sun, and Chao Zhang. Toolqa: A dataset for Ilm question answering with external tools, 2023.
15

---

Published as a conference paper at ICLR 2024
Appendix
Table of Contents
A
TOOLE Dataset Details
16
A.1
Details Overlapped Issue
16
A.2
Efficiency of the Operation
A.3
Guidelines human validation
A.4
Flexibility TooLE
A.5
Others Statistics of TOoLE
Set for Evaluation on Awareness Tool Usage
................................
C Experimental Settings
23
C.I Models and Test Samples
23
C.2 Answer Matching
23
C.3 Task Comparison
23
C.4 Few-Shot Prompt
24
C.5 Human Evaluation
25
Prompt Template
25
D.1 TooLE Dataset Generation
25
D.2 Prompt Template Experiments
E Failure Case Study
29
A TOOLE DATASET DETAILS
In this section, we show the details of TOOLE, including how we solve the overlapped issue (Section A. guidelines for human validation (Section A.3), and the statistics of TOOLE (Section A.5).
A.1 DETAILS OVERLAPPED ISSUE
Operation pipeline.
Weather:
Tool mergence
(I) Embeddings and Hierarchical Clustering. We initiated by generating embeddings for tool
you
Weather Forecast: WeatherWhiz: Accurate curment weather foreaast for location.
deseriptions using the text-embedding-adee -d2-002 model (OpenAI, 2023c), API provided by
OpenAl, aiming to perform hierarchical clus- tering (Sneath al., 1973) different tools, based on the similarity of their embeddings, to
PDF COF&URLTod interact with any POF files provido page references faclechecking,
Summorizo www.torrooonntiiiin. Summarize You vidwos, and POF documents
reveal underlying patterns among them. (2) Tool Merging and Decomposition. Based on the re- sults of clustering, we manually merged and decomposed the data. Specifically, several pop- ular topics (e.g., news, weather) were identified based on their functions if they overlapped func- tionality with other tools (an example is shown in Figure 8). The criteria for merging and decomposition revolved around whether such tools commonly encountered and make practical sense in daily life. For instance, it is logical to merge tool that offers both flight and train ticket bookings with another tool that solely focuses hotel
Somemummmrria iae Tool Enable summaries from YouTube Tube video Inks,
decomposition
Figure 8: Two examples of tool mergence and decomposition.
16

---

Published as a conference paper at ICLR 2024
15.
20
10.
10-
Diransion
Cluster Label
15
15
0-
-10-
10
10
-20
Dimension
Dimension
(a)
(b)
Figure 9: t-SNE (Van der Maaten & Hinton, 2008) visualization of original tool description embed- ding (a) and new tool description embedding (b).
0.045
New
Original
0,040
0.035
Score
0.025
0.020
0.015
10
20
30
40
50
Cluster Number
Figure 10: Silhouette score (Rousseeuw, 1987) new tool description embedding and original tool description embedding in different cluster numbers.
reservations. For merged tools, we only needed to modify the original labels; for decomposed tools, manually assigned original queries to the appropriate decomposed tools and changed their labels accordingly. We manually created the names of new tools and employed ChatGPT to generate descriptions for both merged and decomposed tools, based on the initial descriptions of these tools. (3) Similarity Verification and Human Validation. We iterated each tool and searched for the ten most similar tools by its description embedding, then we checked whether the tool could be further merged or decomposed with the tools in Loim-
A.2 EFFICIENCY OF THE OPERATION
To evaluate the effectiveness of our operations in solving overlapped issues, we use the silhouette coefficient (Rousseeuw, 1987) to measure the degree of functional overlap between tools. Based on it, we compare the changes in the silhouette coefficients before and after the operations. Specifically, we aim to significantly reduce the functional overlap between tools after merging and decomposition to achieve more uniform distribution of tool functionalities in the embedding space. We embed the tool description before (390 tools) and after the operations (195 tools) and compared the changes in the silhouette coefficients under the same number of clusters. A greater silhouette coefficient indicates better clustering performance and higher functional overlap between tools, while smaller coefficient suggests lower overlap, reflecting the effectiveness of the operations.
17

---

Published as a conference paper at ICLR 2024
In Figure 10, present the variation of silhouette coefficients with changes in the number of clusters. can be observed that the silhouette coefficients of tools after the operations are significantly smaller than before. indicating that the operations have made the distribution of tools more uniform and effectively reduced tool functionality overlap. Additionally, in Figure 9, we visualize the results using t-SNE (Van der Maaten & Hinton, 2008) when the number of clusters is 30. It is evident that the distribution on the left side is more uniform compared to the right side.
A.3 GUIDELINES FOR HUMAN VALIDATION
We conducted rigorous manual evaluations to ensure the integrity and quality of TOOLE. We estab- lished the following rules to guide the manual evaluation:
Low-quality tool descriptions. In some cases, ChatGPT was unable to understand the purpose of tool due to low-quality or overly brief tool descriptions. We conducted a manual review these descriptions and eliminated tools with unclear or low-quality explanations.
High repetition queries. Since we generated multiple queries for single tool in one batch, some batches had issues with high query repetition. To address this problem, we selected one query and removed the others.
Queries contain tool name. The inclusion of tool's name in query can significantly bias our evaluation as an obvious hint for all tasks in METATOOL. Therefore, we removed queries containing the tool's is name. For example, 'How can calculate my MBTI type through [tool name]?'
Calculation-related tools. TOOLE contained numerous tools related to calculations. For simple calculations (c.g.. 'What is the value of sin 30 degrees?" or 9 ?"), LLMs can perform them without the need for tool. However, for complex calculations, recent research (Yuan et al., 2023a) suggests that LLMs still perform poorly. For queries corresponding to calculation-relate tools, we removed queries involving simple calculations and retained those involving complex calculations.
Tool retrieval-related tools. We found that
tools designed for users to retrieve other tools. This kind of tool conflicted with our task, so we removed these tools.
Al comprehensive tools. We identified some AI comprehensive tools that encompassed a wide range of Al-related tools, making them impractical for evaluation. Therefore, removed these tools.
Mentions of "ChatGPT" in queries. Some queries included the term 'ChatGPT;' for example, 'Hi, ChatGPT! ...' We uniformly replaced "ChatGPT" with "Chatbot'."
A.4 FLEXIBILITY OF TOOLE
Sometimes, when LLM undergoes special- ized training in a particular domain (Wu al., 2023b; Wang et al., 2023a; Qi et al., 2023; Wang et al., 2023b; Yang et al., 2023a), its capabili- ties improve significantly, and in some cases, some L.MS are also capable of handling vari- ous types of information (e.g., images or audio) (Liu et al., 2023; Chen et al., 2023; Wang et al., 2023d; Lyu et al., 2023; Li al., 2023b). These improvements render some externl tools that may not be necessary for some LLMs in the fu- ture. Therefore, we have annotated the reasons why L.MS need use these tools solve user problems.
Tool: ChariToo!
Description:
display
diagrams using networkor matpiatila allows you build, modily, and drew verious charts graphs within the interface.
8 Hendling spacializad inputa/buiouts..
Enhancing domain LLM capabiiliis
D User oustomizatio on, personalization and Interaction.
Specifically, we use four kinds motivation for tool usage (some examples are shown in Table 6): A. Solving issues with real-time or external data, databases, APIs. B. Handling specialized inputs/outputs. C. Enhancing do- main tasks beyond LLM's capabilities. D. User
Submit
Skip
Figure 11: The motivation labeling interface.
18

---

Published as a conference paper at ICLR 2024
Table 6: Possible reasons for the motivation of tool usage.
Tool
Description
Reason
airqualityforeast
Planning something outdoors? Get the 2-day air quality forecast for your city.
A
Now
Get Google Trends. In Japan, you can also get Twitter trends and search Twitter keywords.
A
abc_ tbc__t__ut to_ao
Converts ABC music notation to WAV, MIDI, and PostScript files. The best way read text from from any document. CharOCR will scan and read aloud any text document you provide.
ChatOCR
B
Tool
Begin an exciting journey through time, interact with historical events, and Iearn about the past in fun and engaging way.
C
LawTool
Enables quick search functionality for relevant laws.
C
TicTac'Toe
Playing a game of Tic Tac Toe with varying board sizes. You can submit your move and get the Al's response move.
D
Planfit
Get your tailored workout plan and instructions with videos Al- powered Workout Coach, Planfit.
D
customization, personalization, and interaction. We have enlisted the expertise of two experts to annotate the usage motivations of tools in TOOLE (the annotation interface shown in Figure 11).
A.5 OTHERS STATISTICS OF TOOLE
Data quantity. Due the constraints of API us- age costs, we exclusively employ GPT-4 for direct diverse generation, utilizing ChatGPT for all other forms of generation techniques. This strategy yielded total of 29,000 dataset entries. Following metic- ulous human review process, we ultimately curated the TOoLE dataset, culminating in collection com- prising 20,881 entries. A comprehensive overview of dataset statistics can be found in Table 11.
Deraily
0.00
Length distribution. Figure 12 displays the distri- bution of dataset lengths. can be observed that the majority of the data falls within 40 words less, aligning with the typical question lengths in people's daily lives.
20
Length
80 100
Figure 12: Density distribution of all queries' length.
Visualization and relcase. We used Nomic AI (AI, 2023) to embed user queries in the ToOLE dataset, then clustered the embeddings, and finally visualized the results. The visualization is shown in Figure 13, and you can view it through the following link: http://allaa..oo.aaaaaaa a4 443668444453-428. 2012228--778---1 7738-553447600689/22222
Dataset comparison. As shown in Table 7, we compare the other datasets with TooLE. Compared other datasets, believe that ToolE has main advantages: (1) Our dataset exhibits greater diversity, and this diversity is tailored to real user scenarios, such as variations in expression style, mood, and level of detail. We employ various prompt methods to induce LLMs generate a more diverse range of user inputs, ensuring that TOOLE covers broad spectrum of inputs resembling those of actual users. (2) By employing pipeline process to address overlapped issues, we can ensure the rigor of the data. As outlined in Appendix A. employ multiple steps to address overlapped issues, ensuring that there is no overlap between tools, which is crucial for maintaining the quality of the dataset.
Tool description. These original tool descriptions encompass two distinct categories: machine- readable descriptions and user-facing descriptions. The machine-readable descriptions prioritize considerations such as token context length or keyword incorporation, aiming to enhance tool prompting within 8, 8,000-character limit. Conversely, the user-facing descriptions offer succinet
19

---

Published as a conference paper at ICLR 2024
Table 7: Comparison of previous work and METATOOL.
APllbank
ToolLLM
ToolAlpaca
GPT4Tool Yang et al. (2023c)
ToalQ
ToolE
Dimension
Li al. (2023d)
Qin et al. (2023b)
Tang et al. (2023)
Zhuang et al. (2023) (Ours)
Multi-Tuul
x
x
X
Real Secnario
x
V
Overlapped Issue Solved
X
x
x
X
X
Diversity Generation
x
X
V
Table 8: Data examples in TooLE.
Example
Type
I can't 't seem to remember anything I study. 1 need help with learning and retaining information effectively.
Emotion-and angry
I'm feeling really down today, you summarize this YouTube video for me? [Link YouTube video]
Einotion-dpprese
Good day! I'm hoping find some amazing bargains and discounts today. Can you guide me in the right direction?
Emotion- Emotion-happy
Hey Chatbot, I'm looking for new pair of sneakers on GoFynd. Can you help me find the latest designs and recommend some popular brands?
Emotion-exeited
I'm interested in Japanese cosmetics. Can you recommend some good hrands? Draw a state diagram for vending machine.
Direct-request
Direct-order
Twitter Trends: What are the trending topics on Twitter in Japan?
Keyword
Could you recommend some popular shopping malls or markets in Singapore that offer a wide range of products, including local and international brands,
Details
diverse food options, and unique shopping experiences?
Table Top 15 tools ranked by the number of merged tools.
Tool Name
Merged Tools
Tool Name
Merged Tools
Tool Name
Merged Tools
FinanceTool
22
ResearchFinder
7
TripAdviceTool
ProductSearch
19
NewsTool
7
WeatherTool
JobTool
12
RepoTool
6
HousePurchasingToor
5
TripTool
10
ResearchHelper
6
Discountt
5
PDF&
8
CourseTool
6
MusicTool
5
20

---

Published as a conference paper at ICLR 2024
Table 10: The tool lists of different scenarios.
Scenario
Tools
Ref.
Software
Engi- Rool Al2sql, SSH, AutoInfral, noteable, dart, hackit web scanner, LarkBarelmporter, webhooks, uni- versal
(von Werra Allal, 2023;
neer
Odendahl, 2023; Welsh,
2023; Bridgwater, 2023;
Mendelevitch, 2023)
Elders
NewsTool, PolishTool, CharityTogg Tool, MapTool, Memo-
(Science & Council, 2019; Mears, 2015)
ryTool, WoatherTool, RestaurantBookingTTon DietTool, NotesTool, TripAdvice
Finance Staff
FinanceTool, ChartTool, PDF&URLTool, NotesTool, Ex- changeTTool, CreditYelp, LawTool, DataRetrievalTool, fundsdbsearch, CompanyInfoTool
(Raicli, 2023; Zhang, 2023; Vidal, 2023; Yang et al., 2023b; Team, 2023) (Empresaria, 2023)
House manager
tira, Discount, ProductSearch, ABCmouse, Restaurant- Booking Tool, IndoorPlants, recipe retrieval. HouseR- enting Tool, TripTool, CreditYelp
Students
CourseTool,, ResearchFinder, ResearchHelper, speak. (Xiao, 2023; Awan, 2023; noteable, search, MemoryTool, NotesTool, Mixer- Hosni, 2023; Writings, Box Box_Tranc Lartollttt__laaaguaaeeeee tutor, ABCmouse
2023)
Artists&designess placid, find agency, ArCCollection, ChartTool, story- (Kashyap, 2023; BIRCH bird stories, MeddiaModiyyToo) PolishTool, MusicTTool, VALEANU, 2023) ImageSearch, BookTool
Table 11: Dataset Statistics of TOOLE.
Generation method
Model
Sample number
Direct generation
ChatGPT, GPT-4
11,700
Emotional generation
ChatGPT
7,800
Keyword generation
ChatGPT
1,950
Details generation
ChatGPT
7,800
Multi-tool generation
ChatGPT, GPT-4
1624
After checking - 21,127 (20630 single-tool + 497 multi-tool)
and simplified explanations of each tool's functionality. While the majority of tools share identical descriptions across both categories, we opt to adopt the user-facing descriptions as the definitive tool descriptions. This choice is informed by the tendency of machine-readable descriptions be overly verbose, often delving into instructing the language models on how to handle tool input and output- aspects that are not pertinent
benchmark.
Generation times. For each type of original tool, we perform two rounds of direct diverse generation, producing ten queries each time. In the case of emotional generation, we generate five samples for each of the four distinct emotions. For keyword generation, we extract five keywords from the tool's description and subsequently formulate query for each identified keyword. Concerning details generation, the number of samples generated aligns with that of the direct diverse generation.
B TEST SET FOR EVALUATION ON THE AWARENESS OF TOOL USAGE
To assess to what extent LLMs are aware of their limitations, we construct the dataset for the awareness of tool usage by merging the positive samples from subset of TOoLE dataset and the negative samples from some subsets of public datasets. To exclude the ambiguous situation where the queries in TOoLE can be solved either with or without the assistance of external tools, we manually check the output of the queries by feeding them directly into the LLMs and retain those whose responses are unsatisfactory(i.c the model apologizes, the response contains errors, and etc). this way, verify that solving the queries in selected subset(containing 515 samples) is beyond
21

---

Published as a conference paper at ICLR 2024
Teaio Metors
Exchandee
ostologg
Persona Data Breach
Einancial Performanc
News
Expoite
Astiologca pedictioos financial o7
Weekend asthma forecast
Compestn
Landrcape
generator
optimization
Connertinn Images Intortex!
warkyww
Domain availability
Games
playlist
Cooking inn for your father
Dea errshp Ads
coupons
Location
Uaptops
Apartment Rentals
Flight Singapore Animal
Shoee
Figure 13: TOOLE embedding visualization.
the capacity of the existing LLMss and the queries can therefore be treated as positive samples when evaluating the awareness of tool usage. Mcanwhile, we collect negative samples of equal size from the public datasets of daily conversations (Wang et al., 2022; Zhou et al., 2023b) and common sense (Talmor al., 2019), the queries of which are expected to be answered by LLMs without the help of external tools.
In order to gain clearer understanding of the Tool Usage Awareness dataset, will deseribe how we selected positive and negative samples for the dataset. Firstly, we categorized user queries into three types:
The first type is "Queries must be solved by tools" (positive), such multimodal input and real- timmeme information retrieval.
The second type is "Queries can be solved well by all LLMs" (negative), such as telling jokes, basic conversational functions, sentiment classification, and other basic NL.P tasks.
The third type represents the middle ground between the first and second types of user queries. i.c., queries we hope LLMs can solve but currently cannot, such as complex calculations, long text summarization, and information extraction.
With the aforementioned types of user queries, we classified user queries into positive or negative samples using human evaluation and model checking.
(1) Human evaluation: For the first and second types, we determined the samples through unanimous agreement from two human experts and referenced the four reasons in Appendix A.4 for selection.
22

---

Published as a conference paper at ICLR 2024
(2) Model Checking: Regarding the third type of user queries, objective is to find those within this category that can be solved well by all LLMs (classified as negative) and those that none of the LLMs can solve (classified positive). We discarded user queries that only portion of the LLMs can solve them. We conduct validation in the following two steps:
We initially input the queries into GPT4, Since GPT-4 currently has the best performance in terms of utility, if GPT-4 declines to answer (i.e., unable to solve the problem or refuse to answer), we classify it as positive query.
Following the above operation, if the query is not classified as positive, we conducted inference on eight LLMs and then evaluated the answers through human experts. If all output from L.LSs solves the problem well, we classify them as negative queries.
C EXPERIMENTAL SETTINGS
C.I MODELS AND TEST SAMPLES
We conducted extensive experiments on eight commonly used LLMs including ChatGPT (OpenAI, 2023a). ChatGLM2 (6B) (THUDM, 2023). Llama2 (7b, 13b) (Touvron et al., 2023), Vicuna (7b, 13b, 33b) (Chiang et al., 2023), Baichuan2 (13b) (Baichuan, 2023) and Koala (13b) (Geng et al., 2023). The temperature parameter for these models was uniformly set to 0.0 in our experiments. For tool selection with similar choices and with possible reliability issues, we sampled five samples for each tool, resulting in total of 975 samples. For tool selection in specific scenarios, we sampled 20 samples for each tool within each scenario. This means that for aepularity-relatter scenarios, namely the top 5, 10, and 15 scenarios, we obtained 100, 200, and 300 samples, respectively. As for group-related scenarios, each scenario yielded 200 samples. the case of" mulliiitool selection, given the relatively small dataset size, we utilized the entire set of samples for testing.
C.2 ANSWER MATCHING
Due to the model's s practice of providing an explanatory context for its responses rather than directly outputting answers, it is necessary to perform answer matching (obtaining "yes." "no," or the name of a tool) for a more accurate evaluation.
Tool usage awareness. We use the following rules to match the results:
If "yes" present in the sentence and "no," "not," " or "don't" is absent from the sentence: The answer is "yes."
"no" is present in the sentence and "yes" is absent from the sentence: The answer is "yes." phrases like "not seem necessary," "not think it is necessary," "not need to use," "not necessary to use," and "do not think need to use" are present in the sentence: The answer is "no."
necessary." " "I may need to use." " "I would need to use," "I believe it is necessary to use," "e "would need access," "might be necessary to use," "I might need use," " "tools would be necessary," " "may be necessary use," " "be beneficial to use," will need to use," "might need to use." "would need to rely," and "may need to access" are present in the sentence: The answer is "yes."
If phrases like "I need to use." " "I think
For cases not covered by the above rules, the answers are analyzed manually.
Tool selection. We match the names of tools from the results that contain tool list. For single-tool tasks, if no matches are found, it is recorded as "None." If one match is found, and "None" is not present the results, it is compared with the ground-truth label. If more than two matches are found, manual analysis is performed. For multi-tool tasks, if the number of matches is less than two, it is considered an incorrect answer. If the number of matches is equal to two, it is compared with the ground-truth label. If more than two matches are found, the answers are manually evaluated.
C.3 TASK COMPARISON
In order to have a more intuitive understanding of the different subtasks in tool selection, we show the comparison of the four sub-tasks in Table 12.
23

---

Published as a conference paper at ICLR 2024
Table 12: Comparison of four tasks in tool selection.
Task
Tool list
Ideal output
Tool selection with similar choices Tool selection specific scenarios Tool selection possible reliahility issues
Tool and its similar (n tools.
Specified tools certain scenario.
remaining set of tools, which excludes tool t and its most similar tools.
tools randomly
are
Multi-tool Selection
The tools randonly chosen from the remaining set of tools, which excludes the union set most similar tools of St, where St the ground-truth tool
S.
Table 13: CSR (%) results of zero-shot and five-shot in different scenarios. A is the percentage change of CSR between zore-shot and five-shot, as calculated by CSR CSR Abbreviation: Finance Staff (Finan.), House manager (Home.), Software Engineer (Soft.), Student (Stud.), and Artist & Designer (Artis.).
Scenario
Finan.
Home.
Soft.
Stud.
Artis.
Elders
Top
Top 10
Top 15
0
46.70
57.59
28.50
40.31
42.78
68.42
82.29
56.19
48.07
ChatGLM2 5
62.57
62.94
41.53
34.97
45.64
69.66
86.17
57.22
50.54
A
15.87
5,35
13.03
-5.34
2.86
1.24
3.88
1.03
2.47
= 0 69.70
82.23
76.70
66.67
80.00
83.76
88.89
77.39
75.08
ChatGPT
= 5 74.24
86.73
69.11
72.45
87,00
89,45
91,00
80.50
77.33
A
4.54
4.50
-7.59
5.78
7.00
5.69
2.11
3.11
2.25
I = 0
59.32
69.47
47.69
43.75
73.58
77.78
86.42
55.56
52.98
Koala-13b
5
51.83
61,85
46.99
53.44
64.38
82.35
81.71
69.82
52.59
A -7.49
-7.62
-0.70
9.69
-9.20
4.57
-4.71
14.26
-0.39
52.04
$7.07
30.00
32.99
59.09
57.14
65.66
45.73
44.86
Llama2-7b
5
64.97
62.12
42.35
40.00
69.00
76.17
67.68
47.24
54.08
A
12.93
5,05
12.35
7.01
9.91
19.03
2.02
1.51
9.22
0
35.18
47.00
34.85
35.00
30.00
41.54
42.00
43.72
38.93
Llama2-13b
5
55.50
65.33
48.73
47.47
56.78
67.86
74.00
51.76
61.09
A
20.32
18,33
13.88
12.47
26.78
26.32
32.00
8.04
22.16
= 0
50.54
67.35
45.45
30.77
64.58
74.53
74.23
69.73
63.20
Vicuna-7b
x 5
50.51
73.30
43.32
36.79
63.75
77.51
82.65
52.82
55.47
A
-0.03
5.95
-2.13
6.02
-0.83
2,98
8.42
-16.91
-7.73
0
69.19
83.92
49.24
57.07
79.50
81.03
87.76
68.50
65.77
Vicuna-13b
5
72.45
83.82
59.78
59.69
80.25
79.59
88.66
72.86
65.31
A
3.26
-0.10
10.54
2.62
0.75
-1.44
0.90
4.36
-0.46
0
79.90
84,92
69.63
64.50
86.08
94.42
92.00
73.23
70.90
Vicuna-33b
5
76.26
84.44
66.15
65.66
90.48
91.88
90.00
77.16
70.99
A -3.64
-0.48
-3,48
1.16
4,40
-2.54
-2.00
3.93
0.09
C.4 FEW-SHOT PROMPT
For Subtaskl, we randomly sampled five different tools and, for each tool, randomly selected corresponding query. The sampling method for Sub-task was the same as Sub-task 1. For Subtask3, to balance different types answers, we ensured that the ratio of answers being 'none" to answers being tool was either 2:3 or 3:2. Due to the task setup of Sub-task 4, we did not conduct few-shot experiments on Sub-task 4. This because we have ensured that the 15 tools most similar to the ground-truth tool were not present in the tool list, and the number of multi-tool queries limited, making it impossible to guarantee that each query could have five exemplars containing two-tool combinations.
24

---

Published as a conference paper at ICLR 2024
Table 14: Comparison between human and LLMs. ModelMAx the best performance of LLMs and Avd is the average performance of eight LLMs.
Sub-task
Similar
Scenario
Reliability
Multi-tool
Modelaxx
73.46
79.51
50.35
88.28
Model
56.90
61.51
8.59
55.
Human
86.00
91.00
96.00
66.00
C.5 HUMAN EVALUATION
To investigate human performance in tool selection, we evaluated human abilities through question- naires.
Specifically, we mixed questions from four sub-tasks, asking participants to select 0 to tools for each question. Each questionnaire comprised 10 15 questions, with participants making choices based on provided queries and candidate tools as options. We collected a total of 240 valid responses. The results of the human evaluation are presented in Table 14.
We observe a notable discrepancy between the CSR humans and LLMs in sub-task 1, sub-task 2, and sub-task 3. Human CSR surpasses both the average and maximum CSR of LLMs. Notably, in sub-task 3, human performance reaches an impressive 96%, a stark contrast to the model's 's meager 9%. This discrepancy highlights the challenges LLMs face, particularly in addressing issues like hallucination, significantly impacting their reliability.
Moreover, in sub-task 4, human performance, while surpassing the average level of LLMs, falls short of reaching their maximum CSR. This implies that, when confronted with intricate language tasks, such multiple-choice questions, LLMs still maintain distinct advantage.
D PROMPT TEMPLATE
D.1 TOOLE DATASET GENERATION
We show the prompt templates of TOOLE dataset generation follows:
Direct diverse generation
Here is a tool for ChatGPT, which can help it solve users' requests better. The description of this tool includes description of users and a description of ChatGPT.
The description of users: (human description)
The description of ChatGPT: (model description)
Please give examples where you would use this plugin answer a user's question and you should only tell me what users will say.
Please ensure that the provided examples are distinct from one another. Feel free to employ various sentence styles, such as instructions or requests, and vary the level of detail as needed.
The format of your answer should be like: 1. User: [Your
Answer]. 2. User: [Your Answer]
25

---

Published as a conference paper at ICLR 2024
Details diverse generation
Here is a plugin designed to enhance ChatGPT's responsiveness to users' needs. ChatGPT only uses the plugin when it thinks the tools will enhance its response. Now, would like you to complete the following tasks:
will provide you with a description of the plugin, and based on that description, you need to provide five examples of user inputs that would prompt ChatGPT to utilize the plugin in order to enhance its responses for users. Please ensure that your answers satisšy the following conditions:
1. Each example should be the first input. in a new conversation, without any prior context.
2. The sentence should contain description information.
3. Your answers should be as detailed possible.
4. Format your answers as follows: 1. User: [Answer], 2. User: [Answer],
5. Utilizing this plugin has the potential to significantly improve ChatGPT's s ability to address users' requests.
The plugin includes descriptions for both users and ChatGPT. The description for users is as follows: (human
description), and the description for ChatGPT is: (model description).
Keywords extraction for keywords generation
Here is a tool for ChatGPT, which can help it solve users' requests better.
The description of this tool includes a description of users and description of ChatGPT.
The description of users: (human description)
The description of ChatGPT: (model description).
Now your task is to give me five words or phrases to label this tool.
These labels can be not mentioned in the description and labels should be as short as possible.
The format of your answer should be 1. [label 1), 2. [label
2], .... [label 5].
Keywords generation
Here is tool for ChatGPT, which can help it solve users' requests better.
The description of this tool includes description of users and a description of ChatGPT.
The description of users: (human description)
The description of ChatGPT: (model description;
New will give you five labels of this tool and what you need to do is give me five sentences the user will input to ChatGPT when they may need the help of this tool.
Here are the labels: (labels)
The format of your answer should be: 1. [label 1]: [sentence 11, 2. [label 2]: [sentence 2], .... 5. [label
5]: [sentence 5].
26

---

We show the experimental prompt in this section including the Thought part (1) and the Action part Published as a conference paper at ICLR 2024
Emotional generation
Here is a plugin designed to enhance ChatGPT's responsiveness to users' needs. ChatGPT only uses the plugin when it thinks the tools will enhance its response. Now, would like you to complete the following tasks:
will provide you with a description of the plugin, and based on that description, you need to provide five examples of user inputs that would prompt ChatGPT to utilize the plugin in order to enhance its responses for users. Please ensure that your answers satisšy the following conditions:
1. Each example should be the first input. in a new conversation, without any prior context.
2. The sentence should contain description information.
3. The example should be in (Emetion type) mood.
4. Format your answers as follows: 1. User: [Answer], 2.
User: [Answer],
5. Utilizing this plugin has the potential to significantly improve ChatGPT's s ability to address users' requests.
The plugin includes descriptions for both users and ChatGPT. The description for users is as follows: (human
description), and the description for ChatGPT is: (model description).
Multi-tool queries generation
New you are query generation assistant. Here are two tools for ChatGPT, which can help it solve users' requests better. The first tool is (first tool name), its description is "(first tool description)". The second tool is (second tool name), its description is "(second tool description)". Please give five examples where you would these two tools AT THE SAME TIME to answer user's s query and you should only tell me what users will say.
Remember that the queries you give must be related to both two tools!
Please ensure that the provided examples are distinct from one another. Feel free to employ various sentence styles, such as instructions or requests, and vary the level of detail as needed.
Remember that the 5 queries you generate should include both parallel tool usage (two tools without any relationship) and causal tool usage (one tool dependent the result of the other tool). Also, the user's query cannot include the specific name of the tool.
The format of your answer should be like: 1. User: [Your Answer]. User: [Your Answer]
D.2 PROMPT TEMPLATE OF EXPERIMENTS
27

---

Published as a conference paper at ICLR 2024
Thought (Q) prompt
You are an intelligent agent, and you need constantly be aware of your own limitations. I will provide you with user's s query, and you should assess, based on your own capabilities, whether you need to use external tools to better address the user's query. Typically, there are four reasons why you might need to use external tools:
[Reasons Beqin]
(tool_reason] [Reasons End]
Here is the user's query:
[User Query Begins]
(user_query)
[User Query Ends] Based on the above query, if you think it's necessary to use external tools, please respond with "yes"; otherwise, respond with "no." Additionally, you should provide a brief explanation for your answer.
Action (2) prompt in the single-tool tasks
You are a helpful AI assistant. Your current task is to choose the appropriate tool solve the user's s query based their question. I will provide you with the user's question and information about the tools.
If there is tool in the list that is applicable to this query, please return the name of the tool (you can only choose tool). If there isn't, please return 'None.' Additionally, you will need to support your answer with brief explanation.
User's s Query:
[User's Query Start]
(user_query) [User's s Query End].
List of Tools with Names and Descriptions:
[List of Tools with Names and Descriptions Start] (tool_list)
[List of Tools with Names and Descriptions End]
28

---

Published as a conference paper at ICLR 2024
Table 15: Examples of tool usage awareness on ChatGPT.
ID
Query
Answer
I need remember to attend meeting a.m. Can you set reminder me?
No, it is not necessary
external tools for this query. an intelligent agent, have capability to set reminders for users and can do so without the need for external tools.
No. As an Al language model, have access to a vast amount of information on the internet and can gather information about services offered by Magnetis by searching relevant web pages and extracting information. Therefore, do not need to use external tools to address this query.
Can you tell me more about the services offered by Magnetis?
No. The query is straightforward and does require exter- nal tools. The agent can search for musie-related gift ideas using own search capabilitics.
My brother is music lover. Can you find gift related to music for him?
No. As an Al language model, I can perform basic mathe- matical calculations, including exponentiation. can directly calculate the result 3.01 raised the power of 3.82488 without the need for external tools.
What is the result of 3.01173.82488y
Yes, it is necessary to use external tools to find the four smallest perfect numbers.
Find four smallest perfect numbers.
Action (2) prompt in the multi-tool task
You are helpful AI assistant. Your current task is to choose the appropriate tool to solve the user's query based their question. I will provide you with the user's question and information about the tools.
there is tool in the list that is applicable to this query, please return the name of the tool (you can choose two tools at most). If there isn't, please return 'None.' Additionally, you will need to support your answer with brief explanation.
User's Query:
[User's s Query Start]
(user_quers
(User's s Query End].
List of Tools with Names and Descriptions:
[List of Tools with Names and Descriptions Start] (tool_list)
(List of Tools with Names and Descriptions End]
FAILURE CASE STUDY
We show some failure cases of tool selection motivation in LLMs, as shown in Table 15 and Table 16. The following conclusions can be drawn from these cases: (1) LLMs can be overconfident or lack confidence. On one hand, LLMs demonstrate overconfidence their capabilities, as exemplified by the first three instances in Table 15, where LLMs erroneously perceive themselves as having the capacity to offer reminders and conduct internet searches. On the other hand, LLMS display unquestioning reliance external tools, evidenced by three instances in Table 16. Even for conventional language model tasks like sentiment classification and text generation, they prefer tool- based solutions. (2) LLMss have confticting perceptions of their
capabilities. As demonstrated by the 4th, 5th, and 6th examples in Table 15, ChatGPT provides contradictory responses regarding the utilization of external tools when handling mathematical and logical queries. Based the aforementioned findings, it is imperative to conduct additional research and training to enhance LLMs' proficiency in correctly utilizing tools.
29

---

Published as a conference paper at ICLR 2024
Table 16: Examples of tool usage awareness on vicuma.
ID
Query
Answer
Detect the Reddit thread contains hate speech, r/worldnews
Yes. To detect if the Reddit thread contains hate speech, external tools such natural language processing (NLP) models or sentiment analysis algorithms may required. These tools can analyze the text and identify any offensive or hateful language used in the comments.
Posted by u/madneon hours ago
Russia 'borrows $13.60 1bn in largest ever debt in single day' continue faltering war macross 1984 ago Russia will become deadbeat.
Yes. The query involves proving mathematical statement, specifically about the countability of sets. language model, can provide a general explanation or approach to solving this problem, but I cannot perform real-time calcula- tions or generate formal proofs. To fully address the user's 's query, it would be beneficial to external tools such mathematical software (e.g., Mathematica, MATLAB) consult with a human expert in the field of mathematics..
Prove that every countable.
Yes, is necessary to external tools. The query requires the creation of fun math question for children, which in- volves generating specific type of content that may not fall within the capabilities the language model. External tools or resources may be needed to generate appropriate and engaging math questions that suitable for children.
3
Create fun math question for children.
30


Reviews:
[{"review_id": "gFQbQwxtTB", "reviewer": "Reviewer_MgrT", "summary": "Summary: The paper presents a dataset and a set of tasks that help evaluate Tool calling ability of LLMs. While the work differentiates the abilities into four tasks (A) whether to employ tools (b) Which tools to employ (c) handle the results from the tool (d) return the outcomes to the use, the paper tries to focus on (a) and (b) motivating that the existing works focus on (c) and (d). ToolE dataset with 21,127 user queries, with both single-tool and multi-tool queries. They compare multiple different LLMs on four different tasks based on the dataset and show that ChatGPT performs the best in comparison to others. \n\nConcerns: \n1. One of the things mentioned is that other works lack diverse user inputs whereas this work does have diverse user inputs. Experiments or evidence for this is missing in this paper?\n\n2.  Keyword Generation and Details Generation: What’s the exact difference between the two?\n\n3.  Multi-tool selection: We select top-15 popular tools — for each pair generate 5 queries\n            1. How do you select the popular tools? Manual? \n            2. What do you think of the quality of multi-tool queries?\n            3. Multi-tool queries seems to have been created with no semantics associated to the queries or there are no specific details on it. It Is important of have multi-tool queries where the tool combinations are commonly used rather than random tool combinations. Are there any human validations on the percentage of tool combinations that are useful in the dataset?\n            4. The results are also pointing in this direction where it seems to be very evident for ChatGPT, Llama2 and Vicena-13B that the queries are multi tool queries — Seems unnatural queries where the patterns are evident? \n\n 4. Awareness dataset: positive examples from ToolE dataset and negative examples from other datasets\n            1. Tt’s unclear how this dataset can be useful. The negative examples will be starkly different from that of the positive examples. Furthermore, it’s unclear if the negative examples truly does not require any tools. There might be setting specifically in commonsense QA where tool use can be useful such as entity extraction and relation extraction from the text that can inform answers. The aspect of tools is probably too domain specific to make such assumptions\n\n5. Tool selection with possible reliability issue: There are concerns how this experiment is setup or the motivation behind this experiment. If the tool is not available and a similar tool is available (Overlapping tool) and if the LLM is able to detect that then shouldn’t the LLM be given those points. This experiments seems non-realistic and would need more investigation on the reliability of LLMs (hallucinations)\n\n6. No Related work: How does this work compare to ToolLLM, toolAlpaca, API-Bank, etc etc. Can those datasets be transformed to Meta-Tool in which case how would that work and what are the drawbacks. The motivation of this work is not very concrete because of the existence of these datasets and no comparison to those. \n\n7. While at a meta-level these tasks make sense, the aspect of arguments for each of the tools and execution is not focused in this work. A good explanation for this would be useful and makes it easier for the reader to understand. Specifically because when discussing tools, the arguments and executions plays a very important role.", "questions": "In the summary", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper states an important problem for using Tools in LLMs\n2. The experiments are well done and proves or concludes some hypothesis in the paper regarding LLMs abilities with tools.", "weaknesses": "1. The paper needs to be self sustained -- There are aspects that are unclear in the paper\n2. Related work is very important given the number of papers in this domain\n3. The experimental setup for some of the tasks does seem unnatural", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ze60vPfHfb", "reviewer": "Reviewer_sQgt", "summary": "This works studies the ability of LLMs to (1) decide whether to use tools and (2) which tools to use. They introduce a benchmark with over 20,000 prompts that trigger single- or multi-tool usage. Unlike recent related benchmarks, they study the reliability of nine LLMs at tool selection between similar choices, across challenging and diverse scenarios, and when multiple tools are needed. To generate diverse scenario prompts, the authors use algorithms that apply prompting and tool merging & decomposition. Through multiple studies and error analyses, the authors characterize ways in which most LLMs are inconsistent and poorly calibrated at tool usage.\n\nThe authors emphasize tool usage as a concrete way to test hallucination (does the LM know when it needs help from a tool? does the LM know which tool to use?) and sycophancy (does the LM know when to defer to tools despite context in the prompt?). The work also makes interesting connections between recommendation (or retrieval) and tool usage from an evaluation standpoint.", "questions": "Questions and comments:\n\n1. What kind of dev vs. test guidance do you recommend for this benchmark? Is it meant to be a truly \"blind zero-shot, no tuning\" dataset? What did the authors do for tuning? Did they use the prompt unchanged for each LLM? (Having no dev set is completely reasonable in principle, but practical considerations tell us that much of the community will, in effect, reuse the test set so often it becomes a dev set. I recommend having at least a few examples explicitly for tuning.)\n\n2. Which Llama2 models were used, vanilla or chat? I presume it's chat. It may be good to include more explicit information in the main text. You find that \"the worst-performing model, Llama2-13b, has an F1 Score of only 11.53%\". This is surprisingly low. Is this some kind of unintended interaction between the chat format or some other minor detail and the task?\n\n3. Is the word \"task\" is overloaded in Sec 2.3, where it refers both to the two major tasks \"2.3.1\" and \"2.3.2\", and to the four sub-tasks of \"2\"? Maybe rename the four tool selection tests to \"sub-tasks\".", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This benchmark tackles an area with much hype, and offers a much-needed challenging \"agent\" benchmark.\n\n2. The authors thoughtfully construct MetaTool, following an extensive consideration of the desiderata (Sec 2). This work could be part of making \"LLM general-purpose agents\" a more systematic and empirical area. \n\n3. The construction of the dataset is relatively novel. The authors collect 390 tools from OpenAI plugins. They merge (or decompose) tools to reduce benchmark ambiguity.  To construct the queries for evaluation, they prompt OpenAI models in a set of four pipelines. They also generate multi-tool queries from pairs of tools. All queries in the dataset were checked by a human.\n\n4. The definition of the four tasks and the evaluation analyses conducted (3.2, 3.3) are quite rich.", "weaknesses": "1. Section 2 is rich in substance but the presentation is disorderly. There's probably at least 2 different sections that should be there, like \"Task Formulation & Desiderata\" first then \"Dataset Generation\" second, but these two considerations are mixed up repeatedly in the current presentation. I'm happy to change my mind on this if the authors would like to defend the current organization.\n   \n2. It's unclear (and seemingly undiscussed) how fundamentally the generation of the test queries with LLMs biases the evaluation for/against particular models, or reduce the true diversity of these queries.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yo9M6d2HVw", "reviewer": "Reviewer_hidW", "summary": "The METATOOL Benchmark paper introduces a new benchmark designed to evaluate the tool usage awareness and selection abilities of large language models (LLMs). The TOOLE dataset contains various user queries that prompt LLMs to use tools, and the benchmark includes tasks for both tool usage awareness and tool selection. The paper presents the results of experiments with several LLMs on the benchmark, and analyzes the performance of the models on different subtasks. The authors also provide insights into the factors that influence tool selection, and discuss potential real-world applications for LLMs with strong tool usage awareness and selection abilities. Overall, the paper's contributions include the development of a new benchmark for evaluating LLMs, insights into the factors that influence tool selection, and a framework for analyzing the performance of LLMs on tool usage tasks.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy-to-follow. The visualizations are clear and can help readers easily understand the task definitions, model performances, and framework.\n- The paper is highly original in its approach to evaluating the tool usage awareness and selection abilities of large language models. The authors introduce a new benchmark, the TOOLE dataset, which includes a wide range of user queries generated using various prompting methods.\n- The authors provide detailed descriptions of the TOOLE dataset and the four subtasks in tool selection, and they analyze the performance of several LLMs on the benchmark. The paper's results are presented clearly and are supported by statistical analysis. \n- The conclusions are very insightful: the more detailed the description, the more efficient tool selection.", "weaknesses": "- I think the comparison with existing benchmarks still needs polishing. To the best of my knowledge, I have found these places to be potentially incorrect: (1) API-Bank (https://arxiv.org/pdf/2304.08244v1.pdf) has included the task ① of determining whether LLMs need to leverage external tools or not (level-1 description in Section 1 Page 2. Thus, it might be incorrect to describe API-Bank as only focusing on tasks ③④; (2) ToolBench (https://arxiv.org/abs/2305.16504) also involves the task ②, requiring the retriever to retrieve the most relevant tools. Thus, it is also incorrect to describe the ToolBench as a benchmark that only focuses on tasks ③④; (3) There are also some benchmarks that are missing in the table, like ToolQA (https://arxiv.org/pdf/2306.13304.pdf). In summary, the authors better clarify the differences compared with these existing benchmarks more clearly.\n- Most statements in the experiments are not well explained. I think the authors should focus more on the potential reasons and analysis behind these observations.\n- I am a little bit confused about the second conclusion you have obtained in Section 3.3. Can you provide more details about the comparison between generated tool descriptions and those provided by tool developers. Like, if we rewrite all the tool descriptions by tool developers with ChatGPT, will we observe a performance gain?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aFazVLQjKq", "reviewer": "Reviewer_xJ5F", "summary": "The paper investigates whether models know when to use tools, and whether they know what tools to use. \n\nThe first task is “awareness of tool usage” and it’s a binary classification task. The authors construct the ToolE dataset for this task; the queries are generated using various prompting methods (“emotional generation,” “keyword generation,” “direct diverse generation,” “detailed generation).\n\nThe second task is “tool selection.” Four subtasks are proposed to evaluate tool selection: (1) Tool selection with similar choices: select the correct tool from a list containing similar tools. (2) Tool selection in specific scenarios (specialized scenarios). (3) Tool selection with possible reliability issues: given query and tool t, construct tool list such that the t is not in the tool list; assess if LLMs can avoid choosing tools that do not exist in the list. (4) Multi-tool selection. \n\nThe tasks seem to be done in a zero-shot manner based on the prompts in the appendix. For the first task, most models achieve ~random accuracy (except for ChatGPT and Vicuna which are better than the others – also potentially because these models are largest in the batch). Performance on the second task is unsatisfactory too.", "questions": "Did you use the SFT-tuned & RLHF-tuned llama (which should be llama2-chat) or the plain pretrained llama (without SFT/RLHF finetuning)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Tool use is one of the most powerful ways to improve LLM performance, and there has been lots of recent interest (Toolformer, or even generating new tools).\n\nIt’s good that many of the examples shown in this paper are actually how humans may use an assistant in the future. So although the benchmark is artificial, the queries are often relevant to the real world.", "weaknesses": "The “awareness of tool usage” dataset contains positive and negative examples. Positive examples are the ones that require tools, and negative examples are the ones that can be directly solved by LLMs. For negative examples, the authors use three instruction tuning datasets like Commonsense QA and instruction datasets in LIMA. I’m a bit confused because a lot of those questions are quite difficult, and intuitively would definitely benefit from retrieval, for example. The reason LLMs can solve them may be because they appeared in the pretraining/fine-tuning datasets already, so LLMs remember the examples. So why are those instruction tuning datasets in the negative subset (if it wouldn’t harm if they search for info on the internet)? \n\nSome answers (on whether LLMs need tools for a Q) in Table 12 and Table 13 are debatable in my opinion. I wonder how the authors define a reference answer (on whether LLMs need to use a tool) if with or without the tool, LLM can both solve the question. Similar issue for the second task (tool selection). What would be the human performance on these tasks? \n\nRelevant to the above two: would it be more prudent to measure “whether LLMs know when to use tools / whether they can use tools” by actually performing tool use experiments – the metric would be the accuracy on the downstream task (e.g., whether the weather actually matches internet search, or whether some calculation actually equals the correct answer, or whether a 2-hour reminder is created in some simulator)?\n\nThe experiments are done zero-shot. I wonder if few-shot prompting (with chain of thought reasoning) can improve the performance (on both tasks) by a lot. \n \nI wonder what the tool descriptions are. I don’t see detailed tool descriptions in the paper. I also don’t see any supplementary materials in the submission. In the tool description, is there info on *when* a *language model* should use the tool? If not, then I naturally don’t expect LLMs to do well on the tasks in this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)