T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step (2024)

Abstract: Large language models (LLMs) have achieved remarkable performance on various NLP tasks and are augmented by tools for broader applications. Yet, how to evaluate and analyze the tool utilization capability of LLMs is still under-explored. In contrast to previous works that evaluate models holistically, we comprehensively decompose the tool utilization into multiple sub-processes, including instruction following, planning, reasoning, retrieval, understanding, and review. Based on that, we further introduce T-Eval to evaluate the tool-utilization capability step by step. T-Eval disentangles the tool utilization evaluation into several sub-domains along model capabilities, facilitating the inner understanding of both holistic and isolated competency of LLMs. We conduct extensive experiments on T-Eval and in-depth analysis of various LLMs. T-Eval not only exhibits consistency with the outcome-oriented evaluation but also provides a more fine-grained analysis of the capabilities of LLMs, providing a new perspective in LLM evaluation on tool-utilization ability. The benchmark will be available at https://github.com/open-compass/T-Eval.

heading: 1 Introduction
text: Large language models (LLMs) have fueled dramatic progress and emerged as a promising path to more advanced intelligence (Zhao et al., 2023; Kaddour et al., 2023; Chen et al., 2024a,b). To further extend the capability of LLMs, tool utilization, which empowers LLMs to leverage external tools to solve more complicated problems, has spurred vast research interests in both research and industry (Parisi et al., 2022; Schick et al., 2023; Ji et al., 2024; Mialon et al., 2023).
Despite the attractive ability achieved by aiming LLMs with tools, how to evaluate LLMs in
∗ Equal Contributions † Corresponding author
tool learning has not been fully explored. Existing works evaluate the tool utilization ability based on the final output (Qin et al., 2023b) or only consider the single-step tool calling (Li et al., 2023b). However, real-world problems usually involve complex planning and executing multiple tools. Simply judging the quality through the final output omits the assessment of the intermediate steps, making it hard to identify the main bottlenecks of the tool-use capability in LLMs. Besides, current benchmarks mainly rely on real-time tool interactions (Qin et al., 2023b; Li et al., 2023b), which overlooks the external factors (instability of API service or temporal information shift) on the overall judgment, leading to evaluation variance and unfair comparison.
To overcome the above problems, we introduce T-Eval, a step-by-step Tool Evaluation benchmark for LLMs. Unlike prior works that appraise the model from a holistic perspective, we explicitly decompose the evaluation into several sub-tasks along the basic capabilities of the language model. Specifically, given the golden tool-utilization annotations verified by human experts, we dedicatedly designed the evaluation protocols and corresponding instruction prompts based on the intermediate steps along the annotation path. Such a paradigm enables us to separately benchmark each competence of the LLMs, including planning, reasoning, retrieval, understanding, instruction following, and review. Additionally, thanks to the decomposed evaluation protocols, our benchmark significantly alleviates the exogenous influences (such as online tools) during the evaluation process, yielding a more stable and fair model assessment.
By conducting extensive experiments on T-Eval, we carry out in-depth analysis and insights on the results, pinpointing the main bottlenecks of current LLMs in tool learning. Furthermore, we also prove that our benchmark reveals consistent evaluation of individual and comprehensive model abilities, where higher individual ability scores lead to better
9510
performance on complex downstream tasks, providing a new perspective in LLM evaluation on tool utilization. Our major contributions are as follows:
• We introduce T-Eval, a step-by-step tool utilization evaluation benchmark, which decomposes the evaluation into several sub-tasks, gauging the fine-grained abilities of LLMs as tool agents.
• T-Eval uses a multi-agent data generation pipeline verified by human experts. This approach significantly reduces the impact of external factors, leading to a more stable and fair assessment of the LLMs.
• Extensive experiments conducted with various LLMs validate the effectiveness and generalization of T-Eval, providing valuable insights into bottlenecks of current LLMs, and offering new perspectives in improving tool-utilization capabilities.

heading: 2 T-Eval
text: Benchmarking LLMs as tool agents involves multiple dimensions of evaluations of LLM abilities and suffers from the external influence of tools. Therefore, we first thoroughly investigate each critical dimension of the tool-calling process (§2.1), and then establish tailored evaluation protocols for each dimension (§2.2), named T-Eval, to enable a detailed evaluation of tool utilization capability. To
guarantee the high quality of the golden solution paths and tool-calling responses, we adopt a humanin-the-loop data generation pipeline (§2.3), ensuring the stability and longevity of T-Eval. Lastly, we provide a statistical overview of T-Eval (§2.4).

heading: 2.1 Evaluation Decomposition
text: Tool utilization with large language models (LLMs) encompasses a variety of scenarios, touching upon multiple dimensions of capabilities. To better understand the whole process, we first deconstruct the tool-calling process into several key aspects, as depicted in Fig. 1.
First, solving complex real-world problems frequently requires a multi-step approach to tool calling. For example, to know the weather from a month ago, an LLM must first confirm the current date before it can query a weather tool. This requires a robust planning ability (Fig. 1(a)) to develop a strategy for tool calling that guides subsequent actions. Moreover, the contexts in which tools are utilized can be intricate, involving tool descriptions, documentation, user queries, previous interactions, and prior observations. Strong reasoning abilities (Fig. 1(b)) are essential for LLMs to understand these contexts and tools, generating logical thoughts for the next steps. After generating a thought, selecting the appropriate tools from a given list is crucial, demanding effective retrieval skills (Fig. 1(c)). Additionally, integrating the correct parameters requires the understanding
ability (Fig. 1(d)) to interpret tool documentation and corresponding thoughts. Finally, executing the tool-calling action mandates adept instruction following skills (Fig. 1(e)) to formulate precise requests for the relevant APIs. Each tool call executed by LLM must be evaluated to ensure the response meets the intended objective, especially when tools might be unavailable or not perform as anticipated. This crucial evaluation, named the review ability (Fig. 1(f)), involves examining tool responses and ascertaining if adequate information has been obtained to resolve the query.
In summary, thorough analyses of each dimension are vital for a comprehensive evaluation of tool-utilization capabilities. Therefore, we introduce T-Eval, a framework that decomposes the multi-step tool-calling process into fundamental abilities, evaluating them individually for a more nuanced understanding of tool utilization.

heading: 2.2 Fine-Grained Evaluation Protocol
text: T-Eval takes all the ability dimensions as mentioned above (PLAN, REASON, RETRIEVE, UNDERSTAND, INSTRUCT, and REVIEW) into consideration, measuring not only the overall performance of tool-utilization but also detailed scores in each dimension.

heading: 2.2.1 Definition
text: To formalize, this paper considers a piece of query data as a tuple (T, q), where T = [tool1, ..., toolk] is the tool list and q is the query. For each query data piece (T, q), we define the solution path S = [(ti, ai, oi, ri)] n 1 for the query q as a sequence of thought(t)-action(a)-observation(o)-review(r) pair along with the final answer A, where ti, ai, oi, ri denotes the thought, the tool-calling action, the observation (i.e., the tool response), and the review on the response at step i. Moreover, an action is regarded as a pair (tool, args), where tool is the tool name and args is the parameters to call the tool. Besides the solution path, a plan for a query data piece is defined as a sequence P = [a1, ..., an] donating the proposed action sequence to call at each step.

heading: 2.2.2 Single-Index Evaluation
text: We create individual metrics to quantitatively analyze LLM abilities for each dimension of tool utilization. Here we describe the measurement of each dimension, and the detailed metric function can be found in Appendix C.
• PLAN: Given a tool list T and query q, the LLM is asked to generate a proposed tool-calling action sequence P pred = [apred1 , a pred 2 , ..., a pred n ], where a pred i is the pre-
dicted LLM action at step i. The planning evaluator then compares P pred with the golden answer P gt by matching actions in both sequences using Sentence-BERT (Reimers and Gurevych, 2019) for similarity calculation and Hopcroft-Karp matching (Hopcroft and Karp, 1973) for maximal similarity pairing. The planning score is measured by the length of the longest-ordered action sequence in the similarity pairing.
• REASON: Given a tool list T , query q, and a prefix of the solution path, the reasoning evaluator asks the LLM to generate the next thought tpredi+1 . The similarity between t pred i+1
and the golden answer tgti+1 is then measured.
• RETRIEVE: Given a tool list T , query q, and a prefix of the solution path, the retrieval evaluator asks the LLM to choose the next tool toolpred to call, comparing it with the golden answer toolgt.
• UNDERSTAND: Given a tool list T , query q, the understanding evaluator asks the LLM to generate appropriate parameters argspred for the next step and then compares them with the golden answer argsgt for similarity.
• INSTRUCT: Given a thought ti with the desired tool and parameters, the LLM is tasked with generating a tool-calling request in a specified format. The evaluator then calculates the accuracy of the tool name and parameter values in this format.
• REVIEW: Given a thought ti and a tool response oi, the LLM is tasked with judging whether the tool response successfully achieves the goal mentioned in the thoughts. It must also determine the type of errors if the goal is not achieved. This evaluation is conducted as a multiple-choice problem with five options: Success, Internal Error, Input Error, Irrelevant Response, and Unable to Accomplish.

heading: 2.2.3 End-to-End Evaluation
text: End-to-end evaluation requires LLMs to generate the whole solution path Spred as well as the fi-
nal answer Apred given one query data piece. We adopt the win rate proposed in ToolBench (Qin et al., 2023b) to gauge the overall performance, which evaluates LLM abilities by comparing their response quality against that of GPT-3.5. The result shows that our single-index evaluation is consistent with the overall performance.

heading: 2.3 Dataset Construction
text: The construction of T-Eval consists of three main phases: tool collection, instruction generation, and golden solution annotation. The overview of the construction is shown in Fig. 2.

heading: 2.3.1 Tool Collection
text: The collection quality of tools has a direct impact on instruction generation and tool utilization evaluation. We follow two principles during the collection process: • High Availability and Usage Rate. Considering that T-Eval is expected to cover most daily and practical use cases, we carefully select 1 ∼ 2 tools for each specific domain, including Research, Travel, Entertainment, Web, Life, and Financials, resulting in 15 tools as our basic tool set. • Complete Documentation. Despite the numerous tools collected in ToolBench (Qin et al., 2023b) from RapidAPI, the documentation quality is not guaranteed. To reduce the failure of toolcalling cases caused by inadequate tool descriptions, which focus the evaluation attention on pure LLM abilities, we manually generate high-quality and detailed tool documentation for each tool.

heading: 2.3.2 Instruction Generation
text: The testing instructions determine the practicality and difficulty of our evaluation. To guarantee the diversity of the queries, we uniformly sample 2 ∼ 3 tools each time and prompt GPT-3.5 to generate N
instructions Q = {q1, ..., qN} that need these tools. Concretely, the prompt consists of three parts: (1) the instruction that requests LLM to generate corresponding queries, (2) detailed tool documentation, and (3) few-shot examples. We randomly shuffle the tool documentation list and select different fewshot examples each time, so that the LLM can pay different attention to the text thereby encouraging the model to create wide-ranging instructions. After that, the stronger GPT-4 is utilized to revise and refine the generated instructions, aiming to further enhance the feasibility and diversity. The detailed prompts are listed in the Appendix B.

heading: 2.3.3 Golden Solution Annotation
text: Annotating the solution path manually to various queries is labor-intensive and unable to scale up the dataset quickly. To overcome this problem, we leverage a novel multi-agent paradigm plugging with simple human verification to resolve the complicated and massive solution annotations. Specifically, instead of instantiating only one LLM to handle the whole solution annotation path, we explicitly disentangle the annotation task into three different functionalities, including planner, executor, and reviewer: the planner decides what should be done in the next step; the executor is responsible for generating the exact tool name as well as its parameters and executing the tool to obtain the response. The reviewer is assigned to revise the response from the tool and judge if the task is finished given the external feedback.
Thanks to the decomposition of functionalities, each agent can accomplish its duty without switching the role required by each step, therefore, significantly reducing the error generation during the annotation process compared to conventional CoT (Wei et al., 2022) or ReAct (Yao et al., 2022) ap-
proaches. To guarantee the quality of the calling process, we adopt GPT-3.5 to accomplish the whole machine annotation phase. In the end, human annotators are employed to review the solution chain and manually filter invalid instructionsolution pairs.

heading: 2.3.4 Inclusive Difficulty Evaluation
text: During the evaluation, we empirically found that a few LLMs, especially small-scale ones, demonstrate poor ability in instruction following, therefore yielding responses that are unparseable with the pre-agreed format. Due to the large amount of parse failures on the response, the evaluation score can get distorted, losing the authenticity to reflect the real ability of the model.
To address the issue, we carefully designed the instruction task prompt and the evaluation granularity with both easy and difficult levels, providing inclusive evaluation on most language models. Specifically, the easy level adheres to a simple string format and focuses more on the semantic quality of the text, while the difficult level adopts JSON format, which is more commonly used in products (e.g., the JSON mode1 in GPT4 (OpenAI, 2023)), and conducts a more strict, finegrained evaluation of the response content, e.g., exact match on tool name and parameters.

heading: 2.4 Dataset Summary
text: To this end, we generate 1,500 initial instructionsolution pairs and pick 553 after two-round human verifications. We extract the desired information required by each evaluation protocol to construct the respective INSTRUCT, RETRIEVE, PLAN, REASON, UNDERSTAND, REVIEW subsets for T-Eval benchmark, resulting in 23,305 test cases in total (Refer to Appendix A.1 for more detailed statistics of T-Eval).

heading: 3 Experiments

heading: 3.1 Experimental Setup
text: We evaluate both API-based commercial and opensource LLMs on T-Eval, with a total number of 20 models, aiming to provide a comprehensive benchmark for current large language models. (1) For API-based LLMs, we select three representative models: GPT-3.5 and GPT-4 from OpenAI,
1https://openai.com/blog/new-models-and-developerproducts-announced-at-devday
and Claude2 from Anthropic. 2 (2) For open-source LLMs, we choose a wide spectrum of models, including LLaMA2 (Touvron et al., 2023), CodeLLaMA (Roziere et al., 2023), QWen (Bai et al., 2023), InternLM (Team, 2023), Baichuan2 (Yang et al., 2023), WizardLM (Xu et al., 2023b), Vicuna (Chiang et al., 2023), AgentLM (Zeng et al., 2023), Mistral (Jiang et al., 2023) and ChatGLM3 (Zeng et al., 2022).

heading: 3.2 Main Results
text: The detailed experimental results are shown in Tab. 1. In this section, We aim to answer three research questions below. Q1: Which Model is Better at Tool Utilization? The results in Tab. 1 show that GPT-4 achieves the highest score, with an overall score of 86.4, setting the pilot of the well-instructed and skillful tool-utilization LLMs. Apart from GPT-4, APIbased commercial LLMs, including GPT-3.5 and Claude2, get competitive scores on both string and JSON formats, indicating their strong abilities in acting as tool agents.
As for open-source models, we evaluate models with three different scales: around 7B, 13B, and 70B. It can be concluded from Fig. 3 that the performance of the model monotonically increases as the model scale increases. Among them, Qwen-7B gets the best of two worlds in terms of the model sizes and evaluation scores. With 7 billion parameters, Qwen-7B exhibits a competitive ability to understand complicated instructions and reply in
2Our experiments are conducted between 12/01/2023 and 12/10/2023. The version for GPT-4 is gpt-4-1106-preview, for GPT-3.5 is gpt-3.5-turbo-16k, and for Claude2 is claude-2.1.
a strict format in JSON. When scaled to 72B, the overall score of Qwen rises to 71.4%, significantly reducing the gap between open-source and APIbased models. We attribute this to the training on a human-in-loop self-instruction dataset, which encompasses high-quality format-specific instructions generated by Qwen team (Bai et al., 2023). Q2: How Far Are We from Skillful Tool Agents? By explicitly disentangling the evaluation through model abilities, we can gain a deeper understanding of the pros and cons of current LLMs, providing new perspectives in developing better tool agents.
First, open-source LLMs lack the instructionfollowing ability to respond with specific formats, which is the very first step to constructing wellregulated and high-usable tool agents. Without a legal response format, the system can not successfully extract the information generated by LLMs by a constant protocol, not to mention the correct tool executions. However, only a small amount of
models achieve both high scores on INSTRUCT subset under string and JSON format. Besides, there are still large performance gaps between string and JSON evaluation protocols on other subsets. Considering that the understand ability of the Qwen72B is comparable with GPT-4 evaluated in string format (84.5 vs 83.2), its JSON format result is more than 20 points lower, pinpointing the necessity to enhance the ability of open-source LLMs solving problems with specific formats.
Second, tool retrieval presents a relatively challenging task for most LLMs. Even the largest opensource model, Qwen-72B only reaches 65.0%, which is more than 20 points lower than GPT-3.5.
Lastly, compared to planning, LLMs are more likely to perform worse in reviewing the status of API responses (i.e., review in T-Eval), which is a core capability to interact with the dynamic environment when acting as an agent. Most models only reach 50%∼60%, compared to 95% achieved
by GPT-4, indicating that more attention should be paid to the review ability of current LLMs. Q3: What Makes for Good Training Data for Tool Utilization? Supervised finetuning is an efficient and necessary practice in empowering LLMs with certain abilities on downstream tasks. Recent research (Ouyang et al., 2022; Touvron et al., 2023) finds that high-quality dialogue-style instruction data is the key to well-instructed LLMs. However, the analysis of what makes for good training data for tool utilization is under-explored. We identify two types of training data: (1) general instruction following data, and (2) task-specific tuning data.
As for high-quality general instructions, Vicuna adopts user-shared conversations collected from ShareGPT3, and WizardLM uses complex and diverse instructions by evolving existing data in depth and width. Both of them are trained starting from LLaMA2, providing natural ablation on the effectiveness of high-quality (diverse and complex) instructions to the tool learning ability of LLM. When the model scale is small (7B), these data types enhance the model considerably. However, the increments diminish (even worse) as the model scales up (see WizardLM-70B). This further indicates that simply scaling the model scale does not always bring improvements, proper training data also matter in the scaling law.
In terms of task-specific tuning data, we select two typical types of corpus: code and agent, corresponding to CodeLLaMA and AgentLM, respectively. Compared to CodeLLaMA, which uses code data, AgentLM obtains better scores, showcasing that agent-related data may bring more benefits to tool learning. However, neither CodeLLaMA nor AgentLM appears to have significant advantages to Vicuna, indicating the necessity of high-quality instruction following data for tool utilization.

heading: 4 Discussion

heading: 4.1 Format Following v.s Problem Solving
text: Format following is an essential ability of LLMs, i.e., reply with a specific format when executing certain tasks. There are massive efforts devoted to enhancing this ability when using LLM as agents (Zhou et al., 2023; Xu et al., 2023b). In our experiments, we find that this ability may need to be acquired jointly with problem-solving. For instance, both ChatGLM3-6B and Baichuan2-7B obtain roughly 80% with the JSON evaluation pro-
3https://sharegpt.com/
tocol on INSTRUCT subset, which reveals them holding a strong ability in JSON format rendering. However, they struggle to generate valid JSON format responses on PLAN and REASON subsets, i.e., the divergences of JSON and string scores are quite large on these subsets. Such a phenomenon suggests that the ability to output specific formats, i.e., JSON, does not guarantee the ability to resolve all the problems with this format. This further indicates that one should integrate the requested format into the tasks and train them jointly so that the model can understand and behave well under certain protocols.

heading: 4.2 Inclusive Evaluation Protocol
text: From Tab. 1, it can be observed that quite a few amount of open-source LLMs struggle on the JSON evaluated protocols, especially on the REASON, RETRIEVE and UNDERSTAND subsets. Although the JSON format evaluation best approximates the real use cases of tool agents, it fails to provide hierarchy discriminations across various models when they are not adept at specific instructions requested by the task. For instance, Baichuan2-13B exhibits poor abilities in JSON format instruction (only 51.7 on INSTRUCT, JSON), which leads to low scores under JSON evaluations in PLAN subset. However, it has little relationship with the basic ability of model planning, since it achieves 69.1% when evaluated with string format, which is 17.4 points higher than its JSON format (52.1%). This validates the necessity to provide continual difficulty level evaluation protocols (Schaeffer et al., 2023), otherwise, one can get little understanding of the detailed abilities of LLMs on this benchmark but a low score simply due to the incorrect format, especially for weak models. In T-Eval, by seamlessly converting the strict format matching into semantic sentence analysis, our inclusive evaluation protocol gets rid of the inflexible measurement, unearthing the inner capability of the model.

heading: 4.3 Comparison to Other Benchmarks
text: We compare our fine-grained evaluation protocols with existing tool evaluation approaches and investigate if they show the same trends as ours. We adopt the win rate proposed in ToolBench (Qin et al., 2023b) as the representative holistic evaluation method, and evaluate several open-source models by comparing the response quality with GPT3.5-turbo, judged by GPT-4. The results are shown in Fig. 4. We can find that the holistic evaluation
reveals similar trends in these models with that in T-Eval, which validates the reasonability and generalization of our benchmark. When taking a close look at the results, we can observe that Qwen-7B achieves a 52% win rate over GPT-3.5-turbo response. However, there still exists a gap between Qwen-7B and GPT-3.5 in various ability domains under human judgment (Zheng et al., 2023), which implies that holistic evaluation is sometimes inaccurate. Our evaluation protocol not only reflects such divergence clearly, but also showcases the details of abilities in tool utilization, suggesting that T-Eval to be a more rational and comprehensive evaluation benchmark for tool utilization.

heading: 5 Related Work
text: Augmenting LLMs with Tools There are two paradigms to empower LLMs with external tools, and the first one is regarding external tools as specific tokens and fine-tuning parts or full of the model (Schick et al., 2023; Parisi et al., 2022; Lewis et al., 2020; Hao et al., 2023). However, these methods need a large amount of tool-relevant data and struggle to adapt newly appeared tools. Recently, the strong in-context learning ability (Brown et al., 2020) promotes researchers to focus more on the second paradigm, which is augmenting LLMs with tools by giving in-context tool descriptions and demonstrations (Hsieh et al., 2023; Mialon et al., 2023; Ruan et al., 2023; Patil et al., 2023). This paradigm has achieved great tool-calling potentiality and resulted in successful applications such as ChatGPT plugins. T-Eval focus on this paradigm and evaluate scores of various foundation models.
Evaluating LLMs LLM evaluation is essential to ensure that LLM can be effective in understanding and generating human-preferred text and reliable for deployment in real-world applications (Guo et al., 2023; Chang et al., 2023). Many benchmarks have been established to evaluate base abilities on question-answering tasks (Rajpurkar et al., 2016; Clark et al., 2018; Glockner et al., 2018), natural language understanding tasks (Wang et al., 2018, 2019; Hendrycks et al., 2020), and commonsense reasoning tasks (Lu et al., 2022). Recently, LLM evaluation has extended towards specific directions like code generation (Chen et al., 2021; Austin et al., 2021; Du et al., 2023) and hallucination (Li et al., 2023a; Chen et al., 2023). Some benchmarks also test the performance of LLMbased agents in a wide range of scenarios (Liu et al., 2023a; Wang et al., 2022).
Several benchmarks exist for evaluating tool utilization, focusing primarily on aspects of response comparison (e.g., ToolQA (Zhuang et al., 2023)), tool call accuracy (e.g., Gorilla (Patil et al., 2023)), or a combination of both (e.g., API-Bank (Li et al., 2023b)). ToolBench (Qin et al., 2023b) introduces a novel approach by employing an LLM as a judger to assess the overall solution path. Furthermore, the study in (Qin et al., 2023a) investigates the performance improvement attributable to tool utilization. Different from above, T-Eval emerges as the first benchmark dedicated to the fine-grained evaluation of tool utilization capabilities.
Prompting LLMs as Agents Prompting enhances the reasoning capabilities of LLMs by providing instructions or examples. Techniques such as the Chain of Thought (CoT) and Tree of Thought (ToT) (Wei et al., 2022; Yao et al., 2023) encourage LLMs to engage in comprehensive thinking for more accurate reasoning. Advanced systems like ReAct, ReWOO, SwiftSage, DyLAN, and DPLLM (Yao et al., 2022; Xu et al., 2023a; Lin et al., 2023; Liu et al., 2023b; Dagan et al., 2023) further develop LLM agents. These systems use advanced prompting methods to guide LLMs, unleashing the potential of models.

heading: 6 Conclusion
text: In this paper, we propose T-Eval, a comprehensive and fine-grained tool utilization evaluation benchmark for LLMs. T-Eval explicitly disentangles the tool utilization tasks along the model ability, with dedicated evaluation protocols designed for
respective tasks, unearthing the real ability of the evaluated models. Such a step-wise evaluation delivers a thorough analysis and pinpoints the main bottlenecks of current LLMs in tool learning, providing valuable insights into further development of tool agents.

heading: 7 Limitations
text: Despite T-Eval mainly utilizing the advanced GPT model for dataset generation, it still requires human efforts to verify the correctness of the generated content. It is essential to acknowledge that the generated content may not be well aligned with human preference. Therefore, it would be hard to scale up the construction process to a larger scale. We aim to address this problem in our future work.

heading: 8 Ethical Considerations
text: We used publicly available reference documents/APIs for our benchmarks, effectively circumventing any possible harm toward individuals or groups. The generated data by LLMs were carefully selected and processed by humans to secure privacy and confidentiality. No personal identification information was involved, and all data were made anonymous before any analysis was conducted. Besides, We use ChatGPT and Grammarly to polish the writing.

heading: 9 Acknowledgments
text: This work was supported by the JKW Research Funds under Grant 20-163-14-LZ-001-004-01, and the Anhui Provincial Natural Science Foundation under Grant 2108085UD12. We acknowledge the support of GPU cluster built by MCC Lab of Information Science and Technology Institution, USTC.

heading: A.1 Dataset Statistics
text: T-Eval originates from 533 high-quality querysolution annotation pairs, consisting of 23,305 test cases in total, ranging from INSTRUCT, PLAN, REASON, RETRIEVE, UNDERSTAND and REVIEW subsets. Detailed statistics of each subset are shown in Tab. 2. We also visualize the distribution of tool calling steps in the whole annotation paths in Fig. 5. T-Eval covers all tool sets and yields 5.8 average calling steps for each query, validating the generalization and discrimination for tool utilization evaluation.
B Implementation Details
Experimental Details. To evaluate the pure ability of the single model, we adopt ReAct (Yao et al., 2022) as the basic agent paradigm for end-to-end evaluation and limit the maximum action step to 20 to ensure the efficient and accurate questionsolving ability of LLM. As for the single-index evaluation, we prompt with a multi-turn conversation style to the LLM and gauge the response. If not specified, we choose the ‘chat’/‘instruct’ version of open-sourced models for evaluation.

heading: B.1 Prompts Demonstration
text: Please refer to the respective prompt block for a detailed demonstration.

heading: B.1.1 Query Generation
text: The corresponding prompt is presented in Fig. 6.

heading: B.1.2 Query Refinement
text: The corresponding prompt is presented in Fig. 7.

heading: B.1.3 Multi-Agent Annotation Prompt
text: The corresponding prompt is presented in Fig. 8.

heading: B.2 Dataset Demonstration
text: Please refer to the respective prompt block for each detailed dataset demonstration.
B.2.1 INSTRUCT The corresponding prompt is presented in Fig. 9.
B.2.2 PLAN The corresponding prompt is presented in Fig. 10.

heading: B.2.3 REASON
text: The corresponding prompt is presented in Fig. 11.

heading: B.2.4 RETRIEVE
text: The corresponding prompt is presented in Fig. 12.

heading: B.2.5 UNDERSTAND
text: The corresponding prompt is presented in Fig. 13.

heading: B.2.6 REVIEW
text: The corresponding prompt is presented in Fig. 14.

heading: C Detailed Evaluation Metrics
text: T-Eval decomposes tool utilization capability into six ability dimensions: INSTRUCT, PLAN, REASON, RETRIEVE, UNDERSTAND and REVIEW, we carefully designed evaluators and metrics for all dimensions with two difficulty formats: JSON and string. The JSON format asks the LLM to generate standard JSON format responses, while the string format allows the LLM to answer in a relatively loose format.
Let us recap the formalization of each toolcalling component first: A piece of query data is considered as a tuple (T, q), where T = [tool1, ..., toolk] is the tool list with k tools and q is the query. For each query data piece (T, q), the solution path S = [(ti, ai, oi, ri)] n 1 is defined as a sequence of thought(t)-action(a)-observation(o)review(r) pair along with the final answer A, where ti, ai, oi, ri denotes the thought, the tool-calling action, the observation (i.e. the tool response), and the review on the response at step i, respectively. Moreover, an action a is regarded as a pair (tool, args), where tool is the tool name and args is the parameters to call the tool. Besides the solution path, a plan for a query data piece is defined as a sequence P = [a1, ..., an] donating the proposed actions to call at each step.
C.1 INSTRUCT The LLM is required to generate a tool-calling request using a specified template, based on the provided tool name and parameters. This request must adhere to a predetermined format, either in JSON or string, with varying template structures. Initially, the evaluator determines if the request meets
the format requirements. A passing score of 0.5 is awarded for successfully meeting these format standards. Once the format check is passed, the request is further evaluated for parameter accuracy, with a parameter score assigned. This score is calculated as 0.5 multiplied by the percentage of correctly matched parameters. The final score is the sum of the passing score and the parameter score.

heading: C.2 PLAN
text: The LLM is tasked with generating a plan using a provided list of tools to solve a query. To evaluate the similarity between the predicted plan from the LLM P pred = [apred1 , a pred 2 , ..., a pred npred ] and the gold answer P gt = [agt1 , a gt 2 , ..., a gt ngt ] from human annotators, the planning evaluator begins by computing a similarity matrix S. This matrix represents the similarity scores for all action pairs (ai = (tooli, argsi), aj = (toolj , argsj)) between the prediction and the golden answer:
Si,j = βσ(tooli, toolj)+(1−β)σ(argsi, argsj).
In this approach, σ is the similarity function between two sentences. We employ Sentence-BERT (Reimers and Gurevych, 2019) as σ, which involves embedding the two sentences and then calculating the cosine similarity between these embeddings as the similarity score. The underlying BERT model used is all-mpnet-base-v2.4 Furthermore, β is a hyperparameter that determines the relative importance of the tool name in comparison to the tool parameters in similarity calculation. In our implementation, we set β = 0.75.
After getting the similarity matrix S, a bipartite graph is built where one part is the set of predicted actions and another part is the set of golden answer actions. Two actions are linked if their similarity is greater than a predefined threshold, set at 0.7 in our implementation. We then employ the Hopcroft-Karp matching algorithm (Hopcroft and Karp, 1973) to compute a max-weighted match from this graph. Subsequently, the Longest Increasing Subsequence (LIS) algorithm is used to determine the longest-ordered action sequence within this max-weighted match. Denoting the length of this sequence as l, we calculate the precision and recall as p = l/npred and r = l/ngt, respectively.
4https://www.sbert.net/docs/pretrained_models.html
The plan score is thus defined as:
plan score = 2pr
p+ r .
Regarding the input formats, in the JSON format, the LLM is tasked with generating a list of actions, with each action represented as a dictionary comprising the tool name and its corresponding parameters. Conversely, in the string format, the LLM articulates each action in a separate line.

heading: C.3 REASON
text: Given a tool list T , query q, and a prefix of the solution path, the LLM is asked to generate the next thought tpredi+1 . The similarity between t pred i+1 and the golden answer tgti+1 is then measured, and the similarity is calculated by Sentence-BERT (the same as the planning evaluator). In the JSON format, LLMs need to generate a dictionary containing the next thought, as well as the next tool name and the corresponding parameters, which means we evaluate REASON, RETRIEVE, UNDERSTAND using the same LLM output. Regarding the string format, LLMs only need to generate the next thought in a single line.

heading: C.4 RETRIEVE
text: Given a tool list T , query q, and a prefix of the solution path, the LLM is asked to generate the next tool name toolpredi+1 to call, and then the evaluator judges whether the name is the same as the golden answer toolgti+1, achieving score 1 if the same and 0 otherwise. As mentioned in Sec. C.3, the LLM needs to generate a dictionary containing the next thought, as well as the next tool name and its corresponding parameters in the JSON format, while in the string format, the golden answer’s thought is given, and the LLM only needs to generate the next tool name in a single line.

heading: C.5 UNDERSTAND
text: Given a tool list T , query q, and a prefix of the solution path, the LLM is tasked with generating the parameters argspredi+1 to call the next tool, and the score is the similarity between argspredi+1 and argsgti+1 calculated by Sentence-BERT (the same as planning evaluator). As mentioned in Sec. C.3, the LLM needs to generate a dictionary containing the next thought, as well as the next tool name and its corresponding parameters in the JSON format, while in the string format, the golden answer’s
thought and tool name are given, and the LLM needs to generate the parameters in a single line.

heading: C.6 REVIEW
text: Given a thought ti and a tool response oi, the LLM is required to evaluate the tool response. It must select one of the following categories to classify the response: Success, Internal Error, Input Error, Irrelevant Response, or Unable to Accomplish. The evaluation is scored as 1 for a correct classification and 0 if the classification is incorrect.

heading: D API Documentation
text: We manually curate extensive API documentation for each tool, following the annotation format defined by OpenAI. Compared to official RapidAPI documentation, our descriptions are more extensive and detailed, which facilitates the understanding of various tools and circumvents the failure cases due to the incomplete API documentation provided in the benchmark. Here, we provide the API documentation for BINGMap as an example in Fig. 15.

heading: Future Work
text: We aim to address this problem in our future work