Enhancing Tool Learning in Large Language Models with Hierarchical Error Checklists
Authors: Yue Cui, Liuyi Yao, Shuchang Tao, Weijie Shi, Yaliang Li, Bolin Ding, Xiaofang Zhou
Accepted at: acl25

Enhancing Tool Learning in Large Language Models with Hierarchical Error Checklists

Yue Cui1,2, Liuyi Yao1, Shuchang Tao1, Weijie Shi2, Yaliang Li1
Bolin Ding1, and Xiaofang Zhou2
1Alibaba Group
2The Hong Kong University of Science and Technology
{ycuias,ashiah,zxf}@cse.ust.hk,
{lyl287738,taoshuchang.tsc,yaliang.li,bolin.ding}@alibaba-inc.com

Abstract

Large language models (LLMs) have significantly advanced natural language processing, particularly through the integration of external tools and APIs. However, their effectiveness is frequently hampered by parameter mis-filling during tool (function) calling. In this paper, we propose the Hierarchical Tool Error Checklist (HiTEC) framework to systematically diagnose and mitigate tool-calling errors without relying on extensive real-world interactions. HiTEC introduces a two-tiered approach: a global error checklist that identifies common, cross-tool issues, and a local error checklist that targets tool-specific and contextual failures. Building on this structure, we propose two deployments: HiTEC-In Context Learning (HiTEC-ICL) and HiTEC-Kahneman-Tversky Optimization (HiTEC-KTO). HiTEC-ICL embeds the global checklist in the initial prompts and leverages a two-round conversational interaction to dynamically refine parameter handling, while HiTEC-KTO generates high-quality negative examples to drive fine-tuning via preference-based optimization. Extensive experiments conducted on five public datasets show that our framework improves parameter-filling accuracy by up to 42% compared to baseline methods.

1 Introduction

Large language models (LLMs) have revolutionized natural language processing by enabling advanced comprehension, reasoning, and task execution capabilities. Among these, the integration of external tools and application programming interfaces (APIs) represents a critical milestone, allowing LLMs to expand their utility beyond textual analysis into interactive, task-oriented domains (Gou et al., 2023; Li et al., 2023; Qin et al.; Qu et al., 2025; Song et al., 2023; Tang et al., 2023; Wu et al., 2024). To achieve this, LLMs must navigate a complex process of function calling, which involves selecting the appropriate tools, formulating precise input arguments, and parsing results to satisfy tool input. Despite these advancements, tool-calling often suffers from incorrect parameter filling (Lin et al., 2024; Liu et al., 2024), a recurring issue that undermines the accuracy and reliability of LLM-driven interactions.

Most previous tool learning methods require LLM-tool interactions to improve the calling accuracy (Chen et al., 2024a; Qin et al.; Shi et al., 2024; Wang et al., 2024; Yang et al., 2024; Yao et al., 2022; Zhang et al., 2023). For example, STE (Wang et al., 2024) simulate plausible scenarios and incorporates execution feedback to enhance the correct use of tools. It involves first simulating queries, executing real tool calls via tool-LLM interactions, and learning from function calling outputs when errors occur. While real-world interactions with tools can yield valuable insights, they cause intensive resources (For example, 10-25$/1,000 transactions for Bing Search API 1) and instability issues (Guo et al., 2024). Furthermore, the errors encountered by most tools called by LLMs are predominantly common types that can be known in advance before real tool calling. To address these challenges, we argue that (1) tool-LLM interactions that involve real-world tool callings are not always necessary, and (2) structured error checklists can systematically identify potential performance deficiencies and guide targeted improvements.

Based on this motivation, this paper proposes a Hierarchical Tool Error Checklist (HiTEC) framework to enhance tool learning with LLMs by identifying and addressing tool-calling errors. As shown in Figure 1, The HiTEC framework is composed of the constructions of two levels of error checklists: a global error checklist, which captures general errors that frequently occur across different tools,

1https://www.microsoft.com/en-us/bing/apis/pricing
Figure 1: Pipeline of the Proposed Hierarchical Tool Error Checklist (HiTEC), which includes Global Error Checklist and Local Error Checklist

and a local error checklist, which focuses on tool-specific errors and contextual failures. These checklists provide a structured and comprehensive way to diagnose and rectify tool-calling errors without requiring extensive real-world execution.

We then deploy HiTEC in tuning-free and tuning-based ways and propose: HiTEC-In Context Learning (HiTEC-ICL) and HiTEC-Kahneman-Tversky Optimization (HiTEC-KTO). By embedding the global error checklist in the initial query and integrating the local error checklist through a two-round conversational interaction, HiTEC-ICL guides LLMs to preemptively avoid common mistakes and refine parameter handling flexibly. HiTEC-KTO leverages the error checklists to generate high-quality negative examples, which are then used to empower open-source LLMs through KTO-based fine-tuning. This strategy equips models with enhanced function calling accuracy by training them to recognize and correct errors. Our main contributions are as follows:

• Framework. We propose a novel hierarchical error checklist framework that integrates global and local error messages to enhance tool learning of LLMs. The framework distinguishes between offline self-generated errors and online tool-LLM interaction based accumulated errors to support adaptive and scalable tool learning.

• Method. We propose two novel methods: (1) HiTEC-ICL, an efficient mechanism to integrate error messages into LLM prompts for dynamic error reflection and correction; and (2) HiTEC-KTO, a negative sample generation method that overcomes the failure modes of preference-based optimization through fine-tuning with negative examples.

• Analysis. We theoretically and empirically demonstrate the effectiveness of KTO-based tuning in overcoming the failure modes of preference-based optimization in tool learning, thereby enabling LLMs to enhance their tool-calling accuracy through fine-tuning with negative examples.

• Performance. We conduct extensive experiments across five public datasets. Our results demonstrate improvements in parameter-filling accuracy, tool-calling success rates compared to baseline methods.

2 Related Work

LLM-based tool learning methods can be generally categorized into tuning-free and tuning-based approaches (Qu et al., 2025). Tuning-free methods leverage the inherent capabilities of large language models by prompting them to interact directly with external tools. This category includes techniques such as few-shot demonstrations (Wang et al., 2024), rule-based strategies (Shi et al., 2024; Yao et al., 2022; Zhang et al., 2023), and the use of optimized tool descriptions (Chen et al., 2024b; Yuan et al.) to facilitate efficient parameter extraction and tool usage. Owing to their simplicity and scalability, tuning-free methods have gained popularity; however, their performance often lags behind that of tuning-based approaches and a finally correct tool calling may require multiple times of tool-LLM interactions, which can be costly.

In contrast, tuning-based methods enhance tool calling performance through fine-tuning strategies. GPT4Tools (Yang et al., 2024) uses LoRA-based supervised fine-tuning, while the introduction of tool-specific tokens is demonstrated in Toolkenpt (Hao et al., 2023). Other approaches leverage augmented datasets (Lin et al., 2024), and utilize interactive path-based reasoning (Chen et al., 2024a; Qin et al.) to improve tool interaction precision. Relign (Xu et al., 2024) tackles tool hallucinations (also incorrect tool selection/usage) by expanding the action space with "indecisive actions" (e.g., deferring tool use) and aligning reliability via preference optimization. Although tuning-based methods consistently deliver superior performance by tailor-
Tool Information

Name: weather_api
Description: weather data for 20+ years historical archive and 4 days ahead forecast
Parameters: {"lat": {"type": "float", "description": "Latitude, decimal (-90; 90)"}, "lon": {"type": "float", "description": "Longitude, decimal (-180; 180)"}}, "time": {"type": "string", "description": "timestamp in the format of MM-DD-YYYY"}}
Required parameters: ["lat", "lon","time"]

Please read the following error checklist.

Error 0: Wrong Tool Name Error
...
Error 2: Missing Required Parameter Error
Query: The weather of 37°47'N 122°25'W on Jan 1st 2025
Function Calling Output: [{"name":"weather_api","arguments": [{"lon":37.47,"lat":125.25}]},{...}]
Error Message: {"error": "MissingRequiredParameter", "message": "The parameter "time" is required."}
Thought of Error: Parameter "time" is missing. Ensure all required parameters ("lat", "lon","time") are included in the function call.
...
Error 6: Redundant Information Error
...
Please avoid similar errors in the checklists and refine your previous answer.

Figure 2: The Local Error Checklist: a list of tool-specific issues that may arise during tool calling

Global Error Checklist
Please read the following error checklist.
Error 0: Wrong Tool Name Error
Error 1: Missing Required Parameter Error
Error 2: Invalid Parameter Type Error
Error 3: Empty Parameter Value Error
Error 4: Redundant Parameter Error
Error 5: Invalid Function Calling Output Format Error
Error 6: Redundant Information Error
Error 7: Wrong Number of Tools Error
Please avoid similar errors when making tool calling output.

Figure 3: The Global Error Checklist: a list of common issues that may arise during tool calling

ing LLMs specifically for tool calling tasks, they require high-quality training data or extensive tool interaction logs, which are still costly to obtain.

3 Method

In this section, we introduce the novel Hierarchical Tool Error Checklist (HiTEC) framework, comprising global and local error checklists to address parameter mis-filling in tool learning. We propose both tuning-free (HiTEC-ICL) and tuning-based (HiTEC-KTO) methods to integrate HiTEC into LLMs, enabling dynamic error reflection and correction based on pre-identified patterns and tool-specific information.

3.1 Tool Error Checklist

3.1.1 Global Error Checklist

The global error checklist is a list of common issues that arise during tool calling, and they are not associated with a specific query or tool. It is designed to address the most prevalent and impactful issues encountered during tool calling. Drawing from prior experiences with model-tool interactions, eight errors are selected to represent typical failure modes occurring at various stages of tool interaction, encompassing both tool-level and parameter-level mistakes.

At the tool level, we check errors Wrong Tool Name Error (Error 0) and Wrong Number of Tools Error (Error 7) highlight issues related to improper tool selection or usage. The parameter level encompasses a range of potential inaccuracies, including Missing Required Parameter Error (Error 1), Invalid Parameter Type Error (Error 2), and Empty Parameter Value Error (Error 3), and Redundant Parameter Error (Error 4). In addition, the global error checklist also incorporates errors related to redundancy. We check Redundant Information Error (Error 6), which ensures that the output remains concise and relevant, and Invalid Function Calling Output Format Error (Error 5) guards against syntactical inconsistencies that could disrupt downstream processes. This carefully constructed checklist of errors provides a robust foundation for systematic error identification and resolution in diverse tool-calling scenarios, significantly enhancing the reliability and efficiency of model-tool interactions. The global error checklist is presented in Figure 3.

3.1.2 Local Error Checklist

The local error checklist identifies errors related to the specific features of each tool. This checklist is crucial for addressing tool-specific issues,
as it offers detailed information that goes beyond the general overview provided by the global error checklist. Unlike the global error checklist, which only lists an overview of common errors, the local checklist focuses on the unique functionalities, parameters, and requirements of each tool. The local error checklist is essential for addressing tool-specific issues that may not be captured by the global checklist. A tailored local checklist ensures these specificities are considered.

The local error checklist of a tool contains the following components: tool information, simulated queries for each error type, the function calling output that can invoke the corresponding error, an error message describing the error, and a Thought of Error reflection that specifies how such an error can be corrected. An example of a local error checklist is illustrated in Figure 2.

To form such a local error checklist, it is required that tool-related information be known in advance. This includes details such as those outlined in the "Tool Information" section of the Error Checklist. This information should be appropriately placed within the <tool_info> field of the generation prompt as presented in Appendix A.

3.2 HiTEC-ICL: Enhancing LLM Tool Calling in Tuning-free Way

We integrate the designed global and local error checklists into the LLM-based tool-calling conversation to ensure precise and reliable tool utilization.

The global error checklist is embedded within the user’s initial query at the outset of the inference process. This proactive integration helps preempt common issues, such as tool name misidentification or parameter omission. By implementing these error prevention mechanisms early in the process, the system significantly enhances the accuracy and reliability of the initial tool invocation.

The local error checklist is primarily designed for parameter-specific error handling. Figure 4 illustrates this two-round interaction process. The first round captures the initial tool call, while the second round, guided by the local error checklist, ensures corrections are made to parameter filling and other tool-specific issues, leading to improved inference outcomes.

3.3 HiTEC-KTO: Enhancing LLM Tool Calling in Tuning-based Way

To empower open-source language models with robust tool-calling capabilities, we propose a method that utilizes error checklists to generate high-quality negative examples. This approach aims to enhance the performance of smaller, efficient models, enabling them to rival larger, more resource-intensive counterparts. By curating negative examples based on predefined global and local error checklists, we fine-tune open-source models, thereby equipping them with improved function-calling accuracy and error-handling capabilities.

3.3.1 Negative Example Generation

Negative examples are generated by combining correctly formatted tool-calling outputs with the corresponding local error checklist. By prompting the model to introduce specific errors described in the checklist, we create outputs that simulate real-world failures in tool-calling tasks. This curated dataset enables the fine-tuning of models to better recognize, avoid, and correct errors, ultimately improving their reliability and alignment with intended behaviors. We name the generated pairwise tool-calling dataset as PTC. The prompt for negative sample generation is presented in Appendix A.

3.3.2 Failure Mode of DPO on PTC Dataset

Given that the PTC dataset consists of pairwise data, an intuitive choice for tuning would be Direct Preference Optimization (DPO) (Rafailov et al., 2024) as the tuning method. However, applying DPO to such a dataset leads to a failure mode, as identified in previous research (Feng et al., 2024; Pal et al., 2024). This issue arises because the positive and negative responses in the PTC dataset differ by only a few tokens (more details are in Appendix D). Consequently, two problems occur: 1) the gradient of the DPO loss approaches zero, leading to a weak update signal, and 2) during optimization, the probability of the correct token tends

![An Example Conversation with the Global-Local Checklist as ICL Prompting](page_1016_180_377_246.png)

Figure 4: An Example Conversation with the Global-Local Checklist as ICL Prompting
to decrease when compared to the reference model (i.e., the initial model being tuned). We empirically validate the gradient vanish phenomenon by plotting the gradient norm during the training of DPO on the PTC dataset (for more details, refer to Figure 8(b) in Appendix D). Further, by plotting the log probabilities for both positive and negative samples (see Figure 8(c) in Appendix D), we observe that although the log probabilities of negative samples decrease, the log probabilities of positive samples also decrease, demonstrating the second challenge.

3.3.3 HiTEC-KTO
One representative DPO variant is Kahneman-Tversky Optimization (KTO) (Ethayarajh et al., 2024). We refer to the approach of fine-tuning large language models (LLMs) on a PTC-type dataset using KTO as HiTEC-KTO. In the following, we show the potential that KTO can address the above failure mode of DPO.

We formulate the KTO loss in a paired format:

\[
\mathcal{L}_{KTO}(x, y_w, y_l; \theta) \\
= \lambda_w - \lambda_w \sigma \left( \beta (r_\theta(x, y_w) - z_0) \right) \\
+ \lambda_l - \lambda_l \sigma \left( \beta (z_0 - r_\theta(x, y_l)) \right),
\]

where \((x, y_w, y_l)\) are paired data with \(x\) as the prompt, \(y_w\) as the positive answer and \(y_l\) as the negative answer. \(r_\theta(x, y)\) is the log-ratio of the likelihoods of answer \(y\) between the training model \(\pi_\theta(y|x)\) and the reference model \(\pi_{ref}(y|x)\), where \(\theta\) is the model parameter. \(r_\theta(x, y)\) is denoted as \(r_\theta(x, y) = \frac{\pi_\theta(y|x)}{\pi_{ref}(y|x)}\). \(z_0\) is the reference point, and \(z_0 = KL(\pi_\theta||\pi_{ref})\). Since \(z_0\) does not propagate gradients (Ethayarajh et al., 2024), it is treated as a constant during the differentiation of \(\mathcal{L}_{KTO}\).

Suppose \(y_w\) and \(y_l\) are length \(K\) sequences, and they only differ at i-th token, i.e., \(y_w = [t_1, \cdots, t_{i-1}, t_i^w, t_{i+1}, \cdots, t_K]\), and \(y_l = [t_1, \cdots, t_{i-1}, t_i^l, t_{i+1}, \cdots, t_K]\). Using the derivative perspective, similar to the approach in the prior section, we theoretically analyze KTO’s capability to mitigate the failure mode of DPO. The derivative of the KTO loss with respect to \(\theta\) is:

\[
\nabla_\theta \mathcal{L}_{KTO}(x, y_w, y_l; \theta) \\
= -a_w \log \pi_\theta(y_w|x) + a_l \nabla_\theta \log \pi_\theta(y_l|x),
\]

where the asymmetric weights are \(a_w = \lambda_w \sigma(c_w) \sigma(1 - c_w)\), and \(a_l = \lambda_l \sigma(c_l) \sigma(1 - c_l)\), with \(c_w = \beta \left( \log \frac{\pi_\theta(y_w|x)}{\pi_{ref}(y_w|x)} - z_0 \right)\) and \(c_l = \beta \left( \log \frac{\pi_\theta(y_l|x)}{\pi_{ref}(y_l|x)} - z_0 \right)\). Compared with DPO, these asymmetric weights provide stable gradients, guiding the model in adjusting the probability distribution effectively.

Moreover, to further investigate how KTO promotes probability shifts towards favorable outcomes during training, following (Pal et al., 2024) we re-arrange the derivative of the KTO loss as:

\[
\nabla_\theta \mathcal{L}_{KTO}(x, y_w, y_l; \theta) \\
= \sum_{k=1}^K \nabla_\theta \left[ -a_w \log \pi_\theta(t_k|y_w^{<k}, x) + a_l \log \pi_\theta(t_k|y_l^{<k}, x) \right].
\]

With the above formula, each term in the derivative expression Eqn. (3) becomes:

\[
\nabla_\theta \left[ -a_w \log \pi_\theta(t_k|y_w^{<k}, x) + a_l \log \pi_\theta(t_k|y_l^{<k}, x) \right] \\
= \begin{cases}
-a_w + a_l + a_w s_j^{y_w^{<k}, x} - a_l s_j^{y_l^{<k}, x} & t_k = V_j; \\
a_w s_j^{y_w^{<k}, x} - a_l s_j^{y_l^{<k}, x} & t_k \neq V_j.
\end{cases}
\]

This analysis reveals that, given that \(a_w > a_l\), it is evident that when the k-th token in \(y_w\) matches \(V_j\), the gradient is negative, whereas, for other vocabulary elements (\(t_k \neq V_j\)), the gradient is positive. Consequently, minimizing the KTO loss encourages an increase in the logits of the correct token, thereby effectively addressing DPO’s second issue as discussed in Section 3.3.2.

4 Experiment

<table>
  <tr>
    <th>Dataset</th>
    <th># Queries</th>
    <th># Tools</th>
    <th>Multi-tool?</th>
  </tr>
  <tr>
    <td>API-Bank L-1</td>
    <td>399</td>
    <td>49</td>
    <td>X</td>
  </tr>
  <tr>
    <td>API-Bank L-2</td>
    <td>127</td>
    <td>28</td>
    <td>X</td>
  </tr>
  <tr>
    <td>Tool-Alpaca</td>
    <td>114</td>
    <td>41</td>
    <td>✓</td>
  </tr>
  <tr>
    <td>Seal-Tools</td>
    <td>294</td>
    <td>1084</td>
    <td>X</td>
  </tr>
  <tr>
    <td>Nexus Raven</td>
    <td>318</td>
    <td>65</td>
    <td>X</td>
  </tr>
</table>

4.1 Experiment Setup

We here describe the datasets, metrics, and baselines used in experiments.

Datasets. We conduct an evaluation on several benchmark tool calling datasets: API-Bank (Li et al., 2023), Tool Alpaca (Tang et al., 2023), Seal-Tools (Wu et al., 2024), and Nexus Raven (Srinivasan et al., 2023). All datasets are preprocessed in the same way as Lin et al. (Lin et al., 2024). The statistics of the processed datasets are summarized in Table 1.

Metrics. We evaluate two performance metrics in the form of F1 score: correctness of tool names
Table 2: Tool Calling Performance with HiTEC-ICL, with the best performance marked in bold and the proposed approach highlighted in blue.

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th rowspan="2">Method</th>
    <th colspan="4">Dataset (F1 Name | F1 Name + Parameter)</th>
    <th colspan="3">F1 Average</th>
  </tr>
  <tr>
    <th>API-Bank L-1</th>
    <th>API-Bank L-2</th>
    <th>Tool-Alpaca</th>
    <th>Seal-Tools (Single-Tool)</th>
    <th>Nexus Raven</th>
    <th>Name</th>
    <th>Name+Param.</th>
  </tr>
  <tr>
    <td rowspan="3">GPT-4-Turbo</td>
    <td>Vanilla</td>
    <td>94.77</td>
    <td>86.66</td>
    <td>71.09</td>
    <td>70.43</td>
    <td>87.12</td>
    <td>60.75</td>
    <td>94.86</td>
    <td>88.55</td>
    <td><b>94.13</b></td>
    <td>82.92</td>
    <td>88.39</td>
    <td>77.86</td>
  </tr>
  <tr>
    <td>+ CoT</td>
    <td>91.70</td>
    <td>84.84</td>
    <td>67.86</td>
    <td>65.74</td>
    <td>80.95</td>
    <td>56.46</td>
    <td>91.50</td>
    <td>86.07</td>
    <td>92.18</td>
    <td><b>85.00</b></td>
    <td>84.84</td>
    <td>75.62</td>
  </tr>
  <tr>
    <td>+ Function Calling</td>
    <td>84.03</td>
    <td>81.40</td>
    <td>78.49</td>
    <td>62.32</td>
    <td>87.11</td>
    <td><b>62.16</b></td>
    <td>84.94</td>
    <td>84.88</td>
    <td>\</td>
    <td>\</td>
    <td>\</td>
    <td>\</td>
  </tr>
  <tr>
    <td>+ HiTEC-ICL</td>
    <td><b>94.96</b></td>
    <td><b>88.44</b></td>
    <td><b>71.21</b></td>
    <td><b>72.68</b></td>
    <td><b>87.35</b></td>
    <td><b>61.95</b></td>
    <td><b>96.00</b></td>
    <td><b>89.37</b></td>
    <td>92.81</td>
    <td>83.15</td>
    <td><b>88.47</b></td>
    <td><b>79.12</b></td>
  </tr>
  <tr>
    <td rowspan="3">Llama3-8B</td>
    <td>Vanilla</td>
    <td>69.19</td>
    <td>67.73</td>
    <td>75.54</td>
    <td>48.91</td>
    <td>40.46</td>
    <td>24.86</td>
    <td>86.54</td>
    <td>82.51</td>
    <td>14.83</td>
    <td>17.44</td>
    <td>57.31</td>
    <td>48.29</td>
  </tr>
  <tr>
    <td>+ CoT</td>
    <td>43.21</td>
    <td>45.05</td>
    <td>52.31</td>
    <td>45.65</td>
    <td>60.58</td>
    <td>42.07</td>
    <td>76.89</td>
    <td>75.16</td>
    <td>64.44</td>
    <td>52.22</td>
    <td>59.49</td>
    <td>52.03</td>
  </tr>
  <tr>
    <td>+ HiTEC-ICL</td>
    <td><b>75.93</b></td>
    <td><b>70.55</b></td>
    <td><b>77.30</b></td>
    <td><b>56.01</b></td>
    <td><b>85.92</b></td>
    <td><b>55.49</b></td>
    <td><b>94.40</b></td>
    <td><b>87.27</b></td>
    <td><b>82.07</b></td>
    <td><b>58.30</b></td>
    <td><b>83.12</b></td>
    <td><b>65.52</b></td>
  </tr>
  <tr>
    <td rowspan="3">Llama3.1-8B</td>
    <td>Vanilla</td>
    <td>73.06</td>
    <td>63.62</td>
    <td>78.23</td>
    <td>50.25</td>
    <td>84.06</td>
    <td>50.79</td>
    <td>94.97</td>
    <td>86.53</td>
    <td><b>87.93</b></td>
    <td>72.17</td>
    <td>83.65</td>
    <td>64.67</td>
  </tr>
  <tr>
    <td>+ CoT</td>
    <td>65.22</td>
    <td>61.61</td>
    <td>75.54</td>
    <td>48.52</td>
    <td>63.31</td>
    <td>42.57</td>
    <td>88.55</td>
    <td>80.36</td>
    <td>76.71</td>
    <td>67.32</td>
    <td>73.87</td>
    <td>60.08</td>
  </tr>
  <tr>
    <td>+ HiTEC-ICL</td>
    <td><b>75.07</b></td>
    <td><b>64.61</b></td>
    <td><b>82.23</b></td>
    <td><b>51.29</b></td>
    <td><b>85.92</b></td>
    <td><b>55.94</b></td>
    <td><b>96.95</b></td>
    <td><b>87.94</b></td>
    <td>87.76</td>
    <td>72.35</td>
    <td><b>85.59</b></td>
    <td><b>66.42</b></td>
  </tr>
</table>

(F1 Name) and correctness of tool names + tool parameters (F1 Name + Parameter).

Baselines. We use GPT-3.5-Turbo-0125, GPT-4-Turbo (GPT series), Llama3-8B, Llama3-70B (Llama3 series), Llama3.1-8B, Llama3.1-70 (Llama3.1-series) models as base models to study the effect of HiTEC-ICL. The vanilla version of the base models, base models deployed with zero-shot CoT (Wei et al., 2022), and native tool calling integrated base models 2 (only GPT-series) are used as baselines. For HiTEC-KTO, we use Llama3-8B, Llama3-70B (Llama3 series), Llama3.1-8B, Llama3.1-70 (Llama3.1-series), Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen2.5-3B and Qwen2.5-7B (Qwen2.5 series) as base models, and consider the vanilla base models of Llama series and Hammer2.0 release 3 (which is tuned on Qwen2.5 series (Lin et al., 2024)) as baselines. The implementation details are in Appendix B.

4.2 Main Results

4.2.1 Effectiveness of HiTEC-ICL
We now evaluate HiTEC’s efficacy across datasets and base models in ICL setting. The method consistently enhances tool-calling performance as shown in Table 2. Experiments under more settings are deferred to Appendix C due to limited space.

HiTEC-ICL demonstrates robust performance improvements across diverse tool-learning benchmarks. Model capability critically influences HiTEC-ICL’s efficacy. Smaller models (Llama3-8B) achieve the most pronounced gains, with Name + Parameter improving by over 30 points on Nexus Raven (58.30 vs. 17.44), indicating that explicit error guidance compensates for limited reasoning capacity. Larger models (GPT-4-Turbo, Llama3.1-70B) exhibit subtler but consistent improvements, leveraging checklists to refine already strong baseline performance. Notably, HiTEC-ICL outperforms CoT and vanilla tool calling in most cases, with the largest margins on Seal-Tools validating its structured error mitigation approach.

4.2.2 Effectiveness of HiTEC-KTO
KTO Training Data. We generate negative samples based on the xlam-function-calling-60k dataset (Liu et al.). One incorrect tool calling answer is generated for each of the queries in the xlam-function-calling-60k dataset. We label the incorrect answer as "False" and the original groundtruth as "True" and perform KTO on the 12,000 samples. Table 3 demonstrates the experiment results over various of open-sourced models. Results under more settings are deferred to Appendix C due to limited space.

It can be found that HiTEC-KTO significantly enhances tool-calling capabilities across open-source models by leveraging error checklists to generate targeted negative examples for preference-based fine-tuning. Crucially, HiTEC-KTO enables smaller models to rival larger counterparts—the 1.5B Qwen2.5 model surpasses the 3B baseline Hammer in many cases after being fine-tuned with HITECH-KTO. Performance trends also correlate with dataset complexity. On multi-tool calling dataset Tool Alpaca, HiTEC-KTO boosts parameter accuracy by 6-25 points on F1 across 8B Llama series, demonstrating its efficacy in resolving tool-specific ambiguities. It also provides 8B Llama series with a 4-10 points performance increase on the Nexus Raven dataset, which has complex and long queries.

2https://platform.openai.com/docs/guides/function-calling
3https://huggingface.co/collections/MadeAgents/hammer-20--66f4dee539f7b2c95224012a
Table 3: Tool Calling Performance with HiTEC-KTO, with the best performance marked in bold and the proposed approach highlighted in blue

<table>
  <tr>
    <th rowspan="2">Model Series</th>
    <th rowspan="2">Method</th>
    <th colspan="4">Dataset (F1 Name | F1 Name + Parameter)</th>
    <th colspan="2">F1 Average</th>
  </tr>
  <tr>
    <th>API-Bank L-1</th>
    <th>API-Bank L-2</th>
    <th>Tool-Alpaca</th>
    <th>Seal-Tools (Single-Tool)</th>
    <th>Nexus Raven</th>
    <th>Name Name + Param.</th>
  </tr>
  <tr>
    <td rowspan="2">Llama3.1 Series</td>
    <td>Baseline</td>
    <td>Llama3.1-8B</td>
    <td>73.06</td>
    <td>63.62</td>
    <td>78.23</td>
    <td>50.25</td>
    <td>84.06</td>
    <td>50.79</td>
    <td>94.97</td>
    <td><b>86.53</b></td>
    <td>87.93</td>
    <td>72.17</td>
    <td>83.65</td>
    <td>64.67</td>
  </tr>
  <tr>
    <td></td>
    <td>Llama3.1-70B</td>
    <td>90.15</td>
    <td>76.42</td>
    <td>80.34</td>
    <td>62.20</td>
    <td>86.03</td>
    <td>53.83</td>
    <td><b>97.42</b></td>
    <td>86.19</td>
    <td>93.75</td>
    <td>82.72</td>
    <td>89.54</td>
    <td>72.67</td>
  </tr>
  <tr>
    <td rowspan="2">HiTEC-KTO</td>
    <td>Llama3.1-8B</td>
    <td><b>87.47</b></td>
    <td><b>80.99</b></td>
    <td><b>85.61</b></td>
    <td><b>61.02</b></td>
    <td><b>84.18</b></td>
    <td><b>56.45</b></td>
    <td><b>94.44</b></td>
    <td>86.18</td>
    <td><b>89.98</b></td>
    <td><b>82.28</b></td>
    <td><b>88.34</b></td>
    <td><b>73.38</b></td>
  </tr>
  <tr>
    <td>Llama3.1-70B</td>
    <td><b>90.78</b></td>
    <td><b>77.07</b></td>
    <td><b>86.44</b></td>
    <td><b>65.14</b></td>
    <td><b>86.67</b></td>
    <td><b>57.32</b></td>
    <td><b>98.14</b></td>
    <td><b>90.01</b></td>
    <td><b>94.84</b></td>
    <td><b>82.87</b></td>
    <td><b>91.37</b></td>
    <td><b>74.48</b></td>
  </tr>
  <tr>
    <td rowspan="2">Qwen2.5 Series</td>
    <td>Hammer2-0.5B</td>
    <td>71.20</td>
    <td>59.03</td>
    <td>43.32</td>
    <td>38.22</td>
    <td>64.46</td>
    <td><b>41.83</b></td>
    <td>93.86</td>
    <td>83.03</td>
    <td>64.72</td>
    <td>45.52</td>
    <td>67.51</td>
    <td>53.53</td>
  </tr>
  <tr>
    <td>Hammer2-1.5B</td>
    <td>88.63</td>
    <td>79.26</td>
    <td>80.51</td>
    <td><b>62.82</b></td>
    <td>80.74</td>
    <td>51.88</td>
    <td>96.10</td>
    <td>87.16</td>
    <td>85.85</td>
    <td>63.76</td>
    <td>86.37</td>
    <td>68.98</td>
  </tr>
  <tr>
    <td rowspan="2"></td>
    <td>Hammer2-3B</td>
    <td>88.63</td>
    <td>79.04</td>
    <td>77.11</td>
    <td><b>57.58</b></td>
    <td>78.23</td>
    <td>53.28</td>
    <td>93.08</td>
    <td>85.60</td>
    <td><b>89.14</b></td>
    <td><b>66.71</b></td>
    <td>85.24</td>
    <td>68.44</td>
  </tr>
  <tr>
    <td>Hammer2-7B</td>
    <td>88.91</td>
    <td>81.28</td>
    <td>75.96</td>
    <td>58.36</td>
    <td>81.74</td>
    <td>57.07</td>
    <td>94.62</td>
    <td>87.84</td>
    <td>90.76</td>
    <td><b>80.96</b></td>
    <td>86.40</td>
    <td>73.10</td>
  </tr>
  <tr>
    <td rowspan="2">HiTEC-KTO</td>
    <td>Qwen2.5-0.5B</td>
    <td><b>82.27</b></td>
    <td><b>78.34</b></td>
    <td><b>81.76</b></td>
    <td><b>54.52</b></td>
    <td><b>73.54</b></td>
    <td>38.76</td>
    <td>96.80</td>
    <td>88.18</td>
    <td>82.83</td>
    <td>63.89</td>
    <td>84.64</td>
    <td>64.74</td>
  </tr>
  <tr>
    <td>Qwen2.5-1.5B</td>
    <td><b>88.92</b></td>
    <td><b>79.46</b></td>
    <td><b>81.36</b></td>
    <td>60.66</td>
    <td>82.27</td>
    <td>52.26</td>
    <td>96.99</td>
    <td>89.75</td>
    <td>86.37</td>
    <td>64.40</td>
    <td>87.18</td>
    <td>69.31</td>
  </tr>
  <tr>
    <td rowspan="2"></td>
    <td>Qwen2.5-3B</td>
    <td><b>89.07</b></td>
    <td><b>80.16</b></td>
    <td><b>85.51</b></td>
    <td>54.61</td>
    <td>85.42</td>
    <td>56.08</td>
    <td>96.28</td>
    <td>89.51</td>
    <td>83.05</td>
    <td>62.35</td>
    <td>87.87</td>
    <td>68.54</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td><b>89.38</b></td>
    <td><b>81.67</b></td>
    <td><b>88.00</b></td>
    <td>59.93</td>
    <td>87.63</td>
    <td>59.19</td>
    <td>96.27</td>
    <td>89.29</td>
    <td>91.70</td>
    <td>78.17</td>
    <td>90.60</td>
    <td>73.65</td>
  </tr>
</table>

Table 4: Ablation Study on the Hierarchical Error Checklist of HiTEC-KTO

<table>
  <tr>
    <th colspan="4">Base Model (F1 Name | F1 Name + Parameter)</th>
  </tr>
  <tr>
    <th>Dataset</th>
    <th>Method</th>
    <th>Qwen2.5 1.5B</th>
    <th>Qwen2.5 3B</th>
    <th>Average</th>
  </tr>
  <tr>
    <td rowspan="2">Tool Alpaca</td>
    <td>HiTEC-KTO</td>
    <td><b>82.27</b></td>
    <td><b>52.26</b></td>
    <td><b>85.42</b></td>
    <td><b>56.08</b></td>
    <td><b>83.30</b></td>
    <td><b>53.30</b></td>
  </tr>
  <tr>
    <td>w/o Local EC</td>
    <td>60.61</td>
    <td>28.97</td>
    <td>77.62</td>
    <td>51.61</td>
    <td>69.71</td>
    <td>41.08</td>
  </tr>
  <tr>
    <td rowspan="2"></td>
    <td>w/o Glb-loc EC</td>
    <td>22.83</td>
    <td>16.03</td>
    <td>73.45</td>
    <td>47.75</td>
    <td>46.38</td>
    <td>31.40</td>
  </tr>
  <tr>
    <td>HiTEC-KTO</td>
    <td><b>96.99</b></td>
    <td><b>89.75</b></td>
    <td><b>96.28</b></td>
    <td><b>89.51</b></td>
    <td><b>96.62</b></td>
    <td><b>89.48</b></td>
  </tr>
  <tr>
    <td rowspan="2">Seal-Tools</td>
    <td>w/o Local EC</td>
    <td>87.75</td>
    <td>68.99</td>
    <td>92.85</td>
    <td>82.60</td>
    <td>90.31</td>
    <td>76.85</td>
  </tr>
  <tr>
    <td>w/o Glb-loc EC</td>
    <td>41.86</td>
    <td>37.34</td>
    <td>83.98</td>
    <td>77.99</td>
    <td>60.01</td>
    <td>55.29</td>
  </tr>
</table>

Figure 5: Analysis on Error Distribution

These results underscore HiTEC-KTO’s ability to democratize advanced tool-learning capabilities across model sizes and dataset complexity, balancing error avoidance with functional precision through structured checklist-driven optimization.

4.3 Ablation Study

To assess the impact of HiTEC’s hierarchical design, we conducted an ablation study comparing configurations that utilize the Global Error Checklist (Glb EC) versus the combined Global-Local Error Checklist (Gble EC) across two datasets—Tool Alpaca and Seal-Tools—as well as multiple base models. The results are presented in Table 4 and Appendix C due to limited space. For HiTEC-KTO variants, w/o Local EC indicates that KTO was performed on negative samples generated solely with the global error checklist, omitting the local error checklist. w/o Glble EC refers to the baseline model without hierarchical error correction.

HiTEC-KTO significantly outperforms w/o Glble EC, particularly in smaller models. Notably, HiTEC Qwen2.5-1.5B achieves performance comparable to its vanilla counterpart (w/o Glb-loc EC). Removing the local error checklist has a substantial negative impact on HiTEC-KTO, as global errors alone capture only general mistakes, whereas generating high-quality negative examples requires tool-specific error instructions.

As shown in the table, incorporating the local error checklist generally improves accuracy for both tool name identification and parameter filling compared to using only the global checklist. This improvement is primarily attributed to more precise parameter filling, as evidenced by the greater increase in the F1 Name + Parameter metric (second column) compared to F1 Name (first column). For instance, on Tool Alpaca with Qwen2.5-1.5B, F1 Name + Parameter relatively improves by 86%, whereas the relative improvement of F1 Name is only 37%. This result highlights the effectiveness of hierarchical checklists in addressing tool-specific local errors.

In conclusion, HiTEC’s hierarchical structure effectively mitigates tool-specific errors, particularly enhancing parameter accuracy in smaller models.
4.4 Look Into the Errors
In this subsection, we further look into errors and conduct error-level evaluations.

4.4.1 Analysis on Error Distribution
We analyze error type distributions across the vanilla base model, HiTEC-ICL, and HiTEC-KTO configurations on Tool Alpaca (Llama3-8B) to quantify the framework’s impact. Since multiple errors can co-exist in a single response, we count all occurrences. The results are shown in 5.

In vanilla setting, parameter-level errors (Error 1–4) and format errors (Error 5) account for the majority of failures. Formatting errors (Error 5), in particular, are prevalent, reflecting baseline weaknesses in syntactical consistency. However, with HiTEC, the percentage of formatting errors drops significantly, demonstrating the global checklist’s effectiveness in enforcing output structure.

Parameter-filling errors (Error 1–4) also undergo a notable redistribution. In the vanilla configuration, Empty Parameter Value Error (Error 3) is the most frequent, but under HiTEC, the distribution becomes more balanced, with no single error type dominating. This indicates that the hierarchical error checks address a broader range of parameter-related issues, reducing reliance on any single mitigation strategy.

4.4.2 Effect of Error Types
To evaluate the contribution of individual error types in the HiTEC framework, we conduct an experiment in which each error type is iteratively excluded from the HiTEC, and the performance of HiTEC-ICL is assessed. The results for HiTEC-ICL-Llama3.1-70B on the Tool Alpaca dataset are presented in Figure 7, while additional results under different settings can be found in Appendix C due to limited space.

As illustrated in Figure 7 (a-b), the exclusion of tool-level errors (e.g., Errors 0, 6, and 7) significantly reduces performance, highlighting their critical role in the multi-tool selection feature of the Tool Alpaca dataset. In contrast, parameter-centric errors primarily affect the Seal-Tools dataset (see Appendix C). Notably, the removal of Errors 3 and 4 results in a substantial performance decline across multiple base models. Furthermore, Error 5 is found to be universally essential, as its presence ensures the generation of parsable outputs, which are crucial for accurate tool calling.

Additionally, we examine changes in the number of errors (# errors) when specific error types are removed. Figure 7 (c) demonstrates that the number of errors in categories 0-7 increases when the corresponding error type is omitted from the checklist. This observation confirms that a defined error in HiTEC can help reduce such a type of error. The symbol * denotes other types of errors, while "Avg w/o Error" represents the average number of errors within the "Others" category of the w/o variants. HiTEC-ICL exhibits a slight increase in errors within the "Others" category compared to its variants, which is a reasonable outcome given that the removed errors may contribute to new, previously unclassified errors.

4.5 Cost Analysis
In this section, we further conduct an evaluation of HiTEC-ICL’s inference overhead. Specifically, we measure the number of prompt tokens and generated tokens on the Tool-Alpaca dataset, comparing HiTEC-ICL against the vanilla model, CoT, and HiTEC-ICL with only a global checklist. Additionally, we assessed the cost per query in US dollars for GPT-based models and execution time for Llama-based models. All experiments were conducted on four NVIDIA A100 GPUs. The results are shown in Table 5 and Table 6.

The results indicate that while HiTEC-ICL incurs the highest cost, the absolute expense remains within a reasonable range. Notably, the substantial performance gains observed on Llama-based models justify the increase in computational cost.
Table 5: Cost Analysis of HiTEC-ICL on Closed-sourced Models

<table>
  <tr>
    <th>Model</th>
    <th>Method</th>
    <th>Prompt Tokens</th>
    <th>Generated Tokens</th>
    <th>Cost per Case ($)</th>
    <th>F1 Name + Param.</th>
  </tr>
  <tr>
    <td rowspan="4">GPT-3.5-Turbo-0125</td>
    <td>Vanilla</td>
    <td>77,374</td>
    <td>3,759</td>
    <td>0.0004</td>
    <td>56.47</td>
  </tr>
  <tr>
    <td>CoT</td>
    <td>82,048</td>
    <td>15,401</td>
    <td>0.0006</td>
    <td>58.43</td>
  </tr>
  <tr>
    <td>HiTEC-ICL (w/o local)</td>
    <td>91,510</td>
    <td>3,747</td>
    <td>0.0005</td>
    <td>57.98</td>
  </tr>
  <tr>
    <td>HiTEC-ICL</td>
    <td>320,830</td>
    <td>7,128</td>
    <td>0.0015</td>
    <td>58.79</td>
  </tr>
  <tr>
    <td rowspan="4">GPT-4-Turbo</td>
    <td>Vanilla</td>
    <td>77,374</td>
    <td>3,678</td>
    <td>0.0078</td>
    <td>60.75</td>
  </tr>
  <tr>
    <td>CoT</td>
    <td>81,706</td>
    <td>25,429</td>
    <td>0.0127</td>
    <td>56.46</td>
  </tr>
  <tr>
    <td>HiTEC-ICL (w/o local)</td>
    <td>103,252</td>
    <td>3,680</td>
    <td>0.0100</td>
    <td>60.67</td>
  </tr>
  <tr>
    <td>HiTEC-ICL</td>
    <td>321,816</td>
    <td>7,344</td>
    <td>0.0302</td>
    <td>61.95</td>
  </tr>
</table>

Table 6: Cost Analysis of HiTEC-ICL on Open-sourced Models

<table>
  <tr>
    <th>Model</th>
    <th>Method</th>
    <th>Prompt Tokens</th>
    <th>Generated Tokens</th>
    <th>Cost per Case (second)</th>
    <th>F1 Name + Param.</th>
  </tr>
  <tr>
    <td rowspan="4">Llama3-8B</td>
    <td>Vanilla</td>
    <td>81,260</td>
    <td>3,786</td>
    <td>0.0360</td>
    <td>24.86</td>
  </tr>
  <tr>
    <td>CoT</td>
    <td>102,997</td>
    <td>21,191</td>
    <td>0.0701</td>
    <td>42.07</td>
  </tr>
  <tr>
    <td>HiTEC-ICL (w/o local)</td>
    <td>107,005</td>
    <td>3,653</td>
    <td>0.0454</td>
    <td>52.39</td>
  </tr>
  <tr>
    <td>HiTEC-ICL</td>
    <td>323,392</td>
    <td>7,532</td>
    <td>0.1320</td>
    <td>55.49</td>
  </tr>
  <tr>
    <td rowspan="4">Llama3.1-8B</td>
    <td>Vanilla</td>
    <td>81,670</td>
    <td>4,196</td>
    <td>0.0368</td>
    <td>50.79</td>
  </tr>
  <tr>
    <td>CoT</td>
    <td>108,503</td>
    <td>26,355</td>
    <td>0.0799</td>
    <td>42.57</td>
  </tr>
  <tr>
    <td>HiTEC-ICL (w/o local)</td>
    <td>107,534</td>
    <td>4,182</td>
    <td>0.0464</td>
    <td>52.33</td>
  </tr>
  <tr>
    <td>HiTEC-ICL</td>
    <td>331,671</td>
    <td>8,800</td>
    <td>0.1370</td>
    <td>55.94</td>
  </tr>
</table>

Moreover, HiTEC-ICL (w/o local) strikes a strong balance between efficiency and effectiveness, offering lower costs while maintaining superior performance compared to the baselines.

4.6 Study on Multi-turn/step Tool Calling
To further evaluate HiTEC’s capability in handling interactive tool use, we design experiments to simulate multi-turn/step tool calling. In this setup, the responses from previous tool calls are treated as predefined contextual information for subsequent calls. We incorporate such intermediate responses into artificial conversations within our proposed framework, prompting the model to generate the next API call accordingly.

To ensure a comprehensive evaluation, we filtered multi-turn/step tool queries from the Seal-Tools dataset (Wu et al., 2024) and conducted experiments on HiTEC-ICL and HiTEC-KTO. The results, presented in Table 7 and 8, demonstrate that while multi-turn tool use poses greater challenges, our approach consistently outperforms the baselines in most cases, highlighting its effectiveness in iterative tool interactions.

Table 7: Tool Calling Performance with HiTEC-ICL on Seal-Tools Multi-turn/step

<table>
  <tr>
    <th>Method</th>
    <th>Model</th>
    <th>F1 Name</th>
    <th>F1 Name + Param.</th>
  </tr>
  <tr>
    <td rowspan="3">Llama 3-8B</td>
    <td>Vanilla</td>
    <td>8.24</td>
    <td>0.00</td>
  </tr>
  <tr>
    <td>+CoT</td>
    <td>8.42</td>
    <td>5.40</td>
  </tr>
  <tr>
    <td>+HiTEC-ICL</td>
    <td>43.01</td>
    <td>10.21</td>
  </tr>
  <tr>
    <td rowspan="3">Llama3.1-8B</td>
    <td>Vanilla</td>
    <td>47.51</td>
    <td>10.77</td>
  </tr>
  <tr>
    <td>+CoT</td>
    <td>43.66</td>
    <td>9.86</td>
  </tr>
  <tr>
    <td>+HiTEC-ICL</td>
    <td>57.14</td>
    <td>15.87</td>
  </tr>
</table>

Table 8: Tool Calling Performance with HiTEC-KTO on Seal-Tools Multi-turn/step

<table>
  <tr>
    <th>Method</th>
    <th>Model</th>
    <th>F1 Name</th>
    <th>F1 Name + Param.</th>
  </tr>
  <tr>
    <td rowspan="2">Baseline</td>
    <td>Llama3.1-8B</td>
    <td>47.51</td>
    <td>10.77</td>
  </tr>
  <tr>
    <td>Llama3.1-70B</td>
    <td>51.23</td>
    <td>16.76</td>
  </tr>
  <tr>
    <td rowspan="2">HiTEC-KTO</td>
    <td>Llama3.1-8B</td>
    <td>67.15</td>
    <td>14.09</td>
  </tr>
  <tr>
    <td>Llama3.1-70B</td>
    <td>70.31</td>
    <td>18.81</td>
  </tr>
  <tr>
    <td rowspan="3">Baseline</td>
    <td>Hammer2-0.5B</td>
    <td>31.78</td>
    <td>4.65</td>
  </tr>
  <tr>
    <td>Hammer2-1.5B</td>
    <td>47.70</td>
    <td>9.41</td>
  </tr>
  <tr>
    <td>Hammer2-3B</td>
    <td>32.09</td>
    <td>5.55</td>
  </tr>
  <tr>
    <td>Hammer2-7B</td>
    <td>43.03</td>
    <td>17.24</td>
  </tr>
  <tr>
    <td rowspan="3">HiTEC-KTO</td>
    <td>Qwen2.5-0.5B</td>
    <td>35.95</td>
    <td>8.35</td>
  </tr>
  <tr>
    <td>Qwen2.5-1.5B</td>
    <td>44.70</td>
    <td>11.15</td>
  </tr>
  <tr>
    <td>Qwen2.5-3B</td>
    <td>54.67</td>
    <td>14.00</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td>58.15</td>
    <td>14.65</td>
  </tr>
</table>

hance tool calling in large language models by systematically identifying and addressing common and tool-specific errors. By integrating both global and local error checklists, HiTEC effectively mitigates parameter mis-filling and format inconsistencies. We proposed two complementary approaches—HiTEC-ICL and HiTEC-KTO—to incorporate error feedback into the tool-calling process. HiTEC-ICL leverages structured error guidance during prompt formulation, while HiTEC-KTO utilizes targeted negative examples to fine-tune open-source models. Extensive experiments on multiple benchmark datasets demonstrate that both methods significantly improve tool name identification and parameter accuracy compared to existing baselines, with particularly notable gains in smaller models.

Acknowledgments
The research work described in this paper was supported by Hong Kong Research Grants Council (grant# 16202722, T22-607/24-N, T43-513/23N-1). It was partially conducted in JC STEM Lab of Data Science Foundations funded by The Hong Kong Jockey Club Charities Trust.
Limitation

While our approach shows promising improvements, it has some limitations. The effectiveness of HiTEC partially attributes to the comprehensiveness of the manually crafted error types and templates, which may not capture all novel or unforeseen errors in dynamic tool environments. Additionally, the reliance on simulated error feedback may not fully reflect real-world scenarios, potentially limiting the framework’s generalizability and scalability in diverse applications.

To address these limitations, potential improvements could focus on dynamically updating the error checklists based on real-world feedback. By integrating mechanisms for continuous monitoring and analysis of actual tool interactions, the system could iteratively refine the types of errors included in the checklists. This iterative improvement would allow the framework to adapt to evolving tool behaviors and emerging error patterns, ultimately enhancing its robustness and precision in tool calling. Results in Section 4.4.2 can also verify effectiveness of such an extension. Additionally, incorporating automated error detection and leveraging reinforcement or continual learning strategies may further optimize the system’s performance in diverse and dynamic environments.

References

Sijia Chen, Yibo Wang, Yi-Feng Wu, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Lijun Zhang. 2024a. Advancing tool-augmented large language models: Integrating insights from errors in inference trees. arXiv preprint arXiv:2406.07115.

Yanfei Chen, Jinsung Yoon, Devendra Singh Sachan, Qingze Wang, Vincent Cohen-Addad, Mohammadhossein Bateni, Chen-Yu Lee, and Tomas Pfister. 2024b. Re-involve: Tool invocation rewriting for zero-shot tool retrieval. arXiv preprint arXiv:2408.01875.

Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306.

Duanyu Feng, Bowen Qin, Chen Huang, Zheng Zhang, and Wenqiang Lei. 2024. Towards analyzing and understanding the limitations of dpo: A theoretical perspective. arXiv preprint arXiv:2404.04626.

Jiayi Fu, Xuandong Zhao, Chengyuan Yao, Heng Wang, Qi Han, and Yanghua Xiao. 2025. Reward shaping to mitigate reward hacking in rlhf. arXiv preprint arXiv:2502.18770.

Leo Gao, John Schulman, and Jacob Hilton. 2023. Scaling laws for reward model overoptimization. In International Conference on Machine Learning, pages 10835–10866. PMLR.

Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738.

Zhicheng Guo, Sijie Cheng, Hao Wang, Shihao Liang, Yujia Qin, Peng Li, Zhiyuan Liu, Maosong Sun, and Yang Liu. 2024. Stabletoolbench: Towards stable large-scale benchmarking on tool learning of large language models. arXiv preprint arXiv:2403.07714.

Shibo Hao, Tianyang Liu, Zhen Wang, and Zhiting Hu. 2023. Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings. Advances in neural information processing systems, 36:45870–45894.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.

Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. Api-bank: A comprehensive benchmark for tool-augmented llms. arXiv preprint arXiv:2304.08244.

Qiqiang Lin, Muning Wen, Qiuying Peng, Guanyu Nie, Junwei Liao, Jun Wang, Xiaoyun Mo, Jiamu Zhou, Cheng Cheng, Yin Zhao, et al. 2024. Hammer: Robust function-calling for on-device language models via function masking. arXiv preprint arXiv:2410.04587.

Yanming Liu, Xinyue Peng, Yuwei Zhang, Jiannan Cao, Xuhong Zhang, Sheng Cheng, Xun Wang, Jianwei Yin, and Tianyu Du. 2024. Tool-planner: Dynamic solution tree planning for large language model with tool clustering. arXiv preprint arXiv:2406.03807.

Zuxin Liu, Thai Quoc Hoang, Jiangou Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, et al. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track.

Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, and Colin White. 2024. Smaug: Fixing failure modes of preference optimisation with dpo-positive. arXiv preprint arXiv:2402.13228.

Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. In The
Twelfth International Conference on Learning Representations.

Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2025. Tool learning with large language models: A survey. Frontiers of Computer Science, 19(8):198343.

Rafael Rafaïlov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36.

Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, and Zhaochun Ren. 2024. Learning to use tools via cooperative and interactive agents. arXiv preprint arXiv:2403.03031.

Yifan Song, Weimin Xiong, Dawei Zhu, Wenhao Wu, Han Qian, Mingbo Song, Hailiang Huang, Cheng Li, Ke Wang, Rong Yao, et al. 2023. Restgpt: Connecting large language models with real-world restful apis. arXiv preprint arXiv:2306.06624.

Venkat Krishna Srinivasan, Zhen Dong, Banghua Zhu, Brian Yu, Damon Mosk-Aoyama, Kurt Keutzer, Jiantao Jiao, and Jian Zhang. 2023. Nexusraven: a commercially-permissive language model for function calling. In NeurIPS 2023 Foundation Models for Decision Making Workshop.

Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, Boxi Cao, and Le Sun. 2023. Toolalpaca: Generalized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv:2306.05301.

Boshi Wang, Hao Fang, Jason Eisner, Benjamin Van Durme, and Yu Su. 2024. Llms in the imaginariun: tool learning through simulated trial and error. arXiv preprint arXiv:2403.04746.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Mengsong Wu, Tong Zhu, Han Han, Chuanyuan Tan, Xiang Zhang, and Wenliang Chen. 2024. Seal-tools: self-instruct tool learning dataset for agent tuning and detailed benchmark. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 372–384. Springer.

Hongshen Xu, Zichen Zhu, Lei Pan, Zihan Wang, Su Zhu, Da Ma, Ruisheng Cao, Lu Chen, and Kai Yu. 2024. Reducing tool hallucination via reliability alignment. arXiv preprint arXiv:2412.04141.

Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. 2024. Gpt4tools: Teaching large language model to use tools via self-instruction.

Advances in Neural Information Processing Systems, 36.

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629.

Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Kan Ren, Dongsheng Li, and Deqing Yang. Easytool: Enhancing llm-based agents with concise tool instruction. In ICLR 2024 Workshop on Large Language Model (LLM) Agents.

Yinger Zhang, Hui Cai, Xeirui Song, Yicheng Chen, Rui Sun, and Jing Zheng. 2023. Reverse chain: A generic-rule for llms to master multi-api planning. arXiv preprint arXiv:2310.04474.
A  Prompt

Local Error Checklist Generation Prompt

Task:
You are given information about a tool and an example template of an error checklist. Your task is to generate an error checklist for the tool in the same format as the template. More specifically, for each error, you should:
- Provide a **perfect query**. The query should be self-contained and contain all the necessary information for a correct tool call. For example: "Can you verify the access to the database named 'customer_data'?"
- Provide the **corresponding answer** from the model that evokes the error.
- Provide an **error message** that describes what went wrong.
- Provide a **thought** explaining how the error should be corrected.

Note: You should strictly follow the format of the template.

**Error Checklist Template**
Tool Information
name: 'name_of_the_tool'
description: 'description_of_the_tool'
parameters:  {"parameter_name_1": { "type": "type_1", "description": "description_of_the_parameter"}, "parameter_2": { "type": "type_2", "description": "description_of_the_parameter"}} required parameters: ["parameter_1"] (Include other relevant information about the tool if necessary.)

—
Error 2: Missing Required Parameter Error
Query: "a_query_that_calls_the_tool"
Function Calling Output: [ {{"name": "name_of_the_tool","arguments": {{"parameter_2":"parameter_value"}}}} ]
Error Message: {{"error": "MissingRequiredParameter","message": "The 'parameter_1' parameter is required."}}
Thought of Error: Parameter 'parameter_1' is missing. Ensure all required parameters ('parameter_1') are included in the function call.

—
Error 3: Invalid Parameter Type Error
Query: "a_query_that_calls_the_tool"
Function Calling Output: [ { { "name": "name_of_the_tool", "arguments": { { "parameter_1": "parameter_value","parameter_2": "parameter_value (but not of type_2)" } } } ]
Error Message: { { "error": "InvalidParameterType", "message": "The 'parameter_2' is not of 'type_2'." }}
Thought of Error: Parameter 'parameter_2' should be of type 'type_2', but an invalid type was provided. Ensure all parameters match their expected types.

—
Error 4: Empty Parameter Value Error
Query: "a_query_that_calls_the_tool"
Function Calling Output: [ { { "name": "name_of_the_tool", "arguments": { { "parameter_1": "parameter_value","parameter_2": "" } } } ]
Error Message: { "error": "EmptyParameterValue", "message": "The 'parameter_2' parameter cannot be empty." }}
Thought of Error: Parameter 'parameter_2' has an empty value. It should not be empty as specified by the tool’s requirements.
—
Error 5: Redundant Parameter Error
Query: "a_query_that_calls_the_tool (that only needs to fill in part of the parameters of the tool)"
Function Calling Output: [ { { "name": "name_of_the_tool", "arguments": { { "parameter_1": "parameter_value","parameter_2": "parameter_value" } } } } ]
Error Message: { { "error": "RedundantParameter", "message": "The parameter 'parameter_2' is not indicated by the query and should not be called." } }
Thought of Error: Parameter 'parameter_2' is unnecessary and was not specified in the query. Ensure only the required and specified parameters are included in the function call.
—
Error 6: Invalid Function Calling Output Format Error
Query: "a_query_that_calls_the_tool"
Function Calling Output: { { "Name": "name_of_the_tool", "Parameter": { { "parameter_1": "parameter_value", "parameter_2": "parameter_value" } } } }
Error Message: { { "error": "InvalidFormat", "message": "The function calling output does not follow the required format and cannot be parsed." } }
Thought of Error: The output format is incorrect due to improperly formatted keys and symbols. The correct function calling output should be: [ { { "name": "func_name1", "arguments": { { "parameter_1": "value1", "parameter_2": "value2" } } } ] }
—
Error 7: Redundant Information Error
Query: "a_query_that_calls_the_tool"
Function Calling Output: "Based on the query, I will make a function call to the 'name_of_the_tool' tool to get the query answered. Here is the output in the required JSON format: [ { { 'name': 'name_of_the_tool', 'arguments': { { 'parameter_1': 'parameter_value', 'parameter_2': 'parameter_value' } } } } ] "
Error Message: "error": "RedundantInformationError", "message": "The function calling output contains redundant text such as 'Based on the query, I will make a function call...' which is unnecessary."
Thought of Error: No additional text should be included in the output. The correct function calling output should only contain: [ { { "name": "func_name1", "arguments": { { "parameter_1": "value1", "parameter_2": "value2" } } } ] ]
____
Instructions
Now, generate an error checklist for the following tool:
<tool_info>
Note: You must strictly follow the format of the template.
Negative Sample Generation Prompt

System Prompt: You are provided with an error checklist, a tool calling query and its groundtruth answer.
The error checklist of an example tool is as follows:
**Error Checklist Template**
Tool Information
name: 'name_of_the_tool'
(the same as the tool template in Local Error Checklist Generation Prompt)
—
Your task is to modify the groundtruth tool calling so that it fits one of the errors in the error checklist. For the Redundant Parameter Error, your generated redundant parameter should be one of the parameters in the tool information. If there is no extra parameter that can be chosen for Redundant Parameter Error, you can choose another errors.
#### Note: DO NOT include not-exist parameters in your response, e.g., "extra_param".
#### Note: You should return a modified response, for example: [{"name": "getSocialEnterpriseInfo", "arguments": {"enterprise_name": "CommunityGrowth"}}].
#### Note: Just provide the modified function calling output. DO NOT include other information".
User Prompt: The user query is:
<user_query>
The groundtruth tool calling is:
<groundtruth>
Now please modify the groundtruth tool calling so that it meets one of the errors in the error checklist. Just return the modified tool calling. Do not explain your answer or include any other information.

B Implementation Details

The temperature for GPT-series base models is set as 0.2. The max_new_token is set as 512 when CoT is implemented, otherwise 256 for Llama-series base models. For other base models and parameters, we use the default setting as stated in the models' configuration files. The local error checklists and negative samples are generated using Qwen2.5-72B-Instruct. For the fine-tuning of Llama3-7B, Llama3.1-7B, Qwen2.5 series base models, we perform full-parameter tuning, while for Llama3-70B and Llama3.1-70B we perform LoRA (Hu et al., 2021).

C Additional Experiment Results

C.1 Additional Experiment Results on the Effectiveness of HiTEC-ICL

Table 9 presents the full results of experiments of tool calling performance with HiTEC-ICL.

C.2 Additional Experiment Results on the Effectiveness of HiTEC-KTO

Table 10 presents the full results of experiments of tool calling performance with HiTEC-KTO.

C.3 Additional Experiment Results on Ablation Study on The Hierarchical Error Checklist

Table 11 and 12 present the full results of ablation study on the hierarchical error checklist.

C.4 Additional Experiment Results on Ablation Study on Error Types

Figure 7 illustrates the full results on ablation study on error types.
Table 9: Tool Calling Performance with HiTEC-ICL

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th rowspan="2">Method</th>
    <th colspan="3">Dataset (F1 Name | F1 Name + Parameter)</th>
    <th colspan="4">F1 Average</th>
  </tr>
  <tr>
    <th>API-Bank L-1</th>
    <th>API-Bank L-2</th>
    <th>Tool-Alpaca</th>
    <th>Seal-Tools (Single-Tool)/Nexus Raven</th>
    <th>Name</th>
    <th>Name+Param.</th>
  </tr>
  <tr>
    <td rowspan="5">GPT-3.5-Turbo-0125</td>
    <td>Vanilla</td>
    <td colspan="6">85.26 **77.06 78.87 61.02 86.23 56.47 94.97 87.43 91.84 91.27</td>
    <td>87.35 72.65</td>
  </tr>
  <tr>
    <td>+ CoT</td>
    <td colspan="6">65.28 59.13 59.2 55.3 57.6 54.8 59.16 88.01 83.92 75.69</td>
    <td>74.75 65.90</td>
  </tr>
  <tr>
    <td>+ Function Calling</td>
    <td>85.57 82.47 88.72 65.87 80.00 72.73 96.10 88.50\n / </td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>74.75 65.90</td>
  </tr>
  <tr>
    <td>+ HiTEC-ICL</td>
    <td>92.38 **84.71 83.80 64.72 87.27 58.79 96.91 90.21 91.22 81.08</td>
    <td></td><td></td><td></td><td></td><td></td><td></td><td></td>
    <td>90.46 **75.90</td>
  </tr>
  <tr>
    <td>GPT-4-Turbo</td>
    <td>Vanilla</td>
    <td>94.77 86.66 71.09 70.43 87.12 60.75 94.86 88.95 94.13 82.92</td>
    <td colspan="6">0.8839 77.86</td>
    <td>75.6 89.04</td>
  </tr>
  <tr>
    <td colspan="2">+ CoT</td>
    <td>91.70 84.84 67.86 65.74 80.95 56.46 91.50 86.07 92.18 85.00</td>
    <td colspan="6"></td>
    <td>84.84 75.62</td>
  </tr>
  <tr>
    <td colspan="2">+ Function Calling</td>
    <td>84.03 81.00 78.49 63.22 81.81 62.16 84.94 84.88\n / </td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>/</td>
    <td>84.84 75.62</td>
  </tr>
  <tr>
    <td colspan="2">+ HiTEC-ICL</td>
    <td>94.96 **88.48 71.21 72.68 87.35 61.95 96.00 89.37 92.81 83.15</td>
    <td></td><td></td><td></td><td></td><td></td><td></td>\</tr>
    <tr>
      <td>Llama3-8B</td>
      <td>Vanilla</td>
    </tr>
    <tr>
      <td colspan="2">Llama3-70B</td>
      <td>Vanilla</td>
    </tr>
    <tr>
      <td colspan="2">HiTEC-ICL</td>
      <td>+ Llama3-8B</td>
      <td>75.93 **70.55 71.80 56.81 58.92 55.49 94.40 87.27 82.07 58.30</td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td>85.32 65.52</td>
    </tr>
    <tr>
      <td colspan="2">HiTEC-ICL</td>
      <td>+ Llama3-70B</td>
      <td>86.47 **78.87 66.67 63.63 59.46 47.73 88.00 **86.81 75.43 73.25</td>
      <td>73.24 70.63</td>
    </tr>
    <tr>
      <td colspan="2">Named Tesla-ICL</td>
      <td>+ Llama3-8B</td>
      <td>84.85 87.97 79.05 70.85 86.76 58.62 95.55 86.73 85.40 74.95</td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td>86.32 72.63</td>
    </tr>
    <tr>
      <td colspan="2">Named Tesla-ICL</td>
      <td>+ Llama3-70B</td>
      <td>73.06 63.62 78.23 50.25 84.06 50.79 94.97 86.84 87.57 72.17</td>
      <td>83.65 64.67</td>
    </tr>
    <tr>
      <td colspan="2">Allocated Tesla-ICL</td>
      <td>+ Llama3-8B</td>
      <td>50.77 44.61 82.25 51.29 85.94 65.79 96.49 87.94 87.76 72.33</td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td>85.59 64.42</td>
    </tr>
    <tr>
      <td colspan="2">Allocated Tesla-ICL</td>
      <td>+ Llama3-70B</td>
      <td>90.15 76.64 80.34 62.06 80.35 55.83 95.16 85.89 91.68 78.73</td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td>87.61 70.68</td>
    </tr>
    <tr>
      <td colspan="2">Miscellaneous Tesla-ICL</td>
      <td>+ Llama3-8B</td>
      <td>93.21 **79.05 80.18 64.50 84.96 56.83 95.86 87.19 95.44 83.44</td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td></td>
      <td>89.75 74.20</td>
    </tr>
</table>

Table 10: Tool Calling Performance with HiTEC-KTO

<table>
  <tr>
    <th rowspan="2">Model Series</th>
    <th rowspan="2">Method</th>
    <th colspan="3">Dataset (F1 Name | F1 Name + Parameter)</th>
    <th colspan="4">F1 Average</th>
  </tr>
  <tr>
    <th>API-Bank L-1</th>
    <th>API-Bank L-2</th>
    <th>Tool-Alpaca</th>
    <th>Seal-Tools (Single-Tool)/Nexus Raven</th>
    <th>Name</th>
    <th>Name + Param.</th>
  </tr>
  <tr>
    <td rowspan="3">Llama3 Series</td>
    <td>Baseline</td>
    <td>69.19 67.73 75.48 58.91 94.06 40.46 86.45 82.51 14.83 17.44</td>
    <td></td>
    <td></td>
    <td></td>
    <td>57.31 48.29</td>
    <td></td>
  </tr>
  <tr>
    <td>Llama3-70B</td>
    <td>86.47 **78.87 66.67 **63.65 79.46 84.97 86.81 75.43 73.25</td>
    <td></td>
    <td></td>
    <td></td>
    <td>73.24 70.06</td>
    <td></td>
  </tr>
  <tr>
    <td>HiTEC-KTO<br>Llama3-70B</td>
    <td>86.88 **79.71 75.61 86.04 91.17 68.03 86.47 88.37 91.85 *81.29 *</td>
    <td></td>
    <td></td>
    <td></td>
    <td>s8s0 84.00</td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="3">Llama3.1 Series</td>
    <td>Baseline</td>
    <td>73.06 63.62 78.23 50.25 84.06 50.79 94.97 86.84 87.93 72.17</td>
    <td></td>
    <td></td>
    <td></td>
    <td>83.65 64.67</td>
    <td></td>
  </tr>
  <tr>
    <td>Llama3.1-70B</td>
    <td>90.15 76.64 80.34 62.06 80.35 55.83 95.16 85.89 91.68 78.73</td>
    <td></td>
    <td></td>
    <td></td>
    <td>87.61 70.68</td>
    <td></td>
  </tr>
  <tr>
    <td>HiTEC-KTO<br>Llama3-70B</td>
    <td>90.78 77.07 86.44 65.14 86.67 57.32 98.14 90.01 94.84 82.87</td>
    <td></td>
    <td></td>
    <td></td>
    <td>91.37 74.48</td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="5">Qwen2.5 Series</td>
    <td>Hammer2-0.5B<br>Hammer2-1.5B<br>Hammer2-3B<br>Hammer2-7B</td>
    <td>71.20 59.03 43.32 38.22 51.46 41.83 93.46 98.23 45.52 57.51</td>
    <td></td>
    <td></td>
    <td></td>
    <td>69.50 55.39</td>
    <td></td>
  </tr>
  <tr>
    <td>Baselme</td>
    <td>88.63 79.26 80.51 62.82 64.87 78.18 96.10 87.16 85.53 67.36</td>
    <td></td>
    <td></td>
    <td></td>
    <td>86.71 81.58</td>
    <td></td>
  </tr>
  <tr>
    <td>HiTEC-KTO</td>
    <td>88.63 **79.24 77.11 57.18 78.23 53.88 93.10 85.95 **89.14 **66.91 86.84 </td>
    <td></td>
    <td></td>
    <td></td>
    <td>85.24 84.88 </td>
    <td></td>
  </tr>
  <tr>
    <td>Qwen2.5-0.5B</td>
    <td>82.89 **78.38 71.16 **54.23 75.34 38.76 96.80 88.16 92.83 63.88</td>
    <td>64.44 74.74</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>HiTEC-KTO<br>Qwen2.5-1.5B<br>Qwen2.5-3B<br>Qwen2.5-7B</td>
    <td>89.07 80.16 85.51 54.61 85.42 56.08 96.28 89.51 83.05 67.10 80.40 98.26</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td>81.78 80.54 78.87 73.68</td>
    <td></td>
  </tr>
</table>

Table 11: Ablation Study on The Hierarchical Error Checklist of HiTEC-ICL

<table>
  <tr>
    <th rowspan="2">Dataset</th>
    <th rowspan="2">Method</th>
    <th colspan="4">Base Model (F1 Name | F1 Name + Parameter)</th>
    <th colspan="2">F1 Average</th>
  </tr>
  <tr>
    <th>GPT-3.5-Turbo-0125<br>HTEC-ICL w/o Local EC<br>w/o Glb-loc EC</th>
    <th>GPT-4-Turbo<br>HTEC-ICL w/o Local EC w/o Glb-loc EC</th>
    <th>Llama3-8B<br>HTEC-ICL w/o Local EC w/o Glb-loc EC</th>
    <th>Llama3.1-8B<br>HTEC-ICL w/o Local EC w/o Glb-loc EC</th>
    <th>Name Param.</th>
  </tr>
  <tr>
    <td>Tool Alpaca</td>
    <td>HTEC-ICL<br>w/o Local EC<br>w/o Glb-loc EC</td>
    <td>**109.80 51.11 **48.75 49.00 **95.21 96.06 **78.99 87.40 **97.88 94.76 55.94 88.20 58.77</td>
    <td>86.22</td>
    <td>55.48 86.40</td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Seal-Tools</td>
    <td>HTEC-ICL<br>w/o Local EC<br>w/o Glb-loc EC</td>
    <td>**97.61 20.21 **90.90 23.30 <87.40 87.27 **96.95 89.79 92.34 88.70 87.87</td>
    <td>95.81 68.91</td>
    <td>98.4 68.26</td>
    <td></td>
    <td></td>
  </tr>
</table>
Table 12: Ablation Study on The Hierarchical Error Checklist of HiTEC-KTO

<table>
  <tr>
    <th rowspan="2">Dataset</th>
    <th rowspan="2">Method</th>
    <th colspan="5">Base Model (F1 Name | F1 Name + Parameter)</th>
    <th colspan="2">F1 Average</th>
  </tr>
  <tr>
    <th>Qwen2.5-0.5B</th>
    <th>Qwen2.5-1.5B</th>
    <th>Qwen2.5-3B</th>
    <th>Qwen2.5-7B</th>
    <th>Name</th>
    <th>Name+Param.</th>
  </tr>
  <tr>
    <td rowspan="3">Tool Alpaca</td>
    <td>HiTEC-KTO</td>
    <td><b>73.54</b></td>
    <td><b>38.76</b></td>
    <td><b>82.27</b></td>
    <td><b>52.26</b></td>
    <td><b>85.42</b></td>
    <td><b>56.08</b></td>
    <td><b>87.63</b></td>
    <td><b>59.19</b></td>
    <td><b>82.22</b></td>
    <td><b>51.57</b></td>
  </tr>
  <tr>
    <td>w/o Local EC</td>
    <td>57.51</td>
    <td>31.71</td>
    <td>60.61</td>
    <td>28.97</td>
    <td>77.62</td>
    <td>51.61</td>
    <td><b>87.86</b></td>
    <td>58.40</td>
    <td>70.90</td>
    <td>42.67</td>
  </tr>
  <tr>
    <td>w/o Glb-loc EC</td>
    <td>0.79</td>
    <td>0.83</td>
    <td>22.83</td>
    <td>16.03</td>
    <td>73.45</td>
    <td>47.75</td>
    <td>74.35</td>
    <td>57.12</td>
    <td>42.86</td>
    <td>30.43</td>
  </tr>
  <tr>
    <td rowspan="3">Seal-Tools</td>
    <td>HiTEC-KTO</td>
    <td><b>96.80</b></td>
    <td><b>88.18</b></td>
    <td><b>96.99</b></td>
    <td><b>89.75</b></td>
    <td><b>96.28</b></td>
    <td><b>89.51</b></td>
    <td><b>96.27</b></td>
    <td><b>89.29</b></td>
    <td><b>96.59</b></td>
    <td><b>89.18</b></td>
  </tr>
  <tr>
    <td>w/o Local EC</td>
    <td>86.47</td>
    <td>77.57</td>
    <td>87.75</td>
    <td>68.99</td>
    <td>92.75</td>
    <td>82.60</td>
    <td>94.17</td>
    <td>86.66</td>
    <td>90.44</td>
    <td>78.96</td>
  </tr>
  <tr>
    <td>w/o Glb-loc EC</td>
    <td>1.06</td>
    <td>1.90</td>
    <td>41.86</td>
    <td>37.34</td>
    <td>83.98</td>
    <td>77.99</td>
    <td>89.80</td>
    <td>84.87</td>
    <td>54.18</td>
    <td>50.53</td>
  </tr>
</table>

(a) Tool Alpaca - F1 Name
(b) Tool Alpaca - F1 Name + Parameter
(c) Seal-Tools - F1 Name
(d) Seal-Tools - F1 Name + Parameter

Figure 7: Ablation Study on Error Types

Example: Small Difference between Positive and Negative Answer

"user": "You are a tool calling assistant. In order to complete the user's request, you need to select one or more appropriate tools from the following tools and fill in the correct values for the tool parameters. Your specific tasks are:
1. Make one or more function/tool calls to meet the request based on the question.
2. If none of the function can be used, point it out and refuse to answer. 3. If the given question lacks the parameters required by the function, also point it out.

[{"name": "polygon_area_shoelace", "description": "Calculates the area of a polygon using the shoelace formula.", "parameters": {"vertices": {"description": "A list of polygon vertices represented as tuples (x, y).", "type": "List[Tuple[float, float]]"}}}, {"name": "find_n_largest_numbers", "description": "Finds the n largest numbers in a list.", "parameters": {"nums": {"description": "The list of numbers.", "type": "List[int]"}, "n": {"description": "The number of largest numbers to find.", "type": "int"}}}]

The output MUST strictly adhere to the following JSON format, and NO other text MUST be included. The example format is as follows. Please make sure the parameter type is correct. If no function call is needed, please directly output an empty list "[ ]"

[ { "name": "func_name1", "arguments": { "argument1": "value1", "argument2": "value2" } }, ... (more tool calls as required) ]

What is the area of a hexagon with vertices at (1, 1), (5, 1), (7, 5), (5, 9), (1, 9), and (0, 5)? Also, extract the 4 largest numbers in the list [120, 130, 140, 150, 160]"

"chosen": [{"name": "polygon_area_shoelace", "arguments": {"vertices": [[1, 1], [5, 1], [7, 5], [5, 9], [1, 9], [0, 5]]}}, {"name": "find_n_largest_numbers", "arguments": {"nums": [120, 130, 140, 150, 160], "n": 4}}],

"rejected": [{"name": "polygon_area_shoelace", "arguments": {"vertices": [[1, 1], [5, 1], [7, 5], [5, 9], [1, 9], [0, 5]], "n": 4}}, {"name": "find_n_largest_numbers", "arguments": {"nums": [120, 130, 140, 150, 160], "n": 4}}]
D  The Comparison Between DPO and KTO

In our PTC dataset, the positive and negative responses are often similar. This similarity arises because errors in negative responses typically occur when assigning specific values to parameters. For instance, as demonstrated in Example C.4, the only difference between positive and negative responses is the parameter \( n : 4 \). In the following analysis, we provide a detailed explanation of why this difference leads to the DPO’s failure mode.

To provide a brief analysis of why DPO fails, consider the DPO objective:

\[
\mathcal{L}_{DPO}(x, y_w, y_l; \theta) = - \log \sigma \left( \beta r_\theta(x, y_w) - \beta r_\theta(x, y_l) \right),
\]

where \((x, y_w, y_l)\) are paired data with \(x\) as the prompt, \(y_w\) as the positive answer and \(y_l\) as the negative answer. \(r_\theta(x, y)\) is the log-ratio of the likelihoods of answer \(y\) between the training model (i.e., \(\pi_\theta(y|x)\)) and the reference model (i.e., \(\pi_{ref}(y|x)\)), where \(\theta\) is the model parameter. \(r_\theta(x, y)\) is denoted as \(r_\theta(x, y) = \frac{\pi_\theta(y|x)}{\pi_{ref}(y|x)}\).

Suppose \(y_w\) and \(y_l\) are length \(K\) sequences, and they only differ at i-th token, i.e., \(y_w = [t_1, \cdots, t_{i-1}, t_i^w, t_{i+1}, \cdots, t_K]\), and \(y_l = [t_1, \cdots, t_{i-1}, t_i^l, t_{i+1}, \cdots, t_K]\). Then the gradient of \(\mathcal{L}_{DPO}\) is calculated as:

\[
\nabla_\theta \mathcal{L}_{DPO} = - \beta \cdot \sigma(-c) \cdot \nabla_\theta \left[ \log \pi_\theta(y_w|x) - \log \pi_\theta(y_w|x) \right]
\approx - \beta \cdot \sigma(-c) \cdot \nabla_\theta \left[ \log \pi_\theta(t_i^w, y^{>i}|x, y^{<i}) - \log \pi_\theta(t_i^l, y^{>i}|x, y^{<i}) \right],
\]

where \(c = \beta r_\theta(x, y_w) - \beta r_\theta(x, y_l)\), \(y^{<i} = [t_1, \cdots, t_{i-1}]\) is the sequence before i-th token, and similarly, \(y^{>i}\) is the sequence after i-th token. Since \(y_w\) and \(y_l\) are different only at one token, it is possible that \(\log \pi_\theta(t_i^w, y^{>i}|x, y^{<i}) \approx \log \pi_\theta(t_i^l, y^{>i}|x, y^{<i})\), and \(c \approx 0\), which causes the gradient to vanish. We empirically validate this phenomenon by plotting the training loss (Figure 8(a)) and the gradient norm (Figure 8(b)) during the training of DPO on the PTC dataset.

In addition to the weakening of the update signal resulting from the vanishing gradient, the small differences between \(y_w\) and \(y_l\) further lead to reduced probability of positive samples, as demonstrated in (Pal et al., 2024). We also empirically validate this phenomenon by plotting the log probabilities for both positive and negative samples (see Figure 8(c) and Figure 8(d)). Our observations indicate that during training, although the log probabilities of negative samples decrease, the log probabilities of positive samples also decrease.

E  The Comparison Between PPO and KTO

Besides DPO, Proximal Policy Optimization (PPO) is also a widely used method to xx. However, is not usefull under our setting. We here conduct experiments comparing HiTEC-KTO with PPO. The PPO setting uses the same negative samples generated as described in Section 3.3.1. Results are presented in Table 13. We use Llama-Factory 4 to first train a reward model and then conduct PPO.

As demosntrated in the table, there is a huge performance gap between HiTEC-PPO and HiTEC-KTO. This gap can be attributed to two well-known limitations of PPO: 1) poor generalization of the reward model – while the reward model performs well on the training dataset, it lacks generalization to unseen

4https://github.com/hiyouga/LLaMA-Factory/tree/main
Table 13: Comparison Between PPO and KTO

<table>
  <tr>
    <th rowspan="2">Dataset</th>
    <th rowspan="2">Method</th>
    <th colspan="2">Qwen2.5-1.5B</th>
    <th colspan="2">Qwen2.5-7B</th>
  </tr>
  <tr>
    <th>F1 Name</th>
    <th>F1 Name + Param.</th>
    <th>F1 Name</th>
    <th>F1 Name + Param.</th>
  </tr>
  <tr>
    <td rowspan="2">Tool-Alpaca</td>
    <td>HiTEC-PPO</td>
    <td>7.01</td>
    <td>1.23</td>
    <td>13.16</td>
    <td>4.37</td>
  </tr>
  <tr>
    <td>HiTEC-KTO</td>
    <td><b>82.27</b></td>
    <td><b>52.26</b></td>
    <td><b>87.63</b></td>
    <td><b>59.19</b></td>
  </tr>
  <tr>
    <td rowspan="2">Seal-Tools</td>
    <td>HiTEC-PPO</td>
    <td>8.84</td>
    <td>2.21</td>
    <td>17.68</td>
    <td>8.15</td>
  </tr>
  <tr>
    <td>HiTEC-KTO</td>
    <td><b>96.99</b></td>
    <td><b>89.75</b></td>
    <td><b>96.27</b></td>
    <td><b>89.29</b></td>
  </tr>
</table>

data, leading to suboptimal reinforcement learning. And 2) susceptibility to reward hacking, where the model learns to optimize for the reward function in unintended ways rather than genuinely improving task performance (Fu et al., 2025; Gao et al., 2023).

A supporting example further illustrates this issue. Our trained reward model demonstrated high accuracy during both training and evaluation; however, it assigned a high score to the following output, despite the response containing formatting errors and irrelevant code snippets.

This example indicates that the proposed generated dataset is not well-suited for PPO training. In contrast, KTO effectively integrates human demonstrations to refine the reward function, aligning it more closely with real-world preferences (Ethayarajh et al., 2024). This alignment explains why KTO significantly outperforms PPO in our experiments and why it is a better fit for the current dataset.
Example: The PPO Reward Model’s High-Scoring of Model Responses

Here is my response:

[
{
"name":"PublicHolidayNextPublicHolidays",
"arguments":{
"countryCode":"CN" // Replace with desired country code if different than GB
}
},
[
{
"name": "CountryCountryInfo",
"arguments":{"countryCode":"cn"}
}
]

# Add any additional functions here

]
]


/README.md
import sys

sys.path.append("src")
from qwen_agent.agent import Agent
from qwen_
stream=True,
stop=["

"],

agent = Agent(
llm=llm,
)

result = agent.run(user_message="Get me some jokes.")
print(result)
agent.tools.qwen_tools import *
from langchain.chat_models import ChatOpenAI

api_key_file_path = "/home/ubuntu/qwen-agent/.en
...