T ool R eflection: Improving Large Language Models for Real-World API Calls with Self-Generated Data (2025)

Abstract: While open-source large language models (LLMs) have advanced in leveraging third-party tools, significant challenges remain in real-world API usage, where behavior is unpredictable or poorly specified. Existing benchmarks often fail to capture this complexity. We propose ToolReflection, a novel method that improves LLMs’ ability to self-correct API calls by utilizing real-time API feedback. We also introduce new datasets specifically designed to test model performance under realistic conditions. In ToolReflection, models undergo instruction tuning on a dataset augmented with self-generated errors and corrections. Our evaluation across ToolAlpaca, ToolBench benchmarks, and three newly developed datasets (GPT4Tools-OOD, GPT4Tools-OOD-Hard, and Multistep-100) demonstrates its effectiveness. ToolReflection boosts overall success rates by 25.4% on GPT4Tools-OOD, 56.2% on GPT4Tools-OOD-Hard, and 4% on Multistep-100, outperforming original models. On ToolAlpaca, we show a 14% improvement in the “Simulated” setting and 10.5% in the “Real-world” scenario. Our error analysis highlights ToolReflection significantly enhances recovery from incorrect tool calls, even with incomplete or erroneous API documentation. We have released the code, prompts, and data at https://github.com/polgrisha/ToolReflection.

## introduction modern llms excel at various tasks, including text generation, coding, question answering, and ranking @xcite . however, the knowledge llms gain during pretraining is often inadequate for tasks requiring precise algorithmic reasoning, such as arithmetic, or those dependent on real-time data, for example, travel planning or weather updates. to bridge these gaps, figure 1 : overview of toolreflection. given a retrieved set of tools, the model fine-tuned with toolreflection selects a tool and its parameters, generates an api call, and processes execution feedback. if an error occurs, it provides a rationale and adjusts the call accordingly. llms have been extended with third-party tools, allowing them to execute complex actions using api documentation included in their prompts. significant research has focused on improving llms' tool-usage capabilities. advances include instruction-tuned models for better tool calls (gpt4tools @xcite , toolalpaca @xcite , @xcite ) and sophisticated external frameworks for multi-step reasoning and planning (toolchain* @xcite , toolllama with depth-first search-based decision tree (dfsdt) @xcite ). however, even with higher-level orchestration strategies, the underlying ability of llms to robustly handle individual api invocations in real-world settings remains a considerable challenge. models frequently hallucinate tool names or parameters and fail to recognize and correct api call errors. existing solutions to these granular errors, like guiding models with a finite state machine @xcite , simplifying tool documentation @xcite , or incorporating external model feedback @xcite , are often impractical due to computational demands or the need to rewrite tool documentation. in this work, we explore tool-augmented llms' ability to self-correct using feedback directly from invoked tools. we focus on two key challenges: (1) the lack of diverse, high-quality benchmarks that reflect real-world scenarios and (2) models' frequent failure in real-world tasks. to address these challenges, we propose two solutions. first, we extend existing datasets and develop new, annotated evaluation sets that better mimic real-world conditions, specifically gpt4tools-ood and multistep-100 based on gpt4tools and toolbench respectively. second, we enhance the models' self-correction abilities by introducing a fine-tuning phase using self-generated examples of errors, tool responses, and corresponding corrections. our approach, evaluated on gpt4tools, toolalpaca, and toolbench, shows consistent improvements across all settings. fig. 2 provides examples of how our method successfully corrects tool usage errors. notably, even curated api sets often contain incomplete or incorrect documentation, making self-correction after an error message the only viable solution. our main contributions are: (1) we analyse three existing benchmarks for tool use and propose improvements to make them better adapted to real-world tools, including the training dataset (gpt4tools), evaluation datasets (gpt4tools, toolbench), data cleaning (toolbench), and fixes in dataset format and the evaluation approach (gpt4tools, toolalpaca). (2) we provide a method to improve self-correction abilities of llms after getting error messages from the external tool via additional fine-tuning on self-generated examples with error corrections. (3) we evaluate and prove the effectiveness of our approach with a comprehensive experimental study, which demonstrates that the ability to correct the output based on external error feedback is necessary for successful communication with third-party apis. the rest of the paper is structured as follows: section 2 surveys related work, section 3 discusses benchmark improvements, section 4 introduces our toolreflection method, section 5 presents experimental results, section 6 provides error analysis, and section 7 concludes the paper. ## related work tool-augmented language models significant research has been devoted in recent years to enhancing the tool invocation capabilities of llms. the toolformer model @xcite showed that additional model fine-tuning on api calls in a self-supervised way leads to improvements in zero-shot performance of llms on downstream tasks. based on this idea, gpt4tools @xcite fine-tuned compact models to incorporate multi-modal tools and evaluated the tool usage accuracy on an automatically generated benchmark. the toolalpaca framework @xcite has been designed to address the issue of tool calling abilities on previously unseen tools by massively pretraining on a highly diversified tool use corpus. the toolllama model @xcite took a step towards applying real-world apis from the rapidapi hub. toolllama demonstrates a remarkable ability to execute complex instructions and generalize to unseen apis. to sum up, the studies of tool enhanced llms have mostly focused on creating tool use datasets for fine-tuning, while methods that construct chains of thought leading to the correct answer for tool calling remains problematic. however, recent studies address this challenge via external algorithms and structures. for instance, toolllama proposes to improve its reasoning strategy with dfsdt by allowing the backbone model to choose between different reasoning chains using a tree structure. toolchain* @xcite leverages the a* search algorithm for the same purpose. some works also explore ways to improve the quality of tool-based language models with external algorithms such as finite state machines @xcite or feedback from an external model @xcite . in contrast, our toolreflection approach focuses on a complementary aspect: directly enhancing the llm's intrinsic ability to understand and selfcorrect individual api calls using real-time feedback. while these external frameworks could potentially benefit from a base model improved by toolreflection, our primary focus is on enhancing the model's direct api engagement. self-correction from feedback attempts to incorporate self-correction to llms have led to improvements across a variety of tasks including question answering @xcite , reasoning @xcite , code generation @xcite , and summarization @xcite . selfcorrecting models apply two types of feedback: self-feedback -feedback obtained from the llm itself, and external feedback -feedback derived from external models, tools, or knowledge sources. self-feedback can involve asking the model itself to evaluate the quality of generated outputs via prompting @xcite or additional finetuning of the model on automatically generated self-correction samples @xcite . the approach of learning from external feedback is widely used for code generation tasks since it is relatively easy to receive such feedback through the execution of generated code with the corresponding compilers or interpreters @xcite . the trice framework @xcite recently proposed techniques for learning from execution feedback in tool-enhanced models, enabling a form of self-correction. while it's a notable step forward, trice mainly focuses on single-step mathematical and question-answering tasks using simple tools, where "execution feedback" mostly refers to the correctness of the final, single-answer output. our toolreflection approach builds on similar self-correction ideas but applies them to api calls closer to real-world usage. it focuses on direct, real-time error messages and structured responses from tool invocations. ## datasets analysis and extensions in this work, we focus on zero-shot realistic tool support, where tools are provided as apis or python functions with natural language descriptions of their usage and parameters. at each step, the llm is presented with several unseen tool descriptions and must select the appropriate tool and fill in its parameters. this task is typically solved via instruction tuning, training models on datasets of tool use examples formatted with dataset-specific inputs, chain-of-thought steps, and expected outputs. each dataset includes its own tool execution and evaluation framework. in this work, we aim to provide realistic evaluation with callable tools for every setup. for each dataset, we use its original prompting with minor adjustments. for gpt4tools and tool-bench, we create novel evaluation sets enabling actual tool invocation and exact output checks. for gpt4tools, we also generate additional training data. details on each dataset are discussed below. ## toolreflection in the frameworks we considered, tools can be implemented as python functions (e.g., in gpt4tools) or as external api calls via http (e.g., in tool-bench and toolalpaca). if an error occurs, the response often contains valuable semantic information, such as the error type, description, or the function where it was triggered. although response formats vary, we hypothesize that llms can leverage this feedback to improve their outputs. our experiments in sections 5 and 6 support this intuition, showing that (i) most errors generate feedback that the model can use for corrections, and (ii) many errors cannot be resolved without this feedback since tool documentation is often incomplete. this proves that understanding tool feedback is essential for accurate problem-solving. to enhance this capability, we propose the tool-reflection method, applicable across frameworks. the core idea of toolreflection is to leverage the model's own errors and subsequent corrections for additional fine-tuning. the key novelty of our approach lies not in the fine-tuning strategy itself, but in the source of these error-correction pairs: feedback obtained directly from the model's selfinteractions with apis or executable python functions. generating a dataset with error examples involves three steps. first, we collect queries from existing datasets that require tool use. second, we run a pretrained model on these queries and collect samples where it made tool invocation errors. we use feedback from the python interpreter or http responses to identify errors. finally, we form a new example containing: (i) an incorrect tool call, (ii) feedback from the tool, and (iii) the correct tool call. we integrate this feedback into the model's reasoning process as a self-reflection step (see fig. 1 and fig. 2 for examples). after generating this dataset, we fine-tune the model on it and test it on out-of-domain examples (see below). for synthetic datasets without executable tools, we create dummy functions that check parameter correctness, simulating feedback from tool calls. we propose two setups for toolreflection. in both, we start with a model fine-tuned on tool usage instructions and use it to generate error correction examples. then, we either (i) further finetune on these examples with a smaller sample of original data (post-finetune), or (ii) augment the original data with error corrections and fine-tune from scratch (fine-tune). ## error analysis in this section, we analyze the types of errors made by the model, some of which were corrected using toolreflection, while others were not. tool calls can be generally categorized into three types: (1) a correct tool call; (2) an incorrect tool call that produces error feedback from the tool; (3) a formally correct tool call with no negative feedback, but with wrong choice of tool or parameters. both the second and third types can be considered errors. our fig. 2 (left panel) illustrates an important use case which is successfully handled by the toolreflection-trained model. here, the generated call is semantically correct, but api requires strict input format, which can be extracted from the api response: the pintapi_convert_units tool accepts pa, but not pascals. for further details please refer to appendix b. ## conclusion in this work, we aim to improve tool invocation by llms, a core technology in next-generation virtual assistants and ai applications. modern llms support a paradigm where the backbone model manages user communication and invokes external tools as needed. for this paradigm to succeed, the toolset must be extensible: an instruction-tuned backbone should support new tools based on a sim-ple paragraph of api documentation, learned in context. we demonstrated that existing solutions struggle with complex real-world apis, which often return unpredictable or variable responses. this makes it difficult to accurately evaluate the success of a user request. moreover, third-party apis may be costly, unstable, or subject to failure, making training on their outputs impractical. these issues are pronounced when handling user queries requiring multi-step solutions, where each tool call depends on previous results. our analysis focused on the limitations of three existing solutions: gpt4tools, toolalpaca, and toolbench. we proposed methods to bridge the gap between academic datasets and real-world applications. we developed realistic evaluation protocols and showed that practical models can be built even from entirely synthetic data and descriptions, as shown in our gpt4tools experiments. additionally, our toolreflection approach proved effective in recovering from errors using api feedback, and our experiments show this compensates for insufficient tool documentation, a frequent challenge with third-party apis. however, our error analysis highlights that tool-reflection still cannot address certain error classes. in future work, we plan to explore methods to reduce uncorrectable errors and further enhance llm self-correction capabilities.