R o TB ench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning (2024)

Abstract: Tool learning has generated widespread interest as a vital means of interaction between Large Language Models (LLMs) and the physical world. Current research predominantly emphasizes LLMs’ capacity to utilize tools in well-structured environments while overlooking their stability when confronted with the inevitable noise of the real world. To bridge this gap, we introduce *RoTBench*, a multi-level benchmark for evaluating the robustness of LLMs in tool learning. Specifically, we establish five external environments, each featuring varying levels of noise (i.e., Clean, Slight, Medium, Heavy, and Union), providing an in-depth analysis of the model’s resilience across three critical phases: tool selection, parameter identification, and content filling. Experiments involving six widely-used models underscore the urgent necessity for enhancing the robustness of LLMs in tool learning. For instance, the performance of GPT-4 even drops significantly from 80.00 to 58.10 when there is no substantial change in manual accuracy. More surprisingly, the noise correction capability inherent in the GPT family paradoxically impedes its adaptability in the face of mild noise. In light of these findings, we propose RoTTuning, a strategy that enriches the diversity of training environments to bolster the robustness of LLMs in tool learning. The code and data are available at https://github.com/Junjie-Ye/RoTBench.

## introduction tool learning has emerged as a critical concept for empowering large language models (llms) @xcite @xcite to interact with the real world @xcite @xcite . in this context, the external environment of an llm contains * corresponding authors. get_weather : this tool is used for fetching information weather for specified location. parameters: location (string): designated location, default is current location. please tell me the weather in the new york. abc: this tool is used for fetching information weather for specified location. parameters: location (string): designated location, default is current location. please tell me the weather in the new york. i'm sorry, but as a language model, i don't have access to weather information. figure 1 : example of noise affecting tool selection for llms. although the functionality of the tool remains unaffected by its name, renaming "get_weather" as "abc" impedes llms from utilizing the tool properly. an ensemble of integrated tools. each tool is uniquely identified by its name and is described by a succinct paragraph that explains its functionality. similarly, every parameter within these tools is characterized by its name, along with a description that clarifies its purpose, its optionality, and other pertinent details. recent research has centered on examining how well llms can effectively employ tools within a carefully designed and stable environment. from one perspective, specific studies have scrutinized the outcomes of llms' tool usage, verifying both the accuracy of tool selection and the efficacy of the generated responses @xcite . this analysis involved evaluating the relevance of the selected tools and the final responses in fulfilling users' requirements. on the other hand, other investigations have delved into the intricate process of tool utilization by llms, striving for a more comprehensive assessment of their performance in tool learning @xcite . this includes an analysis of the diverse capabilities necessary for llms to excel in tool learning while also identifying any limitations they may have in this regard. however, these studies fail to account for the robustness of llms in the face of inevitable noise in real-world scenarios @xcite . using figure 1 as a reference, llms recognize the tool for querying weather information when named "get_weather," but not when named "abc," despite the tool's functionality remaining unaffected by its name. consequently, it becomes imperative to investigate whether llms can proficiently identify these tools and configure parameters to meet user needs in noisy realworld environments. this research is essential to guarantee their reliability in practical applications. to fill this gap, we introduce rotbench, a multilevel benchmark for evaluating the robustness of llms in tool learning. specifically, we establish five external environments, which can be categorized as clean, slight, medium, heavy, and union in ascending order of noise levels. by evaluating the performance of llms across three critical stages: tool selection, parameter identification, and content filling, we aim to offer a thorough and intricate analysis of the stability and reliability of llms in tool utilization. through experiments conducted on six widelyused llms, we observe that the performance of these models is remarkably sensitive to noise. for instance, the performance of gpt-4 even drops significantly from 80.00 to 58.10 when there is no substantial change in manual accuracy. this underscores the pressing requirement to enhance the robustness of llms in tool learning. interestingly, the gpt family of models' inherent noise correction capability appears to hinder its performance in mildly noisy environments. in light of these findings, we introduce rottuning, a technique aimed at augmenting the adaptability of llms to a wide range of environments by introducing greater environmental diversity during the training phase. our experimental results demonstrate that our approach yields an average performance improvement of 16.10 points across diverse environments. the main contributions of our work are summarized as follows: robustness testing of llms robustness is a critical factor in determining the stability of llms and plays a pivotal role in their practical deployment in real-life applications, which has garnered significant attention from scholars. in tool: cat_breed # returns a list of cat breeds. param: limit: optional[string] # limit the amount of results returned. delimiter : optional[string] # delimiter between different breeds, defaults is comma. insertion & exchange reversal & nonsense exchange & omission eval tell me the breed of white dogs. cat_breed ( ) deerb_god ( ) deerb_god (@xmath0. parameter identification cat_breed ( @xmath1, asd @xmath2 ## rotbench as depicted in figure 2 , rotbench encompasses five environments, each characterized by varying levels of noise, facilitating a thorough evaluation of the robustness of llms throughout the three stages of tool usage. ## rottuning it is evident that enhancing the robustness of llms in tool learning is imperative. to tackle this issue, we introduce rottuning, a novel approach aimed at bolstering the robustness of llms through increased environmental diversity. ## conclusion in this paper, we introduce rotbench, a multilevel benchmark for evaluating the robustness of llms in tool learning. rotbench contains five environments, each characterized by varying noise levels, shedding light on the pressing need to bolster the robustness of llms. furthermore, we present rottuning, an innovative approach that significantly improves the robustness of llms in tool learning by increasing the diversity of environments during the training phase.