Benchmarking Failures in Tool-Augmented Language Models (2025)

Abstract: The integration of tools has extended the capabilities of language models (LMs) beyond vanilla text generation to versatile scenarios. However, tool-augmented language models (TaLMs) often assume ‘perfect’ information access and tool availability, which may not hold in the real world. To systematically study TaLMs imperfections, we introduce the FAIL-TaLMs benchmark, featuring two major failures: under-specified user queries and non-available tools. FAIL-TaLMS contains 1,749 examples using 906 tools across 21 categories, including single- and multi-tool usage. We evaluate top-performing proprietary and open-source models, and find all current models except for Claude struggle to recognize missing tools or information. Further, to study possible mitigation of the failures, we enable real-time human interaction, named the Ask-and-Help method, to provide missing information or replace non-functional tools. While Ask-and-Help can help models solve tasks more correctly when queries are under-specified, it brings minimal benefit when complex tools are broken.

## introduction tools can greatly enhance language models (lms) by facilitating their problem-solving process @xcite and extending their abilities @xcite . given a user query, a tool-augmented language model (talm) can selectively call tools to gather more information and perform computation activities to accomplish the user's request. such talms have been applied in various scenarios, including interacting with versatile knowledge bases @xcite , real-world data @xcite , and even multi-modal information @xcite . on the other hand, tools can deprecate over time @xcite , to systematically study these practical failures and enable more robust talm systems, we introduce fail-talms -a benchmark designed to examine talms under information insufficiency and tool unavailability ( §3). we gather 906 real-world tools across 21 categories directly from their host sources, and construct execution environments for all tools along with verifying test cases. unlike existing benchmarks with limited tool calls via third-party platforms, our tool environment allows real-time and reproducible testing. with this collection of tools, we created 575 examples with perfect information and tool availability. we then transformed them into 599 and 575 examples with under-specified queries and unavailable tools, by removing key information from the queries and masking out necessary tools from the provided list, to study the two major failure modes mentioned above. overall, fail-talms contains 1,749 queries across 906 tools from 21 categories. we propose three evaluation metrics to study model performance: pass rate to measure task success, awareness of missing tools or information, and further, unexpected outcomes to capture when talm correctly yet unexpectedly solves a task. we experiment with a series of top-performing lms ( §4): open-weight llama 3 models from 8b, 70b to 405b, and proprietary models including claude and gpt. our experiments reveal that most models struggle to identify the lack of tools or information needed to solve a task, except for claude with a 56% awareness rate, 28-54% higher than other models. nonetheless, high awareness does not translate to higher pass rates. for example, gpt-4o achieves 4% higher pass rate than claude-3.5-sonnet, despite scoring 44% lower in awareness. finally, to examine whether simple mitigation measures could address these issues, we study if a method enabling talms to interact with humans, dubbed "ask-and-help" (aah), could help obtain missing information or fulfill the function of unavailable tools ( §5). we measure interaction ratio to see how often talm interacts with humans. aah substantially improves pass rate particularly when user queries are under-specified, where the models actively interact with humans 21-61% of the time to gather missing information. however, this human assistance does not bring improvements when tools are unavailable, regardless of the tool functions are replaceable by humans or not, indicating room for better methodologies. ## problem statement a tool-augmented language model (talm) consists of (1) a backbone language model m and (2) a set of tools @xmath0, . . . , t n }. each tool t i is a callable function (e.g., calculator(expr), document_retriever(query,docs)). given a natural language (nl) query q, the talm selects a series of tools t q ⊆ t to solve the query. for each chosen tool t @xmath1, the talm produces a tool-calling program p q @xmath2, t ) that is executed then yielding output e q i . finally, the talm produces the final answer r q to query q based on all tool outputs {e q i }. however, this classic talm pipeline may not successfully execute in practice due to two primary issues: under-specified queries when the user query q is under-specified, either the subset of relevant tools t q cannot be successfully identified, i.e., m det (q, t ) ̸ ⇒ t q , or the set of tool-calling programs p @xmath3. unavailable tools even when there is sufficient information, the tool may be unavailable due to reasons such as deprecated functions or server execution errors (e.g., server timeout or connection failure). in such cases, the tool execution fails or returns incorrect results, i.e., exec(p) ̸ → e, leading to invalid or inaccurate tool execution outputs. ## the fail-talms benchmark in this section, we first introduce the tool collection ( §3.1) and benchmark curation processes ( §3.2), present the data overview ( §3.3), then establish the set of evaluation metrics ( §3.4). ## experiments and results in this section, we introduce the experimental setup ( §4.1) and talm pefromance without aah ( §4.2) and with ( §5) human assistance via aah. ## conclusion in this paper, we introduced fail-talms, a comprehensive benchmark designed to evaluate toolaugmented language models (talms) under realistic conditions where user queries are underspecified or necessary tools are unavailable, consisting of 1,749 queries using 906 authorizationfree tools across 21 categories. our experiments with both proprietary models (gpt, claude) and open-weights models (llama series) revealed that most talms struggle to recognize missing information or unavailable tools. to address this, we examine the ask-and-help (aah) method, allowing talms to interact with humans in real time to obtain missing information or substitute nonfunctional tools. while we find aah improves the pass rate on under-specified queries, it has minimal impact when complex tools are unavailable.