Butterfly Effects in Toolchains: A Comprehensive Analysis of Failed Parameter Filling in LLM Tool-Agent Systems (2025)

Abstract: The emergence of the tool agent paradigm has broadened the capability boundaries of the Large Language Model (LLM), enabling it to complete more complex tasks. However, the effectiveness of this paradigm is limited due to the issue of parameter failure during its execution. To explore this phenomenon and propose corresponding suggestions, we first construct a parameter failure taxonomy in this paper. We derive five failure categories from the invocation chain of a mainstream tool agent. Then, we explore the correlation between three different input sources and failure categories by applying 15 input perturbation methods to the input. Experimental results show that parameter name hallucination failure primarily stems from inherent LLM limitations, while issues with input sources mainly cause other failure patterns. To improve the reliability and effectiveness of tool-agent interactions, we propose corresponding improvement suggestions, including standardizing tool return formats, improving error feedback mechanisms, and ensuring parameter consistency.

## introduction in recent years, large language models (llms) have shown promising performance in many tasks, but for some professional tasks or tasks that require multi-step processing, relying on a single llm may not be sufficient to meet the task requirements. to address these challenges, researchers have proposed the concept of tool agents, which integrate llms with various external tools to complete numerous complex tasks @xcite @xcite . this expands the capabilities of llms @xcite and highlights their significant application potential. figure 1 illustrates how a tool agent resolves a user query. as we can see, when a tool agent receives a user query, it first retrieves the tool and plans the tool sequence. then, by parsing the existing input (user query or output result of the previous tool), it forms a parameter list for the subsequent tool invocation. consequently, the proper execution of tool agents is highly dependent on the accuracy of parameter parsing, which is a critical process. however, parameter errors or parameter hallucinations often occur during the parameter parsing process due to incomplete or ambiguous user queries, low-quality tool document parameter specifications, non-standard tool return results, and limitations of llm capabilities @xcite . for example, in the erroneous invocation chain on the left side of figure 1 , the tool does not call success because "australia" does not meet the abbreviation requirement for the "region" parameter. it also includes deviations from the intent of the user. on the right side of the figure, setting the region to "us" does not match the country "australia" in the query. more failure cases can be seen in the appendix a.4. as demonstrated in the example, parameter issues often lead to failed tool invocations by the agent and may even confuse the invocation chain, thereby reducing the quality of task completion, like the "butterfly effect" in the toolchains that affects the normal tool agent execution. moreover, according to existing research @xcite , parameter issues commonly exist in the execution traces of tool agents. data shows that about 44% of simple user queries and 48% of complex user queries have parameter issues, which greatly limits the usability of tool agents, especially in some critical domains @xcite . these concerning figures highlight the urgency of addressing the parameter issues in tool figure 1 : the process by which a tool agent resolves a user query, as well as the parameter issues that may occur during the invocation of the toolchain. agent execution traces. many research works have been proposed to explore the above issues @xcite @xcite . however, they either focus on specific tasks or emphasize the completion of overall tasks while neglecting the internal logic of tool effectiveness and the mechanisms of parameter handling. in addition, few studies comprehensively consider the impact of different input sources on the accuracy of parameter filling, including user queries, tool documents, and tool return results. @xcite @xcite . to bridge this gap, we propose an empirical study that analyzes the parameter filling process in tool agents. we aim to construct a comprehensive parameter failure taxonomy, deeply analyze the impact of different input sources on parameter filling accuracy, and provide actionable insights to enhance the reliability of tool agents, alleviating parameter generation errors and hallucination issues. specifically, we used the current mainstream tool agent toolllama @xcite as the investigated target and scientifically applied grounded theory (gt) @xcite to develop a failure taxonomy through open coding and constant comparative analysis. by applying 15 perturbation methods to input sources, we analyzed the impact on agent parameter behavior on four advanced llms. the contributions of this paper are as follows. in addition, to further simulate complex real-world scenarios, we strive to perturb the input sources of parameter information. by analyzing the changes in the llm's parameter behavior, we can provide effective suggestions for improving the reliability and effectiveness of the tool agent. ## failure taxonomy construction the purpose of this study is to explore the impact of parameter failure issues on tool agents, identify the root causes of tool invocation failure involving parameters, and propose improvement suggestions. however, there is not yet a comprehensive parameter failure taxonomy in place. therefore, before conducting experiments, we first need to construct a parameter failure taxonomy. the construction process is shown in figure 2 . we first utilized benchmark datasets reported in @xcite , and adopted toolllama as the foundational llm to obtain the behavior trajectories of the llm when resolving user queries. the selected dataset and llm are widely used in the field of tool agent research, ensuring the representativeness of the results. after obtaining the behavior trajectories, they will be utilized for mining failure patterns. we followed the grounded theory approach (gt) @xcite , which is a qualitative research method that directly constructs theories from empirical data, to identify failure patterns. specifically, the annotators first understand the function of each available tool and clarify the relevant parameter requirements, then apply the open coding approach @xcite to analyze the agent parameter filling behavior in the environment interaction trajectories that we collected. open coding is a data analysis method that involves decomposing and conceptualizing data to generate corresponding codes for identifying key patterns, themes, or phenomena in the data. then, using constant comparative analysis, we systematically compared the new codes created by the annotators with the existing codes. this iterative process of parameter failure pattern identification and open coding continued until no new insights emerged from additional data. eventually, we obtained preliminary patterns from toolllama's behavioral trajectories. to refine the patterns, we conducted a group agreement study to ensure that all three annotators had a consistent understanding of the results and gradually made iterative modifications to form the consensus patterns. this process continues until the cohen's kappa score reaches above 0.9, which, as suggested by (landis and @xcite , indicates an almost perfect level of agreement among annotators and high data quality. finally, we constructed a taxonomy that includes five distinct categories of parameter failures that occur during tool agent invocation as follows: ## methodology based on the constructed taxonomy, we can conduct a corresponding sensitivity analysis on the phenomenon of parameter failure. this process can draw on the ideas of defect detection in the field of software engineering, which leverage test oracle to evaluate the results of the test execution and determine if they meet expectations young (2001) based on this concept, we treat the initial correct behavior trajectories as test oracles and perturb them to generate the test samples. considering that there are three input sources (i.e., tool document, user query, tool return) of tool agent that will have impact on the parameters, to conduct comprehensive analysis, we design targeted perturbation methods for each of the three input sources to generate corresponding test samples, which will be explained in detail as follows. ## results in this section, we analyze the sources of failure and clarify their impact on task completion. based on these results, we propose further suggestions for improvement in the tool agent design. the main results are shown in original data failure yes filter base llms initial data augmented data prompt to generate user queries based on the given tool list. gpt-4 test agent behaviors toolchain 1 ... ... initial behaviors trajectories toolchain 2 toolchain i toolchain n perturbation methods rp l rp f an cp ap fk uk ck cf rd re wd wt sd co test agent behaviors toolchain 1 ... ... toolchain 2 toolchain i toolchain m no failure taxonomy sensitivity analysis results ## conclusion the study delves into the complex parameter challenges faced by llm tool agents during the execution of the toolchain, revealing five distinct failure patterns that disrupt workflow integrity. missing required parameters will prevent the tool from fully processing the task; redundant information affects the accuracy of the tool's return results; hallucination names and mismatch specifications fundamentally prevent the tool from being invoked, and task deviation affects the practical value of the result to the user. ambiguous user queries, defective tool documents, or poorly written tool return results can spread throughout the toolchain, causing parameter failures and ultimately leading to task failure. this paper provides a blueprint for designing robust tool agents by mapping failure patterns and their interconnections, highlighting proactive input structuring, adaptive error correction, and attention to parameter propagation in the toolchain to ensure reliable and traceable toolchain operations.