Learning to Use Tools via Cooperative and Interactive Agents (2024)

Abstract: Tool learning empowers large language models (LLMs) as agents to use external tools and extend their utility. Existing methods employ one single LLM-based agent to iteratively select and execute tools, thereafter incorporating execution results into the next action prediction. Despite their progress, these methods suffer from performance degradation when addressing practical tasks due to: (1) the pre-defined pipeline with restricted flexibility to calibrate incorrect actions, and (2) the struggle to adapt a general LLM-based agent to perform a variety of specialized actions. To mitigate these problems, we propose ConAgents, a Cooperative and interactive Agents framework, which coordinates three specialized agents for tool selection, tool execution, and action calibration separately. ConAgents introduces two communication protocols to enable the flexible cooperation of agents. To effectively generalize the ConAgents into open-source models, we also propose specialized action distillation, enhancing their ability to perform specialized actions in our framework. Our extensive experiments on three datasets show that the LLMs, when equipped with the ConAgents, outperform baselines with substantial improvement (i.e., up to 14% higher success rate).

## introduction although large language models (llms) have achieved remarkable performance in a broad range of natural language processing tasks @xcite , they still encounter inherent limitations such as out-of-date information @xcite . tool learning is proposed to equip llms with various auxiliary resources, e.g., a search engine (shi et al., 2024b; nakano et al., 2021) * corresponding author or a calculator @xcite , which empower them as tool-use agents and improve their proficiency in tackling concrete complex tasks. as shown in figure 1 (a), most previous studies allow the llm-based agent to interleave multiple actions in a pre-defined order to interact with tools @xcite @xcite . the agent first breaks down the task and plans a series of tools in a stepby-step manner. for each step, the agent executes the tools by passing arguments and continuously incorporates useful intermediates into the next action prediction. despite the advancement of existing methods, they face two challenges in practice. first, most of them alternate the planning and execution with a pre-defined pipeline @xcite , which inevitably constrains their flexibility in handling exceptional errors that frequently occur during a tool-use workflow @xcite @xcite . when failing to invoke tools, it is crucial to enable agents to revise their incorrect actions instead of directly shifting to the next step with the error response of previous steps. second, it is struggling to adapt a single llm-based agent to learn a variety of specialized actions in solving a task @xcite . solving a practical task involves varied actions with substantial differences, e.g., planning, execution, and reflection, drawing upon different facets of the llms @xcite . therefore, developing effective agent flow and organizing tool-use models to solve practical tasks remains a challenging research topic. in this work, we propose conagents, a cooperative and interative agents framework for tool learning tasks. as shown in figure 1 , our conagents decomposes the overall tooluse workflow using three specialized agents: grounding, execution, and review agents. the grounding agent reasons the task description and grounds it into planning by specifying which tool to use. the execution agent follows the planning to execute the selected tool by generating executable code. the review agent reviews the incorrectness in planning or execution, providing feedback for revision. to enable the dynamic cooperation of these specialized agents, we propose two communication protocols, including automatic and adaptive interaction. in the process of automatic interaction, the review agent provides real-time reviews to calibrate incorrect actions. thus, the agent flow alternates between the planning-review and execution-review phases as shown in figure 1 . in the process of adaptive interaction, the review agent only provides feedback when exceptional errors are captured while executing the tools. for a comprehensive evaluation, we conduct experiments on two benchmarks, i.e., toolbench and restbench, using various llms as backbones. we find that conagents outperforms the state-ofthe-art baseline with both communication protocols (6% improvement in success rate on average). despite closed-source llms performing well with our framework, we find the open-source models may struggle with the modulized agent flow. thus, we propose an approach called specialized action distillation (span), enhancing the performance of open-source models in conagents. we heuristically sample 2,919 high-quality tasks from the toolbench @xcite training set, and cluster them based on their similarity, retaining only one task in each cluster to avoid duplication. for each task, we guide the gpt-4 to generate solutions using conagents, and reorganize them into actions tailored to specialized agent functionalities in conagents. these actions are separately distilled into different student models through instruction tuning. we employ parameter-efficient tuning techniques, i.e., @xcite , further extending our distillation method into low-resource scenarios. experiment results show that our distillation method empowers open-source models with strong performance with only 500 training examples. multi-agent cooperation. synergizing multiple agents has demonstrated strong performance on a variety of tasks @xcite @xcite , enhancing the capabilities of individual agents @xcite @xcite . recent studies take multiple agents into a debate for a fixed number of rounds @xcite , boosting their factuality @xcite and reasoning abilities @xcite . in the tool learning tasks, recent work separately implements the task planning and execution with different agents, thereby reducing the workload of a single agent @xcite @xcite . despite their progress, their agent flow is simplified into a predefined pipeline @xcite , struggling to handle exceptional errors that frequently occur during the tool-use workflows @xcite . in our work, we propose two communication protocols, which enable the action calibrations and dynamic cooperation of agents. ## specialization by agent distillation our initial experiment shows that powerful llms such as gpt-4, achieve promising results when equipped with our framework. however, these model are often considered black boxes @xcite with potential privacy issues. thus, we aim to adapt our framework to open-source models. we propose specialized action distillation (span), which distills the tasksolving trajectory of powerful commercial llms into different open-source llm agents tailored to specific functionalities in conagents. ## discussion qualitative analysis for the maximum number of interactions. in our automatic agent interaction, agents m g and m e revise their actions following the feedback of agent m r for up to α and β turns, respectively. we also look at the poorly performing cases where we find that since the planning from agent m g is typically open-ended, the m r struggles to detect all the incorrect planning. for example, the planning may be plausible and clear but lacks the required arguments to execute tools, thus resulting in a failure of m e in subsequent steps. qualitative analysis for the efficiency of inference. due to the intensive inference cost of llms-based agents, we further explore the efficiency of our conagents. to explain more intuitively, we compare the token consumption for the conagents and baselines using the restbench-tmdb dataset with the same settings as in the quality of generated review. we further analyze the quality of reviews given by review agent m r . specifically, we randomly sample 50 task-solving trajectories in runtime consistency. considering the nondeterministic nature of llm generation, we analyze the consistency of our framework. we repeat our method multiple times with the same settings as in ## conclusions we present a cooperative and interactive agents framework (conagents) for tool learning, which diverges from previous work by allowing the cooperation of agents to solve complex tasks. the conagents first modularizes the overall workflow with three specialized agents for tool planning, tool execution, and action calibration, respectively. then, two communication protocols are introduced to enable the dynamic cooperation of these agents. to generalize our framework to open-source models, we propose specialized action distillation, enhancing the models' capability to perform specific actions. extensive experiments conducted on three datasets demonstrate the superiority of our conagents, e.g., pushing the success rate to 77.00 with 13.2% point improvement. our future work includes: (1) extending our method to agents empowered by multi-modal foundation models, incorporating image and sound; (2) coordinating the cooperation between strong and weak agents.