Small LLM s Are Weak Tool Learners: A Multi- LLM Agent (2024)

Abstract: Large Language Model (LLM) agents significantly extend the capabilities of standalone LLMs, empowering them to interact with external tools (e.g., APIs, functions) and complete various tasks in a self-directed fashion. The challenge of tool use demands that LLMs not only understand user queries and generate answers accurately but also excel in task planning, tool invocation, and result summarization. While traditional works focus on training a single LLM with all these capabilities, performance limitations become apparent, particularly with smaller models. To overcome these challenges, we propose a novel approach that decomposes the aforementioned capabilities into a planner, caller, and summarizer. Each component is implemented by a single LLM that focuses on a specific capability and collaborates with others to accomplish the task. This modular framework facilitates individual updates and the potential use of smaller LLMs for building each capability. To effectively train this framework, we introduce a two-stage training paradigm. First, we fine-tune a backbone LLM on the entire dataset without discriminating sub-tasks, providing the model with a comprehensive understanding of the task. Second, the fine-tuned LLM is used to instantiate the planner, caller, and summarizer respectively, which are continually fine-tuned on respective sub-tasks. Evaluation across various tool-use benchmarks illustrates that our proposed multi-LLM framework surpasses the traditional single-LLM approach, highlighting its efficacy and advantages in tool learning.

## introduction large language models (llms) have revolutionized natural language processing with remarkable proficiency in understanding and generating text. despite their impressive capabilities, llms are not without limitations. notably, they lack domain specificity, real-time information, and face challenges in solving specialized problems such as mathematics @xcite and program compilation (openai, 2023a). hence, integrating llms with external tools, such as api calls and python functions, becomes imperative to extend their capabilities and enhance the overall performance. consequently, llm agents have become a prominent area for both academia and industry, employing large language models to determine when and how to utilize external tools to tackle various tasks. in addition to exploring proprietary llms like gpt-4, researchers have also actively engaged in developing customizable agent systems by finetuning open-source llms on diverse tool-use datasets @xcite @xcite . the challenge of tool learning demands sufficiently large and complex llms. these models must not only comprehend user queries but also excel in task planning, tool selection and invocation, and result summarization @xcite . these capabilities draw upon different facets of the llms; for instance, planning relies more on reasoning ability, while tool selection and invocation demand legal and ac-curate request writing, and result summarization requires adept conclusion-drawing skills. while conventional approaches @xcite @xcite focus on training a single open-source llm with all these capabilities, notable performance limitations have been observed, especially with smaller open-source llms @xcite . moreover, the tools could be updated frequently in practical scenarios, when the entire llm requires potential retraining. to address these challenges, we propose a multi-llm agent framework for tool learning, α-umi 1 . as illustrated in figure 1 , α-umi decomposes the capabilities of a single llm into three components, namely planner, caller, and summarizer. each of these components is implemented by a single llm and trained to focus on a specific capability. the planner is designed to generate the rationale based on the current state of the system and weighs between selecting the caller or summarizer to generate downstream output, or even deciding to terminate the execution. the caller is directed by the rationale and responsible for invocating specific tools. the summarizer is guided by the planner to craft the ultimate user answer based on the execution trajectory. these components collaborate seamlessly to accomplish various tasks. compared to previous approaches, our modular framework has three distinct advantages. first, each component undergoes training for a designated role, ensuring enhanced performance for each capability. second, the modular structure allows for individual updates to each component as required, ensuring adaptability and streamlined maintenance. third, since each component focuses solely on a specific capability, potentially smaller llms can be employed. to effectively train this multi-llm framework, we introduce a novel global-to-local progressive fine-tuning strategy (glpft). first, an llm backbone is trained on the original training dataset without discriminating between sub-tasks, enhancing the comprehensive understanding of the toollearning task. three copies of this llm backbone are created to instantiate the planner, caller, and summarizer, respectively. in the second stage, the training dataset is reorganized into new datasets tailored to each llm's role in tool use, and continual 1 in astronomy, the name "α-umi" is an alias of the polaris star ( https://en.wikipedia.org/wiki/polaris ), which is actually a triple star system consisting of a brighter star (corresponding to the planner) and two fainter stars (corresponding to the caller and the summarizer). we employ llama-2 @xcite series to implement the llm backbone and evaluate our α-umi agent on several tool learning benchmarks @xcite . experimental results demonstrate that our proposed framework outperforms the single-llm approach across various model and data sizes. moreover, we show the necessity of the glpft strategy for the success of our framework and delve into the reasons behind the improved performance. finally, the results confirm our assumption that smaller llms can be used in our multi-llm framework to cultivate individual tool learning capabilities and attain a competitive overall performance. to sum up, this work makes three critical contributions. first, we demonstrate that small llms are weak tool learners and introduce α-umi, a multi-llm framework for building llm agents, that outperforms the existing single-llm approach in tool use. second, we propose a glpft fine-tuning strategy, which has proven to be essential for the success of our framework. third, we perform a thorough analysis, delving into data scaling laws and investigating the underlying reasons behind the superior performance of our framework. ## conclusion the objective of this paper is to address the challenge of designing and fine-tuning a single small llm to acquire the extensive abilities required for a tool learning agent. to this end, we introduce α-umi, a multi-llm tool learning agent framework that breaks down the tool learning task into three distinct sub-tasks delegated to three small llms: planner, caller, and summarizer. moreover, we propose a global-to-local progressive fine-tuning strategy and demonstrate its effectiveness in training the multi-llm framework. we evaluate our approach against single-llm baselines on four tool learning benchmarks, supplemented by various in-depth analyses, including a data scaling law experiment. our findings highlight the significance of our proposed method, validating that the system's design for decomposing tool learning tasks and the progressive fine-tuning strategy contribute to enhancing the upper-bound ability of a single llm. besides, we acknowledge the potential to utilize small llms to surpass the capabilities of an agent framework that relies on a single, larger llm.