Planning and Editing What You Retrieve for Enhanced Tool Learning (2024)

Abstract: Recent advancements in integrating external tools with Large Language Models (LLMs) have opened new frontiers, with applications in mathematical reasoning, code generators, and smart assistants. However, existing methods, relying on simple one-time retrieval strategies, fall short on effectively and accurately shortlisting relevant tools. This paper introduces a novel PLUTO (Planning, Learning, and Understanding for TOols) approach, encompassing “Plan-and-Retrieve (P&R)” and “Edit-and-Ground (E&G)” paradigms. The P&R paradigm consists of a neural retrieval module for shortlisting relevant tools and an LLM-based query planner that decomposes complex queries into actionable tasks, enhancing the effectiveness of tool utilization. The E&G paradigm utilizes LLMs to enrich tool descriptions based on user scenarios, bridging the gap between user queries and tool functionalities. Experiment results demonstrate that these paradigms significantly improve the recall and NDCG in tool retrieval tasks, significantly surpassing current state-of-the-art models.

## introduction the community has shown increasing interest in integrating external tools and interfaces with llms since tools often provide complementary functionalities in complex tasks such as dialogues @xcite , mathematical reasoning @xcite , and code generation @xcite . to realize tool augmentation, llm systems typically employ a retriever mechanism to select relevant tools from a candidate pool and write function api calls based on the retrieved tools. the introduction of external tools also allows llms to address complicated user queries. @xcite show that llms, incorporating simple tools, achieve better performance on downstream tasks. gupta and kembhavi 2023 attempt to solve compositional visual figure 1 : comparison between conventional retrieveand-read and pluto paradigm. unlike the conventional one-time retrieve-and-read paradigm that may lead to retrieving an ineffective set of tools, pluto efficiently parses a complex query and distills it into actionable sub-queries that facilitate accurate retrieval of appropriate tools. nonetheless, emergent approaches for llms with tool integration present several distinct challenges. one primary concern is that current llm agents still adopt simple retrieval-and-read strategies @xcite , lacking the dynamic adaptability required for addressing complex queries. as shown in fig. 1 , the conventional retrieve-and-read paradigm, solely relying heavily on similarity matching, falls short of retrieving diverse types of tools to address a complex user query. this limitation is further exacerbated by the semantic gap between user queries and tool descriptions. particularly, user queries can be ambiguous and complex, often requiring a deep understanding of the user's intent and the context of the query @xcite . on the other hand, human-written tool descriptions can be abstract and lack essential details for deciding their utilities, leading to a mismatch between what the user needs and what the tool is perceived to offer. additionally, current models tend to finetune on static tools, posing challenges to their robustness in the ever-evolving tool environment where new tools emerge and existing ones become obsolete @xcite . there is limited research on retrieval enhancement strategies in non-finetuned settings. these gaps highlight crucial areas for future research and development in llm and tool integration. in this paper, we leverage llm's world knowledge and reasoning ability to augment the retrieval and utility of tools in response to complex user queries, by designing a novel framework pluto (planning, learning, and understanding for tools) foot_0 . our first contribution is the introduction of a novel plan-and-retrieve for tool integration. while prior retrieve-and-read approaches only retrieve once at the beginning, our plan-and-retrieve paradigm is designed to adaptively adjust its strategies based on the outcomes of its selfevaluations, ensuring a continuous refinement of the tool selection process. this paradigm is structured into two core modules. the first module, the retriever, leverages neural (dense) retrieval techniques @xcite and lm-likelihood scoring mechanisms @xcite to efficiently shortlist relevant tools from a vast pool of candidates in response to a user query. this process ensures that the most pertinent tools are identified quickly, laying a foundation for more effective tool utilization. inspired by recent advancements of adaptive retrieval-augmented generation (rag; @xcite , we design an llm-based query planner that autoregressively decomposes complex user queries into manageable, task-oriented actions as the second module. following the decompositions, the query planner selects the most suitable ones from the retrieved tools. it goes further by evaluating the effectiveness of selected tools and proposing the next action toward addressing the user query. this plan-and-retrieve paradigm operates dynamically, embodying a sophisticated feedback loop that interlinks the retrieval of tools with subsequent refinement, evaluation, and planning stages. our second contribution is the proposal of editand-ground paradigm that utilizes user queries' rich contextual information and llm's extensive world knowledge for enriching descriptions of tool functionalities. research has shown that informed tool documentations can enhance the interaction between llms and tools @xcite . however, documenting tool functionalities at scale can be tedious for humans. @xcite show llms can follow instructions and optimize realworld applications. leveraging the optimization ability of the llm, our tool-grounding agent optimizes under-informative tool descriptions by learning and abstracting information from tools' user scenarios. by editing tool descriptions to make them more aligned with tools' user scenarios, the agent bridges the gap between user queries and tool functionalities, enhancing the overall effectiveness of tool retrieval and usage. in conclusion, this paper advances the field of tool integration with llms by introducing the novel plan-and-retrieve and edit-and-ground paradigms. experiments show that our paradigms improve the recall and ndcg of tool retrieval tasks, significantly outperforming current state-of-the-art (sota). our downstream evaluation suggests that the improvement gained during the retrieval phase, such as higher accuracy and relevance in responses, significantly contribute to successfully addressing the user queries. however, the massive action space and tool functionality variance in tool-oriented tasks pose challenges to llms during planning. an erroneous step in planning can lead to a faulty loop, such as continually calling a tool in the wrong way or hallucinating non-existing tools. our plan-and-retrieve paradigm, employing furtherest planning assessment @xcite , enforces reasonable and goal-oriented decompositions of user queries. the recently proposed react framework @xcite asks llm to plan future actions based on its observation of environments. in the context of tooloriented tasks, the plan builds upon the execution results of retrieved tools. such practice running and verifying each tool at retrieval time can be expensive and time-consuming at scale. in contrast, our plan-and-retrieve paradigm fully leverages llm's internal representation of world knowledge to propose plans in response to user queries, therefore guaranteeing both time and cost efficiency as an execution-free paradigm. tool learning. tool learning refers to the process where llms not only process and generate language-based responses but also learn to interact with and utilize external tools to enhance their capabilities @xcite @xcite @xcite @xcite @xcite . by incorporating tools, llms can offer solutions in various areas, including visual-language processing @xcite , mathematical reasoning @xcite , and tasks in specialized domains @xcite . however, previous research on tool learning mainly focused on teaching llms to use tools, but ignores the importance of shortlisting relevant tools. in this paper, we focus on using llms to improve the tool retrieval process. in contrast to previous researches that heavily rely on finetuning retrievers @xcite to shortlist tools, we propose a novel edit-and-ground paradigm, leveraging llms' parametric knowledge to learn and create more informative descriptions for tools. this approach seeks to provide richer information for the retriever, leading to more accurate retrieval. ## task and data we hereby formulate the task of tool retrieval and describe the dataset for this task. ## method in this section, we describe the proposed framework to integrate tools with llms for addressing complex user queries. our methodology is grounded in two innovative paradigms: the planand-retrieve (p&r; §4.2) and edit-and-ground (e&g; §4.3). we discuss the coordination between two paradigms in §4.4. ## experiments in this section, we evaluate the proposed pluto framework for tool retrieval and compare it with baseline methods. we will delve into the details of our experimental setup ( §5.1), discuss the results ( §5.2) obtained, and perform an ablation study to understand strengths of different components ( §5.3). by executing the retrieved tools, we evaluate their correctness in addressing user queries to further validate our findings ( §5.4). we present case studies to qualitatively evaluate the strength of pluto framework ( §5.5). ## conclusion we introduced pluto, a framework composed of the plan-and-retrieve and edit-and-ground paradigms, which marks a distinctive departure from traditional methodologies, setting a new stan-dard for tool retrieval. the empirical results illustrate the superiority of pluto across critical retrieval performance metrics as well as pass rate in real-world tool-use evaluation. these metrics collectively attest to the model's efficacy in identifying relevant tools and successfully addressing complex user queries. we hope the adaptability and efficiency of pluto can empower a multitude of domains where accurate and timely retrieval of tools is paramount. from autonomous scientific discovery to software development, the potential applications are as diverse as they are impactful. the edit-and-ground (e&g) may be executed to further optimize the descriptions. however, due to the cost, we currently set a relatively loose stop criterion that is enough to demonstrate the effectiveness of the presented method.