Budget-Constrained Tool Learning with Planning (2024)

Abstract: Despite intensive efforts devoted to tool learning, the problem of budget-constrained tool learning, which focuses on resolving user queries within a specific budget constraint, has been widely overlooked. This paper proposes a novel method for budget-constrained tool learning. Our approach involves creating a preferable plan under the budget constraint before utilizing the tools. This plan outlines the feasible tools and the maximum number of times they can be employed, offering a comprehensive overview of the tool learning process for large language models. This allows them to allocate the budget from a broader perspective. To devise the plan without incurring significant extra costs, we suggest initially estimating the usefulness of the candidate tools based on past experience. Subsequently, we employ dynamic programming to formulate the plan. Experimental results demonstrate that our method can be integrated with various tool learning methods, significantly enhancing their effectiveness under strict budget constraints.

## introduction tool learning @xcite @xcite , which uses external tools to extend the capability of large language models (llms), has achieved remarkable results on various types of tasks. for example, with the aid of external tools, llms may solve difficult mathematical problems with higher accuracy @xcite , handle multimodal information @xcite , or interact with real-world applications @xcite . to better resolve more complex user queries, previous studies have proposed different tool learning methods which allow @xcite dataset. our proposed budget-constrained tool learning with planning (btp) reduces the cost of tool learning and reaches competitive pass rate, significantly improving the performance under a strict budget constraint. despite their effectiveness, existing methods often overlook a critical aspect: utilizing tools incurs expenses, such as money and time, and users may have (implicit) budget constraints in real-world scenarios. without meticulous management, the costs can rapidly exceed the acceptable threshold of a user. in scenarios where the expenses overshadow the benefits, even if the tool successfully addresses the intended problem, users may still perceive the solution as unsatisfactory due to the disproportionate incurred costs. one fundamental challenge in budgetconstrained tool learning is the allocation of the budget. however, determining the optimal budget allocation is not straightforward without a comprehensive understanding of the entire tool learning process. taking figure 1 as an example, the user query may be addressed using either mathematica and ms excel, or gnu octave and ms excel. while mathematica, a commercial software, may provide a more precise solution than the open-source gnu octave, we may incorrectly choose mathematica without knowing the additional requirement for ms excel. unfortunately, acquiring such a comprehensive view is difficult. a simplistic approach is employing trial-and-error methods. however, each attempt introduces additional costs, further complicating adherence to the budget constraint. to this end, we propose budget-constrained tool learning with planning (btp), a novel method for budget-constrained tool learning, which finds a preferable plan under the budget constraint before using the tools (figure 1 ). first, for each candidate tool, we estimate its expected value and frequency constraint based on the past experience. the expected value measures how valuable the tool is for resolving the given query, while the frequency constraint limits the maximum number of times it can be used. then, we try to find a plan that max-imizes the sum of the expected value under the budget constraint. a plan specifies the viable tools and how many times each of them can be used. we regard this as a knapsack problem and use dynamic programming to resolve it. finally, we apply the plan during the tool learning process. as shown in ## methodology in this section, we first introduce a formulation of budget-constrained tool learning (section 2.1). then, we describe our proposed btp in detail. before using the tools, we try to find a preferable plan under the budget constraint (section 2.2). then, we apply the plan to the process of tool learning (section 2.3). moreover, we introduce a blacklist mechanism to further reduce the cost of tool learning (section 2.4). ## related work tool learning aims to extend the capability of llms using external tools @xcite @xcite . with the aid of external tools, llms may be capable of obtaining blacklist pbc↑ ac↓ none 57.5 10.2 chatgpt 58.0 9.7 longformer 58.0 9.6 the basic approach of tool learning is to add the documentation of the tool to the input context of the llm and then let the llm use the tool to resolve the user query @xcite @xcite . to handle complex user queries which require multiple steps to resolve, @xcite propose react, which performs tool learning in a chain-of-thought @xcite process. to increase the probability of successfully resolving the user query, @xcite propose dfsdt, a tool learning algorithm which is based on a tree-based depth-first searching algorithm and allows the llm to traverse back after a failure. similarly, zhuang et al. (2023) however, the aforementioned studies focus on how to better resolve complex user queries, but do not take the budget constraint into consideration. in contrast, our proposed method aims to improve the performance of tool learning within the budget constraint of tool usage. this work is also highly related to @xcite , who use different tools in parallel to im-prove the efficiency of tool learning. however, their work mainly focuses on time efficiency rather than budget constraint. ## conclusion in this work, we propose btp, a novel method for budget-constrained tool learning. the core idea of btp is to find a preferable plan under the budget constraint before using the tools. we also introduce a blacklist mechanism to further reduce the cost of tool learning. experimental results show that btp can be combined with different tool learning methods and improve the performance of budgetconstrained tool learning.