Budget-Constrained Tool Learning with Planning (2024)

Abstract: Despite intensive efforts devoted to tool learning, the problem of budget-constrained tool learning, which focuses on resolving user queries within a specific budget constraint, has been widely overlooked. This paper proposes a novel method for budget-constrained tool learning. Our approach involves creating a preferable plan under the budget constraint before utilizing the tools. This plan outlines the feasible tools and the maximum number of times they can be employed, offering a comprehensive overview of the tool learning process for large language models. This allows them to allocate the budget from a broader perspective. To devise the plan without incurring significant extra costs, we suggest initially estimating the usefulness of the candidate tools based on past experience. Subsequently, we employ dynamic programming to formulate the plan. Experimental results demonstrate that our method can be integrated with various tool learning methods, significantly enhancing their effectiveness under strict budget constraints.1

heading: 1 Introduction
text: Tool learning (Schick et al., 2023; Yao et al., 2023b; Qin et al., 2023b), which uses external tools to extend the capability of large language models (LLMs), has achieved remarkable results on various types of tasks. For example, with the aid of external tools, LLMs may solve difficult mathematical problems with higher accuracy (Cobbe et al., 2021; Gao et al., 2023b), handle multimodal information (Shen et al., 2023; Lu et al., 2023), or interact with real-world applications (Song et al., 2023; Gur et al., 2023). To better resolve more complex user queries, previous studies have proposed different tool learning methods which allow
∗ Corresponding authors: P.Li (lipeng@air.tsinghua. edu.cn) and Y.Liu (liuyang2011@tsinghua.edu.cn).
1Code can be found at https://github.com/ THUNLP-MT/BTP.
the LLM to use multiple tools (Chen et al., 2023; Yao et al., 2023b; Paranjape et al., 2023; Shen et al., 2023; Qin et al., 2023b).
Despite their effectiveness, existing methods often overlook a critical aspect: utilizing tools incurs expenses, such as money and time, and users may have (implicit) budget constraints in real-world scenarios. Without meticulous management, the costs can rapidly exceed the acceptable threshold of a user. In scenarios where the expenses overshadow the benefits, even if the tool successfully addresses the intended problem, users may still perceive the solution as unsatisfactory due to the disproportionate incurred costs. Table 1 presents preliminary results for three strong tool learning approaches, ReAct (Yao et al., 2023b), DFSDT (Qin et al., 2023b), and ToT-DFS (Yao et al., 2023a), evaluated on the ToolBench (Qin et al., 2023b) dataset, with success defined strictly in terms of problem resolution within budget constraints. It is evident that all three methods experience significant declines in performance as the budget constraint tightens from unlimited to 20. Consequently, we argue that more
9039
efforts are worth devoting to the problem of budgetconstrained tool learning, which aims at resolving user queries within a given budget constraint.
One fundamental challenge in budgetconstrained tool learning is the allocation of the budget. However, determining the optimal budget allocation is not straightforward without a comprehensive understanding of the entire tool learning process. Taking Figure 1 as an example, the user query may be addressed using either Mathematica and MS Excel, or GNU Octave and MS Excel. While Mathematica, a commercial software, may provide a more precise solution than the open-source GNU Octave, we may incorrectly choose Mathematica without knowing the additional requirement for MS Excel. Unfortunately, acquiring such a comprehensive view is difficult. A simplistic approach is employing trial-and-error methods. However, each attempt introduces additional costs, further complicating adherence to the budget constraint.
To this end, we propose Budget-Constrained Tool Learning with Planning (BTP), a novel method for budget-constrained tool learning, which finds a preferable plan under the budget constraint before using the tools (Figure 1). First, for each candidate tool, we estimate its expected value and frequency constraint based on the past experience. The expected value measures how valuable the tool is for resolving the given query, while the frequency constraint limits the maximum number of times it can be used. Then, we try to find a plan that max-
imizes the sum of the expected value under the budget constraint. A plan specifies the viable tools and how many times each of them can be used. We regard this as a knapsack problem and use dynamic programming to resolve it. Finally, we apply the plan during the tool learning process. As shown in Table 1, our proposed method can be combined with different tool learning methods and improve their performance under a strict budget constraint.

heading: 2 Methodology
text: In this section, we first introduce a formulation of budget-constrained tool learning (Section 2.1). Then, we describe our proposed BTP in detail. Before using the tools, we try to find a preferable plan under the budget constraint (Section 2.2). Then, we apply the plan to the process of tool learning (Section 2.3). Moreover, we introduce a blacklist mechanism to further reduce the cost of tool learning (Section 2.4).

heading: 2.1 Budget-Constrained Tool Learning
text: Existing tool learning methods can be roughly divided into two lines. The first line finetunes the LLMs to make them capable of using tools without the documentation of the tools (Schick et al., 2023; Hao et al., 2023), while the second line lets the LLM select tools according to the documentation of the candidate tools (Shen et al., 2023; Hsieh et al., 2023; Qin et al., 2023b). In this paper, we mainly focus on the latter line, since it
can better support closed-source LLMs, including those which do not support fine-tuning. Typically, the tool learning process following this line is as follows: First, a set of n candidate tools T = {t1, . . . , ti, . . . , tn} are retrieved according to the user query using a retrieval model. Then an LLM is leveraged to select and invoke tools iteratively until a stop condition is met. Assume that this process runs for N times, we denote the tools invoked as TI = {ta1 , · · · , taj , · · · , taN }. Note that a tool may be invoked multiple times. Finally, the LLM produces the final output based on the user query and the results returned by the executed tools.
To conduct budget-constrained tool learning, we first propose a simplified and approximate but useful mathematical formulation for it. Formally, for any integer i ∈ [1, n], we use ci to represent the cost of using the tool ti once.2 In reality, this cost may be the estimated total cost spent on the tool in different aspects, including the money spent on using the tool and the LLM, as well as the consumed time for using the tool and the LLM:
ci = ci,tool + cLLM + Ti, (1)
where ci,tool represents the estimated cost of using the tool ti itself, cLLM is the estimated cost of using the API of the LLM once, and Ti is the estimated consumed time for using the tool and the LLM. We implicitly assume that all costs involved can be quantified using the same unit of measurement. For example, we can apply a conversion function to translate time costs into monetary terms.
Moreover, we use cs to represent the extra cost produced by the other factors such as the system prompt and the user prompt, and B to represent the budget constraint. Then, the target of budgetconstrained tool learning is to resolve a user query under the budget constraint:
cs + ∑
j
caj ≤ B. (2)
In practice, cs and ci are influenced by numerous factors, complicating the design of a feasible algorithm. To simplify the problem and make it manageable, we introduce an assumption that both cs and ci can be estimated in advance. The assumption does not significantly limit the generality of our approach. On the one hand, both of them can
2For simplicity, we also call ci as “the cost of ti” in the remaining part of this paper.
be estimated using their historical average values. On the other hand, cs can be more precisely estimated based on the price of the API of the LLM, the lengths of the system prompt and the user prompt for the current user query.
Currently, we do not take the cost arising from the external environments into consideration. For example, when the LLM uses an online shopping tool to buy a book for the user, the price of the book also consumes the overall budget and is difficult to be estimated in advance. Addressing this aspect is designated for future research.

heading: 2.2 Tool Usage Planning
text: In this work, we introduce a tool usage plan to offer an estimated overview of the tools likely to be utilized by the LLM, aiding in more effective budget allocation. The plan outlines the set of available tools and the maximum number of times each tool can be used.
However, it is difficult to precisely calculate how many times each candidate tool should be used for resolving a given user query. On the one hand, it is not trivial to estimate whether a candidate tool can provide valuable information for resolving the user query without using the tool. On the other hand, calculating the costs of every possible plan of tool usage for a given user query is unacceptable in terms of time complexity. Thus, we propose an approximate planning method for budget-constrained tool learning. In this planning method, we estimate whether the candidate tools can provide valuable information based on the past experience, and use dynamic programming to estimate how many times each candidate tool should be used.
Specifically, we first estimate the expected value of each candidate tool, which measures how valuable the results returned by the candidate tool are. In this work, we leverage the past experience of tool learning to estimate the expected value. Formally, the past experience E = {U1, . . . , Uk, . . . , Um} consists of a number of tool usages Uk which happened in the past. Each tool usage Uk = ⟨q, d(t), p, r⟩ consists of a user query q, the documentation d(t) of the used tool t, the input parameters p and the returned result r. We define a binary score function score(·) to judge the usefulness of r for resolving q, where score(Uk) = 0 or 1 means that r is unhelpful or helpful, respectively. In practice, score(·) is implemented as a LongFormer (Beltagy et al., 2020) based classifier.
Then, for each candidate tool ti, we fetch all tool
usages Uk related to ti in the past experience E to construct a set of tool usages Eti . We calculate the weighted average score of all Uk ∈ Eti as the expected value of ti for the user query qu:
v(qu, ti) =
∑ Uk∈Eti exp(sim(qu, q))score(Uk)
∑ Uk∈Eti exp(sim(qu, q)) ,
(3) where sim(qu, q) is the similarity between qu and q, which can be calculated using the retrieval model.
We also notice that repeatedly using the same tool too many times may be suboptimal within the budget constraint, since we expect that more diverse information can be obtained if different tools are used. Thus, we set an estimated frequency constraint F̃ (qu, ti) for each candidate tool ti and expect that ti should be used no more than F̃ (qu, ti) times. Specifically, if a tool frequently returns useless or erroneous messages, we should prevent the LLM from using it to reduce the cost of tool learning. Thus, we set a threshold τ and set F̃ (qu, ti) = 0 if v(qu, ti) < τ to filter out the useless tools based on the past experience. Otherwise, for each query q in the past experience where ti is used in the process of resolving q, we count the number of times F (q, ti) for which ti is used, and then calculate the weighted average of F (q, ti) as the estimated frequency constraint F̃ (qu, ti):
F̃ (qu, ti) =
∑ q exp(sim(qu, q))F (q, ti)
∑ q exp(sim(qu, q))
. (4)
Finally, we regard the planning process of budget-constrained tool learning as a knapsack problem and use dynamic programming to find a preferable plan of tool usage within the budget constraint. Formally, for each candidate tool ti, the plan specifies its corresponding frequency f(qu, ti), which means that we expect the LLM to use ti for at most f(qu, ti) times during the tool learning process. Since we expect that the returned results should contain valuable information as much as possible, the plan is determined by maximizing the sum of the expected value:
V =
n∑
i=1
f(qu, ti)v(qu, ti) (5)
Algorithm 1 Tool Usage Planning Input: budget constraint B; estimated cost for system prompt and user prompt cs; estimated cost ci, expected value v(qu, ti) and frequency constraint F̃ (qu, ti) for each tool in T = {t1, . . . , ti, . . . , tn} Output: frequency f(qu, ti)
1: R← B − cs 2: for j ← 0 to R do 3: V0,j ← 0 4: for i← 1 to n do 5: for j ← 0 to R do 6: Vi,j ← Vi−1,j 7: for k ← 1 to ⌊F̃ (qu, ti)⌋ do 8: if j ≥ kc(ti) then 9: Vi,j ← max(Vi,j , Vi,j−kc(ti) +
kv(qu, ti))
10: Vmax ← 0 11: for j ← 0 to R do 12: Vmax ← max(Vmax, Vn,j) 13: Trace back the dynamic programming process
to obtain the frequency f(qu, ti)
under the budget and frequency constraints:
n∑
i=1
f(qu, ti)ci ≤ R, (6)
f(qu, ti) ≤ F̃ (qu, ti), (7)
where R = B − cs. For simplicity, we assume that ci and R are positive integers.3 The details are shown in Algorithm 1.
The process of finding the plan also introduces extra cost. Fortunately, the cost is a near-constant value. Thus, we can simply extend cs to include this extra cost and keep the algorithm untouched.
Moreover, it is worth noting that our proposed dynamic programming method can be regarded as a searching algorithm. Searching-based methods are widely used in previously proposed tool learning methods such as DFSDT (Qin et al., 2023b) and ToT-DFS (Yao et al., 2023a). However, these methods are not feasible for searching tool usage plans as they need to invoke the tools during searching, which consume the budget.
3If ci and R are real numbers, we can use their approximate values and change them into integers. Theoretically, this approximation can achieve any precision. For the detailed proof, please refer to Appendix A.1.
Algorithm 2 Budget-Constrained Tool Learning with Plan Input: user query qu, set of candidate tools T = {t1, . . . , ti, . . . , tn}, frequency f(qu, ti) for each tool ti Output: returned result y
1: for each ti ∈ T do 2: if f(qu, ti) = 0 then 3: T ← T \ {ti} 4: j ← 0 5: while True do 6: j ← j + 1 7: taj , pj ← UseTool(qu, T , r1, . . . , rj−1) ▷Select a tool and determine its parameter 8: rj ← taj (pj) ▷ Invoke the tool to get result 9: f(qu, taj )← f(qu, taj )− 1 10: if f(qu, taj ) = 0 then 11: T ← T \ {taj} 12: if T = ∅ or IsSufficient(qu, r1, . . . , rj)
then ▷ If there are no available tools in T or the returned results are sufficient for resolving the user query 13: y ← Summarize(qu, r1, . . . , rj) ▷ Summarize to get the returned result 14: return y

heading: 2.3 Applying Plan to Tool Learning
text: After we obtain the frequency f(qu, ti) for each candidate tool ti, we apply the plan to the tool learning process. Specifically, before the LLM uses tools, we remove all the tools ti which satisfy f(qu, ti) = 0 from T since they should not be used in the process. Then, after each step j when the LLM uses the tool taj , we reduce the frequency f(qu, taj ) by 1. Once this frequency reaches zero, we remove taj from T , which indicates that taj should be no more used. We add extra information in the input context to inform the LLM that the tool taj is forbidden to use. If the LLM occasionally uses the forbidden tools, we prevent the LLM from the actual action of the tool usage and return an error message. Finally, if T becomes empty or the returned results are sufficient to draw a conclusion, we make a summary about all the returned results. The aforementioned process is shown in Algorithm 2.

heading: 2.4 Blacklist Mechanism
text: To further reduce the cost of tool learning, we introduce a blacklist mechanism to reduce the cost of
repeatedly using unhelpful tools. During the tool learning process, we temporarily build the blacklist according to the judgment about the returned results. Specifically, after the LLM uses a tool and receives the returned result, we use the classifier introduced in Section 2.2 to judge whether the returned result is helpful in resolving the user query.4 If a tool returns unhelpful results, we list it into the blacklist and forbid the LLM from using it. Specifically, we inform the LLM that the blacklisted tools are forbidden to use by giving the blacklisted tools in the input context during the tool learning process. If the LLM occasionally uses a blacklisted tool, we prevent the LLM from the actual action of the tool usage and return an error message.

heading: 3 Experiments

heading: 3.1 Setup
text: Data Preparation. Our experiments are mainly conducted on the ToolBench (Qin et al., 2023b) dataset, which is a tool learning dataset in English. We follow Qin et al. (2023b) and use 6 different subsets (I1-Inst, I1-Tool, I1-Cat, I2-Inst, I2-Cat and I3-Inst) in ToolBench as the test datasets. The subset I3-Inst includes 100 user queries, while each of the other subsets contains 200 user queries.
For the past experience used for planning the tool usage, we construct a dataset using the training dataset of ToolBench. First, we use ChatGPT (OpenAI, 2022) to judge whether the returned result is helpful for approximately 30k instances of tool usages in the training dataset. Then, we use these instances to train the LongFormer-based classifier proposed in Section 2.2 and use the classifier to classify the remaining instances in the training dataset. The total number of these two parts of instances is about 2.5M. During our experiments, all the past experience is obtained from this dataset.
Baselines. We combine our proposed method with the following baseline methods and compare the combined methods with the baseline methods without such combination:
1. ReAct (Yao et al., 2023b): Tool learning is performed using a chain-of-thought process (Wei et al., 2022).
2. DFSDT (Qin et al., 2023b): Tool learning is performed using a tree-based depth-first
4The cost of the classifier can also be added to ci.
searching algorithm. This allows the LLM to traverse back when encountering a failure.
3. ToT-DFS (Yao et al., 2023a): Tool learning is also performed using a depth-first searching algorithm. Different from DFSDT, ToT-DFS makes a vote across the candidate states and expands the best state at each step.
4. ReAct+Prompt, DFSDT+Prompt and ToTDFS+Prompt: The cost of the candidate tools and the remaining budget are added to the system prompt of the corresponding method.
Evaluation. We mainly use the following evaluation metrics to evaluate the effect of budget constraint on different methods:
1. Pass rate under Budget Constraint (PBC): The percentage of user queries which can be resolved by the LLM under the given budget constraint. When the solution given by the LLM exceeds the budget constraint, we count it as a failure.
2. Average Cost (AC): The average cost used for all user queries in the test dataset.
To better compare the quality of the generated solutions of our method with the baselines, we also evaluate the Pass Rate (PR) (Qin et al., 2023b), which measures the percentage of successfully resolved user queries, ignoring the budget constraint. Moreover, we also report the Rate of Failure due
to Budget Constraint (RFBC) to measure the percentage of user queries which the LLM fails to resolve because of the budget constraint.
Implementation Details. We use ChatGPT (OpenAI, 2022) as the backbone model for tool learning. For each user query, we retrieve 5 candidate tools using the semantic retrieval model provided by Qin et al. (2023b). Since the ToolBench dataset does not explicitly provide the cost of the tools, we randomly set an integer in [1, 10] as the cost ci for each tool in the tool library. For the budget constraint, we set the value of R to 20 unless explicitly specified. We set the threshold τ = 0.15 to filter out the useless tools. For further implementation details, please refer to Appendix A.2.
One might suggest that for a more accurate evaluation, a real system capable of calculating costs on the fly should be implemented. However, creating such a system involves navigating numerous real-world challenges, including the unpredictability associated with LLM behavior and network dynamics. Given that this work represents an initial exploration into budget-constrained tool learning, postponing the development of this system to future research seems reasonable. Additionally, it is crucial to clarify that the cost values ci used in the test set above are considered to be aggregate values encompassing all relevant costs, such as monetary expenses, time, and LLM usage, etc. The rationale behind this assumption and its validity is thoroughly discussed at the end of Section 2.1.

heading: 3.2 Main Results
text: The experimental results are shown in Table 2 and 3. When combining our proposed BTP with the baseline methods, the LLM can better resolve the user query within the budget constraint, regardless of which baseline method we use. Specifically, when combining BTP with the baseline methods, the average value of Pass rate under Budget Constraint (PBC) over six subsets of the ToolBench dataset increases by 9.3-53.9 points, and the Average Cost (AC) also decreases by 8.1-69.1 points. This is mainly attributed to two major aspects: First, combining BTP with the baseline methods does not significantly harm the Pass Rate (PR) even if we ignore the budget constraint. Second, the tool learning process cannot be terminated within the budget constraint for a substantial part of the user queries if we do not combine BTP with the baseline methods, while utilizing BTP can guarantee that the tool learning process can be terminated within the budget constraint.
Moreover, adding extra information to the system prompt to remind the LLM about the budget constraint is not effective for budget-constrained tool learning. For example, DFSDT+Prompt only outperforms DFSDT by 0.2 points on PBC. The average cost of DFSDT+Prompt is 73.3, which is much higher than the budget constraint R = 20.

heading: 3.3 Planned v.s. Actual Tool Usage
text: In this section, we compare the plan given by the dynamic programming algorithm and the actual tool usage performed by the LLM. As shown in Table 4, the average cost of the plan is 13.9 and we expect that the LLM uses the tools for 2.9 times on average on the I1-Inst subset. When BTP is combined with the baseline methods, both the actual average cost and the average number of times the LLM uses the tools are less than those given in the plan. The gap between the planned and the actual tool usage is within our expectation, since we do not actually use the tools when making the plan. Moreover, the size of the gap is reasonable, especially when we use a searching based algorithm (DFSDT or ToT-DFS) for tool learning.

heading: 3.4 Effect of Budget Constraint R
text: To investigate how the budget constraint R affects the performance of budget-constrained tool learning, we also conduct experiments on the I1-Inst subset under different budget constraints. The re-

heading: Method PR↑ RFBC↓

heading: Method AC #Use
text: sults are shown in Figure 2. On the one hand, the value of PBC increases as the budget constraint does, since a higher budget constraint gives more chances to the LLM for using the tools to resolve the user query. On the other hand, the average cost will reduce if we set a lower budget constraint, which indicates that we can adjust the cost of tool learning by modifying the budget constraint.

heading: 3.5 Effect of Threshold τ
text: We introduce a threshold τ in Section 2.2 to filter out useless tools. The effect of τ on the I1-Inst subset is shown in Figure 3. First, if we set a higher threshold, the average cost decreases since fewer tools are available for the LLM to use. Second, the value of PBC increases when τ < 0.15, which indicates that setting a proper threshold can filter out the useless tools and improve the quality of the generated solutions. However, the value of PBC
decreases when τ > 0.15, which indicates that a high threshold may incorrectly filter out some useful tools and worsen the quality of the solutions. Therefore, we set τ = 0.15 in our experiments.

heading: 3.6 Effect of Blacklist Mechanism
text: To validate the effectivenss of the blacklist mechanism, we conduct an extra experiment on the I1-Inst subset where the blacklist mechanism is removed. As shown in Table 5, removing the blacklist does not significantly affect the value of PBC. However, the average cost increases, which indicates that the blacklist mechanism can reduce the cost of repeatedly using unhelpful tools.
Moreover, to investigate how the LongFormerbased classifier affects the performance of budgetconstrained tool learning, we conduct an extra experiment on the I1-Inst subset where we directly use the LLM to judge whether the returned result is helpful in the blacklist mechanism. Experimental results show that the value of PBC and the average cost do not significantly change if we use the LLM to judge the helpfulness of the returned results, even if the LLM can give the judgments more accurately than the classifier.5 However, using the LongFormer-based classifier can reduce the number of times of using the LLM in practice, and thus we use the LongFormer-based classifier for our proposed BTP.

heading: 4 Related Work
text: Tool learning aims to extend the capability of LLMs using external tools (Schick et al., 2023; Yao et al., 2023b; Qin et al., 2023a,b). With the aid of external tools, LLMs may be capable of obtaining
5We randomly sample 100 returned results and use the classifier and the LLM to give the judgments, and we find the accuracy of the classifier and the LLM is 91% and 94%, respectively.

heading: Blacklist PBC↑ AC↓
text: recent information (Liu et al., 2023), solving difficult math problems (Cobbe et al., 2021; Gao et al., 2023b), handling multimodal information (Huang et al., 2023; Lu et al., 2023) or solving domainspecific problems (Jin et al., 2023).
The basic approach of tool learning is to add the documentation of the tool to the input context of the LLM and then let the LLM use the tool to resolve the user query (Shen et al., 2023; Hsieh et al., 2023; Qin et al., 2023b). To handle complex user queries which require multiple steps to resolve, Yao et al. (2023b) propose ReAct, which performs tool learning in a chain-of-thought (Wei et al., 2022) process. To increase the probability of successfully resolving the user query, Qin et al. (2023b) propose DFSDT, a tool learning algorithm which is based on a tree-based depth-first searching algorithm and allows the LLM to traverse back after a failure. Similarly, Zhuang et al. (2023) also propose a treebased algorithm for tool learning, which is based on the A* searching algorithm. Moreover, Gao et al. (2023a) and Gao et al. (2023c) add a reflection mechanism to tool learning, which allows the LLM to retry after a failure.
However, the aforementioned studies focus on how to better resolve complex user queries, but do not take the budget constraint into consideration. In contrast, our proposed method aims to improve the performance of tool learning within the budget constraint of tool usage.
This work is also highly related to Kim et al. (2023), who use different tools in parallel to im-
prove the efficiency of tool learning. However, their work mainly focuses on time efficiency rather than budget constraint.

heading: 5 Conclusion
text: In this work, we propose BTP, a novel method for budget-constrained tool learning. The core idea of BTP is to find a preferable plan under the budget constraint before using the tools. We also introduce a blacklist mechanism to further reduce the cost of tool learning. Experimental results show that BTP can be combined with different tool learning methods and improve the performance of budgetconstrained tool learning.

heading: Limitations
text: In this work, we propose a simplified yet effective approximation to model budget-constrained tool learning. As discussed in Section 2.1, our approach does not account for costs incurred by external factors, which are challenging to estimate based on previous experience of tool learning. Exploring a more comprehensive mathematical model that includes these costs is an avenue for future research. Furthermore, our experiments are carried out in a simulated setting, where the costs associated with the tools are not based on real-world data but are simulated instead. Creating similar conditions in a real-world context to accurately estimate tool costs poses numerous challenges. Therefore, developing techniques to estimate real-world tool costs is an objective for future research.

heading: Acknowledgments
text: This work is supported by the National Natural Science Foundation of China (No. 61925601, 62276152). We thank Zonghan Yang for providing helpful suggestions about the idea of this work. We also appreciate the anonymous reviewers for their valuable comments.

heading: A Appendix

heading: A.1 Proof of Approximation of ci and R
text: In this proof, we suppose that the tool learning process consists of a reasonable number of steps, and we use N to represent the number of steps. For any ε > 0, we set λ as:
λ = N + 1
ε (8)
Then, we use c̃i and R̃ to denote the approximate value of ci and R. c̃i and R̃ are defined as below:
c̃i = ⌈λci⌉ λ , (9) R̃ = ⌊λR⌋ λ . (10)
Based on the equations above, c̃i and R̃ satisfy λc̃i ∈ Z and λR̃ ∈ Z.
Besides, c̃i and R̃ satisfy
R− R̃ = λR− ⌊λR⌋ λ
< 1
λ
= ε
N + 1 ,
(11)
c̃i − ci = ⌈λci⌉ − λci
λ
< 1
λ
= ε
N + 1 .
(12)
Thus, the approximate remaining budget R̃ − N∑ j=1 c̃aj satisfies
R̃− N∑
j=1
c̃aj
> R− ε N + 1
− N∑
j=1
( caj +
ε
N + 1
)
= R− N∑
j=1
caj − ε.
(13)
Moreover, since R̃ = ⌊λR⌋λ ≤ R and c̃i = ⌈λci⌉ λ ≥ ci, R̃−
N∑ j=1 c̃aj also satisfies
R̃− N∑
j=1
c̃aj ≤ R− N∑
j=1
caj . (14)
Thus, we conclude that ∣∣∣∣∣∣  R̃− N∑
j=1
c̃aj
 −  R− N∑
j=1
caj   ∣∣∣∣∣∣ < ε. (15)
This indicates that if ci and R are real numbers, we can use λc̃i and λR̃ as their integral approximation. If the number of the total steps taken for tool learning N is reasonable, such approximation can achieve sufficiently high accuracy if ε is small enough.

heading: A.2 Further Implementation Details
text: The training process of the LongFormer-based classification model is implemented on top of the Transformers (Wolf et al., 2020) library. The model is initialized with longformer-base-40966, which has approximately 149M parameters. The model is trained for 10 epochs with the learning rate of 10−5. The batch size is set to 8. The model is trained on a single NVIDIA GeForce RTX 3090 GPU for 5.5 hours.
Moreover, the system prompt used during in the tool learning process is constructed based on the prompt for solution path annotation introduced in Qin et al. (2023b). We add extra content presented in Figure 4 in the system prompt if some of the tools are forbidden to use. Specifically, we append the prompt Part I to the system prompt if at least one candidate tool is removed from T . We append the prompt Part II to the system prompt if at least one candidate tool is blacklisted. We append the prompt Part III to the system prompt if all the candidate tools are forbidden to use (i.e. removed from T or blacklisted).

heading: A.3 Detailed Experimental Results of Table 3
text: In this section, we report the detailed experimental results of Table 3 in Table 6 and 7.

heading: A.4 Case Study
text: In this section, we conduct a case study on the I1Inst subset. As shown in Figure 5, the baseline method DFSDT repeatedly uses the tool 2 which always returns error messages. DFSDT also uses the tools 3 and 4 which return repetitive or irrelevant results. As the result, DFSDT finishes the tool learning process with the total cost of 31. It only finds the list of supported regions but fails to find the trending keywords in the United Kingdom.
6https://huggingface.co/allenai/longformer-base-4096
When combining our proposed BTP with DFSDT, the LLM uses the tool 5 within the plan and successfully finds the trending keywords in the United Kingdom. Thus, DFSDT+BTP successfully resolves the user query with the total cost of 8. This indicates that our proposed BTP can help the LLM resolve the user query within the budget constraint.

heading: A.5 Licenses of Tools, Models and Datasets
text: The licenses of the Transformers (Wolf et al., 2020) library, the LongFormer (Beltagy et al., 2020) model and the ToolBench (Qin et al., 2023b) dataset are Apache-2.0.
User Query: I'm curious about the trending search keywords on Google for today in the United Kingdom. Can you please fetch the list of trending keywords? Additionally, I would like to know if a specific region is supported. Can you provide me with the list of supported regions?
Candidate Tools (Cost): 1. regions_for_google_trends (2) 2. trendings_for_google_trends (9) 3. get_geo_map_for_regions_for_trends_keywords_in_different_regions (3) 4. get_trend_keyword_for_trends_keywords_in_different_regions (8) 5. keywordsearch_for_google_keyword_scraper (6)

heading: Future Work
text: Exploring a more comprehensive mathematical model that includes these costs is an avenue for future research. Furthermore, our experiments are carried out in a simulated setting, where the costs associated with the tools are not based on real-world data but are simulated instead. Creating similar conditions in a real-world context to accurately estimate tool costs poses numerous challenges. Therefore, developing techniques to estimate real-world tool costs is an objective for future research