Large Language Models as Tool Makers
Authors: Tianle Cai
Xuezhi Wang
Tengyu Ma
Xinyun Chen
Denny Zhou
Venue: ICLR 2024 poster (2024)
Keywords: large language models
tool making
tool using
serving efficiency
Primary area: general machine learning (i.e., none of the above)

Abstract: Recent research has highlighted the potential of large language models (LLMs)
to improve their problem-solving capabilities with the aid of suitable external
tools. In our work, we further advance this concept by introducing a closed-
loop framework, referred to as LLMs A s Tool Makers (LATM), where LLMs
create their own reusable tools for problem-solving. Our approach consists of two
phases: 1) tool making: an LLM acts as the tool maker that crafts tools for a set
of tasks, where a tool is implemented as a Python utility function. 2) tool using:
another LLM acts as the tool user, which applies the tool built by the tool maker
for problem-solving. The tool user can be either the same or a different LLM
from the tool maker. On the problem-solving server side, tool-making enables
continual tool generation and caching as new requests emerge. This framework
enables subsequent requests to access cached tools via their corresponding APIs,
enhancing the efficiency of task resolution. Beyond enabling LLMs to create their
own tools, our framework also uncovers intriguing opportunities to optimize the
serving cost of LLMs: Recognizing that tool-making requires more sophisticated
capabilities, we assign this task to a powerful, albeit resource-intensive, model.
Conversely, the simpler tool-using phase is delegated to a lightweight model. This
strategic division of labor allows the once-off cost of tool-making to be spread
over multiple instances of tool-using, significantly reducing average costs while
maintaining strong performance. Furthermore, our method offers a functional
cache through the caching and reuse of tools, which stores the functionality of
a class of requests instead of the natural language responses from LLMs, thus
extending the applicability of the conventional cache mechanism. We evaluate
our approach across various complex reasoning tasks, including Big-Bench tasks.
With GPT-4 as the tool maker and GPT-3.5 as the tool user, LATM demonstrates
performance equivalent to using GPT-4 for both roles, but with a significantly
reduced inference cost.

Published as a conference paper at ICLR 2024
LARGE LANGUAGE MODELS AS TOOL MAKERS
Tianle Cai Xuezhi Wang'
Tengyu 1.3
Xinyun Chen' '
Denny Zhou'
Gooogle Deepmind Princetonn University Sttanford University
ABSTRACT
Recent research has highlighted the potential of large language models (LLMs) to improve their problem-solving capabilities with the aid of suitable external tools. In our work, we further advance this concept by introducing a closed- loop framework, referred to as LLMs As Tool Makers (LATM), where LLMs create their own reusable tools for problem-solving. Our approach consists of two phases: 1) tool making: an LLM acts as the tool maker that crafts tools for set tasks, where tool is implemented as a Python utility function. 2) tool using: another LLM acts the tool user, which applies the tool built by the tool maker for problem-solving. The tool user can be either the same or a different LLM from the tool maker. On the groblem-solving server side, tool-making enables continual tool generation and caching as new requests emerge. This framework enables subsequent requests access cached tools via their corresponding APIs, enhancing the efficiency of task resolution. Beyond enabling L.LMs to create their own tools, our framework also uncovers intriguing opportunities optimize the serving cost of LLMs: Recognizing that tool-making requires more sophisticated capabilittes, we assign this task to a powerful, albeit resource- resource-intensive, model. Conversely, the simpler tool-using phase is delegated to lightweight model. This strategic division of labor allows the once-off cost of tool-making to be spread over multiple instances of tool-using, significantly reducing average costs while maintaining strong performance. Furthermore, our method offers a functional cache through the caching and reuse of tools, which stores the functionality of a class of requests instead of the natural language responses from LLMs, thus extending the applicability of the conventional cache mechanism. We evaluate our approach across various complex reasoning tasks, including Big- Big-Bench tasks. With GPT-4 the tool maker and GPT-3.5 as the tool user, LATM demonstrates performance equivalent to using GPT-4 for both roles, but with a significantly reduced inference cost. The codebase can be found in https: / gi github. com/ ctll111/ Be11111/LLM-BR LLM-TToMMTTToo 1Maker.
INTRODUCTION
Large language models (LLMs) have demonstrated outstanding capabilities across broad array of NI.P tasks (Brown et al., 2020; Chowdhery et al., 2022; Zhang et al., 2022; Hoffmann et al., 2022; OpenAI, 2023; Google, 2023) and have even shown promising signs of achieving certain aspects of artificial general intelligence (Bubeck et al., 2023; Kosinski, 2023). Moreover, analogous to the evolution of human intelligence, recent research has unveiled the potential of augmenting LLMs with external tools, thereby significantly enhancing their problem-solving capacities and efficiencies (Yao al., 2023; Liu et al., 2023; Parisi et al., 2022: Schick al., 2023).
However, the applicability of these tool-using methods is largely contingent on the availability of suitable tools. According to the lessons learned from the evolutionary milestones of humans, crucial turning point was that humans got the ability to fabricate their own tools to address emerging challenges. Inspired by the importance of tool-making for humans, in this work, we embark on an initial exploration to apply this evolutionary concept to the realm of LLMs. We propose closed-loop framework, which we term as LLMs As Tool Makers (LATM), enables LLMs to generate their own reusable tools to tackle new tasks. Our approach comprises two key stages: 1) tool making: an LLM,
"Wrk done as a Student Researcher at Google Deepmind. Work done Visiting Researcher Google Deepmind.

---

Published as a conference paper at ICLR 2024
Use a few instances make too/
Reuse the tool to solve similar instances
Requests
Tool Maker
Reusable Tool
Tool User
The closed-loop framework of LATM. In situations with numerous problem-solving requests, directly utilizing powerful LLM to solve all the instances result in high costs. On the other hand, lightweight models are cost-effective but usually struggle with complex tasks. LATM leverages the strengths of both models by employing a powerful model as the tool maker generate reusable tools (implemented as Python functions) for tasks observed in the requests and pass the tool cost-effective tool user model for solving similar instances in the following requests. This approach allows the lightweight model to achieve performance comparable to the powerful model while maintaining greater cost-efficiency.
Figure 1:
known as the tool maker, designs tools (implemented as Python functions) specifically for a given task. 2) tool using: another LLM referred to as the tool user, which can be the same the tool maker, applies the tools to handle new requests. The two-stage design allows LATM to allocate jobs in each stage to the most suitable LLM. Specifically, the tool-making process, which requires high degree of capability, can be assigned to a powerful albeit -ssource-intensive model (e.g.. GPT-4). On the other hand, the tool-using process, which is comparatively simpler, can be assigned lightweight and cost-effective model (e.g., GPT-3.5 Turbo). This approach not only enhances the problem-solving capabilities L.LMs, but also significantly reduces the average computational cost of addressing a series of tasks.
As the tool mollmaainn process needs to executed only once for given functionality, the resulting tools can be reused across different task instances. This approach paves the way for scalable and cost-efficient solution for handling complex task. For instance, consider a task where a user ask the LLM to schedule meeting that works for everyone (e.g., in email conversations). Lightweight models like GPT-3.5 Turbo often struggle with such tasks that involve complex arithmetic reasoning. In contrast, more powerful models (e.g., GPT-4) can find the correct solutions, despite that the inference costs become much higher. LATM overcomes these hurdles by employing a powerful yet expensive model as the tool maker, and passing to cost-effective model as the tool user, for subsequent usage. After the tool has been forged, the lightweight tool user can use it to solve the task efficiently with high performance. This paradigm can similarly be applied recurring tasks in various workflows, such as parsing and analyzing web documents into specific data formats or formulating routing plans that satisfy several custom requirements, or being used solve popular games like the 24-game, Sudoku.
In the context serving cost reduction, LATM introduces the opportunity of creating a functional cache for the LLM server. Specifically, consider a streaming setting where the LLM server continu- ously receives a sequence of requests. Traditional cache systems, such as GPTCache (Zilliz, 2023), store the responses generated by the LLMs and reuse them for textually similar requests. However, with the capacity for mol-mnainng that LATM introduces, the system can store tools crafted by the tool maker and reuse them for functionally analogous requests. This novel approach, combined with the strategic division of labor between the tool maker and tool user, has the potential to considerably reduce the average cost of serving sequence of requests while maintaining high performance.
2

---

Published as a conference paper at ICLR 2024
Our experiments validate the effectiveness of this approach a range of complex reasoning tasks, including several challenging Big-Bench tasks (Srivastava et al., 2022). The results show that LATM achieve performance on par with more resource-intensiiy models while being more cost-effective. This novel approach to L.Mss, which mimics the evolutionary leap of humans in creating and using tools, opens up exciting possibilities for growing community with L.M--generated tools.
RELATED WORK
Chain of thought (CoT). Recently, significant progress has been made in enhancing the problem- solving abilities of large language models (LLMs) for complex tasks. For instance, CoT prompt- ing (Wei et al., 2022; Wang et al., 2022) has been proposed to bolster LLM reasoning capabilities, demonstrating improved performance across various reasoning and natural language processing tasks. CoT is typically articulated through natural languages (Ling et al., 2017: Cobbe et al., 2021; Suzgun al., 2022; Shi al., 2022; Zhou et al., 2022), yet it might also be effectively represented using programming languages (Amini et al., 2019; Austin et al., 2021; Nye et al., 2021; Chowdhery et al., 2022: Gao et al., 2023; Chen al., 2022). More recently, Arora al. (2023) proposed using LLMs to generate structured views over documents, balancing quality and cost by ensembling extractions from multiple synthesized functions. Our method shares similar spirit with Arora al. (2023) in managing cost and quality trade-offs but focuses on more general use cases.
Augmenting language models with tools. Recent works have explored the potential of using external tools supplement LLMs' capabilities for complex tasks. Yao et al. (2023); Yang et al. (2023) proposed augmenting reasoning traces with task-specific actions in LLMs, enabling models to reason and act synergistically. Various studies (Liu al., 2023; Parisi et al., 2022: Schick et al., 2023; Shen et al., 2023; Lu et al., 2023; Paranjape et al., 2023; Liang et al., 2023) have demonstrated that supplementing LLMs with tools, such as calculators, search engines, translation systems, calendars, or even API calls on other models, can help solve tasks that are not easily addressed by L.LMs alone.
Similar to LATM, methods like Chameleon (Lu et al., 2023) also incorporate Python executors the pipeline. However, their primary focus is on using Python executors to accurately solve sub-steps involving arithmetic reasoning, similar to Gao al. (2023); Chen et al. (2022). In contrast, we use Python executors to create reusable tools for addressing other task instances. Furthermore, the separation of the tool maker and tool user enables the use a lightweight model for most inferences, thus enhancing efficiency and cost-effectiveness in LATM.
Adaptive generation in language models. In addition, recent research has proposed methods to adaptively control decoding in LLMs to improve text generation efficiency (Leviathan al., 2022; Chen et al., 2023a; Xia et al., 2023). Speculative decoding is based on the notion that generating text tokens (a more expensive process) can be expedited with a faster yet less powerful model while approximating the performance of larger, costlier models by using them to score generated tokens (a much faster process). Our approach of passing tools from a more expensive model to a smaller, faster model also shares similar spirit of adaptive computing. Instead of altering the decoding procedure, we transfer newly generated tools between models to boost both the performance and efficiency of an LLM in solving tasks.
Language model cascades. There is recent evidence that LLMs can enable repeated interactions and that multiple LLMs can be combined to extend their capabilities further (Wu et al., 2022: Zhou et al., 2022: Dohan et al., 2022; Chen et al., 2023c). Also, Chen al. (2023b) demonstrated that identifying optimal LLM combinations can help reduce costs while improving accuracy. Our motivation aligns with these findings; however, rather than merely cascading LLMs, we identify task categories that can be better addressed using new tools generated by larger model and assign each individual inference within that task category smaller model.
Early attempts tool-making. Concurrent and independent to work, several early attempts have been made towards using LLMs to make tools. Wang et al. (2023) conducted research within the Minecraft environment and demonstrated the ability of an LLM- agent to acquire new skills in the form of programs. Similarly, Qian et al. (2023) proposes a method of decomposing problem-solving for each individual instance into an abstract tool creation phase and concrete tool
3

---

Published as a conference paper at ICLR 2024
application phase. Our work aligns with the spirit of both Wang al. (2023) and Qian et al. (2023) in the aim to let LLMs to generate their own tools for croblem-solving However, we also underscore the significance of tool reusability and cost-effectiveness stemming from the division of labor. The idea of tool making is also mentioned in recent survey paper (Qin et al., 2023).
3 LLM AS TOOL MAKER (LATM)
Tool making template (One-time
Tool proposing: Write generic Python function (the Tool) to solve three training samples.
Tool verification: Write unit tests to convert three validation samples into function call and validate the correctness.
Tool wrapping: Gather the function from the proposing stage and the examples of how to convert problems to function calls from the verification stage into reusable Wrapped Tool.
Tool Maker (e.g., GPT-4): Strong performance but slow and expensive
Tool using template (Reusable &
Convert problem into function call according to the Wrapped Tool
Tool User (e.g., GPT-3.5 Turbo): Weaker performance but much faster and cheaper 2/.
Wrapped Tool
Figure The pipeline of LATM. LATM can be divided into two stages: 1) tool making: powerful yet more expensive model serves as the tool maker to generate generic and reusable tools from a few demonstrations; 2) tool using: a lightweight and cheaper model serves as the tool user to use the tool to solve various instances of the task. The tool-making stage can be further divided into three sub-stages: (i) tool proposing: the tool maker makes an attempt to generate the tool (Python function) from few training demonstrations, if the tool is not executable, report the error and generate new (ii) tool verification: the tool maker runs unit tests on validation samples, if the tool does not pass the tests, report the error and generate and (iii) tool wrapping: wrapping up the function code and the demonstrations of how to convert a question into function call from unit tests, preparing usable tools for tool user.
one (fix the function);
tests (fix the function calls in unit tests);
3.1 MAKING NEW TOOLS AND REUSE THEM
In the LATM paradigm, the main process can be split into two stages: Tool Making and Tool Using. Each stage utilizes different types of Large Language Models (LLMs) balance performance and cost-effectiveness. All the prompts used in our experiments are shown in Appendix C.
Tool Making. This stage employs powerful yet more expensive model, such as GPT-4, to serve the tool maker. Tool maker's role is create generic and reusable tool (implemented a Python function) from few demonstrations of task. This stage can be further divided into three sub-stages:
Tool Proposing: In this stage, tool maker attempts to generate a Python function to solve the demonstrations from the given task. This process follows the "programming by example" (PbE)
4

---

Published as a conference paper at ICLR 2024
paradigm (Halbert, 1984) where several concrete demonstrations are provided, and the model is required to write programs that produce the demonstrated behaviors. In our experiments, we use 3 demonstrations for this stage. If the proposed tool is unexecutable or encounters errors, tool maker appends the error messages to the history and makes another attempt.
Tool Verification: this stage, the tool maker generates unit tests using validation samples and subsequently executes these tests the proposed tool. We utilize 3 validation samples in our experiments. If the tool fails any of these tests, the tool maker records the error in its history and makes attempt to rectify the issues within the unit tests (this procedure will only correct the function calls in the unit test part and will not correct the function). The ability of LLMs to self-debug has been demonstrated effectively in recent research (Madaan et al., 2023: Chen et al., 2023c; Lu et al., 2023; Kim et al., 2023). However, within the LATM pipeline, the verification stage serves a slightly different usage, This stage fulfills two key roles: 1) provides examples that demonstrate how to convert natural language questions into function calls, and 2) it verifies the tool's reliability, enabling the entire process be fully automated.
Tool Wrapping: If the execution or verification fails over a preset threshold, the Tool Making stage is viewed as failed, Otherwise, tool maker is ready to prepare the wrapped tool for tool user, This step involves wrapping up the function code and providing demonstrations of how to convert a task into function call. These demonstrations are extracted from the Tool Verification step, which converts questions into unit tests. This final product is then ready for use by the tool user. Please see Appendix D for examples of the wrapped tools.
Tool Using. This second stage involves a lightweight and cost-effective model, such as GPT-3.5 Turbo, to serve as the tool user. The tool user's role is utilize the verified tool to solve various instances of the task. The prompt for this stage is the wrapped tool which contains the function for solving the task and demonstrations of how to convert a task query into a function call. With the demonstrations, tool user can then generate the required function call in an in-context learning fashion. The function calls are then executed to solve the task. Optionally, postprocessing can be applied to convert the output to match the required format of the task, such options for multiple-choice questions.
The tool-making stage, including tool proposing, verification, and wrapping, only needs to be performed once for each type of task. The resulting tools can then be reused for all instances of that task. This makes LATM significantly more efficient and cost-effective than using a powerful model alone. Furthermore, the Python function tools are a more generic form of Dhain-of-Thhugeee enhancing the overall utility and flexibility of the LLMs, as they can be used to solve questions that involve algorithmic reasoning ability (Veličković and Blundell, 2021).
4 LATM FOSTERS A FUNCTIONAL. CACHE MECHANISM FOR LLM SERVING
In worldworld scenarios, tasks often arrive in a sequential stream. To address this, we introduce a third LLM, the dispatcher, that decides whether engage the tool user or tool maker for each incoming task. While this tool selection function mirrors existing works (Lu et al., 2023; Shen et al., 2023; Schick al., 2023; Paranjape et al., 2023), our dispatcher distinetively contributes to creating functional cache-it it discerns new tasks that cannot be resolved with existing tools, thereby triggering the tool maker to generate appropriate tools for these tasks.
The dispatcher maintains repository of existing tools crafted by the tool maker in the format of function APIs Upon receipt of new task instance. the dispatcher first attempts to locate a compatible tool within the cache. If such a tool is present, the dispatcher assigns the instance and corresponding tool the tool user for resolution. However, no suitable tool is available, the dispatcher identifies this as a novel task, either solving it with powerful model if necessary, invoking a human labeler. These new instances are then cached until a sufficient number are amassed to craft a new tool, further enriching the functional cache. This mechanism allows for the functionally similar tasks to reuse these tools, expanding the coverage of the classic cache mechanism and reducing the overall serving cost. Given the simplicity of the dispatching task, lightweight model equipped with appropriate prompts (See Appendix C) can efficiently serve as the dispatcher, adding only a marginal cost to the entire pipeline.
5

---

Published as a conference paper at ICLR 2024
Tool user side run several times on a lightweight model
Tool maker side only run powerful model
Tool proposing input
Tool using output
Tool proposing output
Input Five animals
arranged in an unknown order:
animals = =(" Iatt 152
from itertools import permutations
Conditions:
condition1 lambda order: order.index(" "2") order Index!"
+
def find_op orderjobjects, conditions): for order in permutations(chjecss)
condition2 lambdo order: order.index" s2 ") order index" tal condition3 lambdi order: order(1)
is the second one
Question:
valid True for condition in
animal the one? Output 1:
condition4 lambad order: order.index! *0 order. index"
conditions:
not conditionoorder): valld = False break
conditions (condition1, condition2, condition condition4)
Input Output w*w
order find_ uadd_oderrannimlss conditions)
valid:
Please write generic Python function solve this type of problems
return order
print(order(21)
Figure 3: An illustration the Tool Proposing and Tool Using stages of the LATM pipeline for the Logical Deduction task (Srivastava et al., 2022). This task requires determining the order of five objects based on several given conditions. In the Tool Proposing stage, the tool maker (such GPT-4) formulates a generic Python function capable of solving the provided & demonstrations from the task (where equals 3 in our experiments). The tool maker generates a search algorithm that enumerates all possible orderings and verifies each against the provided conditions. During the tool-using stage, the tool user trans lates each natural language question into a series of conditions, generating function calls utilize the tool for each task instance.
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets. We evaluate our approach on six datasets from diverse domains, including Logical Deduction, Tracking Shuffled Objects, Dyck Language, Word Sorting, Chinese Remainder Theorem, and Scheduling Meeting. The first five datasets are sourced from BigBench (Srivastava et al., 2022). We take the objects version of the Logical Deduction and Tracking Shuffled Objects tasks, referred to as Logical Deduction (5) and Tracking Shuffled Objects (5) in the paper. We also constructed the Scheduling Meeting task to demonstrate the effectiveness of LATM in real-world scenarios. Detailed information on dataset generation can be found in Appendix E. We divide each dataset into training, validation, and test sets, containing 3, 3, and 240 instances, respectively.
Model settings. During the tool-making stage, set the temperature to 0.3 to introduce ran- domness to the generation process, allowing for retries if necessary. For this stage, we conduct experiments using GPT-4 and GPT-3.5 Turbo models with the ChatComple eni API, always appending the response to the chat history to create an interactive experience. In the tool-using stage, the LLM API call made only once, and we also perform ablation studies on GPT-3-type models with the standard Complet on inn API. When using the tools, we consistently set the temperature to 0.0. We set the maximal retry times to be 3 for the tool-proposin and nol-verification stages.
5.2 EFFECTIVENESS OF THE TOOL-MAKING STAGE
In the tool-making stage. tailored to a specific task. This step is performed only once for each task, and the overhead is amortized across all instances of that task. In experiments, GPT-4 (OpenAI, 2023)
powerful yet slower model to generate generic Python functions

---

Published as a conference paper at ICLR 2024
Logical Deduction (5)
Tracking Shuffee
Dyck
Word
Chinese
Schedule Meeting
Objects (5)
Language
Sorting
Remainder Theorem
Search
Simulation
Stack
Sort
Scarch/Extended Euclidcan
Interval intersections
Table 1: The utility functions generated by tool maker to solve the tasks.
representative tool maker, while we explore other models' tool-making capabilities in Section 5.5. We provide several few-shot exemplars for the language model, guiding it to generate generic Python programs, illustrated in Figure 3.
Our observations indicate that when GPT-4 is employed as the tool maker, the model frequently devises suitable algorithms for solving tasks. For instance, shown in Table 1, the tool maker creates code to solve the logical deduction task by searching through all permutations and selecting the correct one that satisfies the given constraints. In experiment, the vooliveriiiaation stage is mainly used to provide examples that demonstrate how to convert natural language questions into function calls, and only observe 2 cases out of the 60 trials that the tool maker can correct its mistakes with the guide of стrоr messages. See Section 5,5 for more discussions on the tool maker.
5.3 LATM IMPROVES THE PERFORMANCE LIGHTWEIGHT LLMS
In Table 2, we compare the performance of Chain-of-Thought prompting (Wei et al., 2022) with our method, LATM. We employ GPT-4 as the tool maker generate tools for the six tasks, and evaluate the performance of both GPT-3.5 Turbo and GPT-4 as tool user. The results demonstrate that with the help of the tool, a lightweight model like GPT-3.5 Turbo can achieve performance on par with GPT-4, significantly outperforming CoT prompting. Additionally, the average cost of using GPT-3.5 Turbo with the tool is much lower compared to using GPT-4. This highlights the effectiveness of LATM in enhancing the performance of lightweight models and therefore reducing the cost compared to employing expensive models. Intriguingly, for the Dyck Language task, GPT-3.5 Turbo as the tool user even surpasses GPT-4 in its role the tool user. Upon investigating the failure cases, find that when converting the question into function call, GPT-4 occasionally superfluously closes some brackets within the argument instead leaving the argument unchanged and letting the function solve it, which leads to incorrect function output.
Tool User Model
Method
Logical Deduction
Tracking Shuffled Objects (5)
Dyck Language
Word Sorting
Chinese Remainder Theorem
Schedule Mceting
Cost on a samples
GPT-3.5
CoT
66.4
61.6 99.6 (+38.0)
20.4
59.2 98.3 (+39.1)
0.0 100.0 (-100.0)
18.9 100.0 (481.1)
O(ne)
Turbo
LATM
79.7 (+13.3)
92.2 (471.8)
O C)
CoT
88.8
100.0
63.6
90.9
0.0
55.6
O(nC')
GPT-4
LATM
86.6
100.0
87.5
99.1
100.0
100.0
O(nC)
Table 2: Accuracy comparison between LATM and ohann-of-Thought The six tasks are detailed Section 5.1. For LATM, the tool is created by GPT-4 and utilized by both GPT-3.5 Turbo and GPT- 4. The results demonstrate that the application of LATM significantly enhance the performance of GPT-3.5 Turbo, often surpassing or matching GPT-4's performance with CoT in certain scenarios. The last column depicts the overall cost of processing samples. Here, C' represents the cost of one call to GPT-4, while denotes the cost of one call to GPT-3.5 Turbo. At the time of writing this paper, C is over 1.5x larger than c. The few-shot CoT demonstrations for the first four tasks provided by Suzgun et al. (2022), while for the last two tasks, we apply direct few-shot prompting without CoT.
5.4 ADAPTING LATM TO DYNAMIC STREAM OF DIVERSE TASKS
As discussed in Section 4, we can adapt LATM to handle dynamic stream where instances from potentially different tasks emerge in real-time. In this setting, we introduce an additional model, the dispatcher, tasked with identifying the task which each incoming instance pertains. We employ GPT-3.5 Turbo for this role, evaluating its effectiveness in two key functions: 1) Identifying and employing existing tools from the functional cache to resolve incoming instance, and 2)
7

---

Published as a conference paper at ICLR 2024
Logical Deduction (5)
Tracking Shuffled
Dyck Language
Word Sorting
Chinese Remainder Theorem
Schedule
Tool Maker Model
Objects (5)
Mceting
GPT-3.5
0/5
0/5
5/5
5/5
5/5
0/5
GPT-4
3/5
4/5
5/5
5/5
5/5
3/5
Table 3: Success rate of generating new tools (Python functions that pass the tool-verification
step) in the tool-making stage with GPT-4 v.s. GPT-3.5 Turbo. We run 5 trials for each model on each task, n/5 means n trails out of 5 successes to produce a valid tool. For hard tasks like Logical Deduction and Tracking Shuffled Objects, GPT-3.5 Turbo fails in all trials, showing the necessity of using a more powerful model as tool maker.
Detecting unseen tasks and triggering the tool maker to create appropriate tools for these tasks. This experimental setup helps assess how effectively our system can reduce serving costs by reusing and extending the functional cache in dynamic, multi-tasking scenario.
Identifying inng existing tools. The first part of our evaluation assesses the dispatcher's capability to recognize existing tools within the functional cache that correspond given instance, analogous to the cache fetching phase of traditional cache systems. To this end, we generate test set of 100 samples, randomly mixed from the six tasks discussed in Section 5.1. For each instance, the dispatcher is tasked to determine the appropriate tool from existing ones, utilizing prompts containing task examples associated with these tools (See Appendix C). Success measured by the correct identification of the tool. Over five random constructions of the test set, the accuracy in correctly determining the suitable tool is 95% = 2%.
Requesting tool-making. The second part of our evaluation tests the dispatcher's ability to request tool-making for instances originating from an unseen task. This situation is akin to enqueuing a new instance into the cache when a cache miss happens. We randomly designate four tasks as existing tasks with readily available tools and select four other tasks for testing two of these are unseen, and the other two fall within the realm of existing tasks. Again, a test set of 100 samples generated. For each instance in the test set, the dispatcher determines whether it needs to request tool-making or if an existing tool can solve the instance. Over multiple runs, the accuracy of making the correct decision stands at 96% + 3%, demonstrating the robustness of our approach in efficiently managing the functional cache.
The above results illustrate that the dispatcher can effectively recognize existing tools and accurately request tool-making for unseen tasks, all while maintaining high performance. These findings highlight the potential of LATM to be seamlessly adapted to a streaming environment encompassing a diverse range of tasks. This validation serves to fortify the viability of our framework in real-world applications, particularly where the efficient management of functional cache is paramount.
5.5 ABLATION STUDY
Capacity required for the tool-making language model. We investigate the capacity requirements for the language model used in the tool-making stage (See Table 3). Generally, we found that a more powerful and expensive model better serves the purpose, this stage is performed only once for each task, and high accuracy is crucial for effectively passing tools to a smaller model. Specifically, on hard tasks like Logical Deduction and Tracking Shuffled Objects, GPT-3.5 Turbo fails in all the trails. And the major failure reason is that the tool is not general enough and may only work on the training samples. On the other hand, we also discovered that for easy tasks, the tool maker can be a lightweight language model. For simple tasks like Word Sorting, GPT-3.5 Turbo can effortlessly generate a program that solves the task. Another limitation that may contribute to the tool maker's failure is the context length constraints. Since we use the entire history in each step of tool- moll-aaking to enhance the reliability of the tool-making stage, this also introduces a longer context. In this case GPT-4 with 8192 context length is preferable.
Capacity required for the tool-using language model. In this section, we investigate the capacity requirements for the tool-using model. The results are presented in Table 4, We observed that GPT-3.5
8

---

Published as a conference paper at ICLR 2024
GPT-3.5 Turbo
text-davinci-002
davinci
curie
babbage
ada
Logical Deduction (5)
79.7%
58.2%
11.6%
6.5%
11.6%
3.0%
Tracking Shuffled Ohjects (5)
99.6%
100.0% %
62.1%
20.7%
16.4%
5.2%
Dyck Language
92.2%
35.8%
16.4%
18.1%
9.1%
9.9%
Word Sorting
98.3%
60.8%
26.6%
7.3%
7.3%
0.9%
Chinese Remainder Thcorem
100.0%
100.0%
99.6%
93.1%
75.0%
66.0%
Schedule Meeting
100.0%
100.0%
62.9%
59.1%
2332%%
0.0%
Cost ($ IK tokens)
0.002
0.02
0.02
0.002
0.0005
0.0004
Table 4: A performance comparison of various tool user models, all using the same tool generated by GPT-4. All costs are based on the rates at the time of writing. Of all the models, GPT-3.5 Turbo demonstrates the best trade-off between performance and cost. We opted for GPT-3 models prior to instruction tuning (ada instead of text-ada-001, etc.), as we observed that the models after instruction tuning underperformed in the tool-using stage. We postulate that this due to the instruction tuning impairing the in-context learning ability, which is essential for the tool-using stage.
Turbo offers the best balance between performance and cost among all the models tested. Regarding the older GPT-3 series of models (ada, babbage, curie, davinci), we found that models that before instruction tuning often perform better than their counterparts post instruction tuning (text-ada-001. etc.). We hypothesize that the instruction tuning phase in these models may adversely impact the in-context learning ability, which is crucial for the tool-using stage.
CoT as a tool does not help. In addition to LATM, we investigate if we can improve task performance by reusing Chain-of-Though (CoT) from larger model to smaller model similar to LATM pipeline. Specifically, we use the same larger model (GPT-4) in the "Co'T-making" stage, using zero-shot prompting "Let's think step by step." to elicit the intermediate thought steps, and then use the generated CoT to the same smaller tool-using model (GPT-3.5 Turbo). We test this on two tasks and report the results Table 5. We observe that using CoT from a large model has similar or even worse performance than humtn-written CoT, which is much worse than LATM.
Accuracy
GPT-4 CoT
Human-written CoT
LATM
Logical Deduction (5)
36.8
66.4
79.7
Tracking Shufiled Objects (5)
63.2
61.6
99.6
Table 5: Accuracy of using CoT generated by GPT-4. The performance is similar to wrrtten CoT, which is much worse than LATM.
6 CONCLUSION AND FUTURE WORK
We introduced LATM, a closed-loop framework empowering large language models (LLMs) to create and utilize their own tools for diverse tasks. Our approach, inspired by human's 's evolutionary strides in tool creation, employs two key stages: Tool Making and Tool Using. This division of labor allows us harness the capabilities of advanced LLMs while significantly reducing computational costs. Our experiments confirmed the efficacy of LATM across various complex tasks, demonstrating that our framework performs comparably resource-intensive models while being more cost-effective. In addition, we show that adding another dispatcher LLM can further provide flexibility to our framework, enabling on-the-fly tool creation and usage.
In our evaluation process, we identified a significant lack of high-quality datasets that authentically represent daily human-computer interactions, including recurring tasks such as scheduling meetings or booking flights over email or phone calls, in their raw natural language format. We anticipate that our work will stimulate the research community to create such datasets, which could prove instrumental in cultivating the next generation of AI systems. These systems, capable of generating and applying their own tools, will be equipped to tackle complex tasks more effectively. An exciting avenue for future research is enabling the tool maker refine and upgrade existing tools to manage new problem instances, much like in software development. This adaptability could further catalyze the evolution of the Al ecosystem, unlocking wealth of opportunities.

---

Published as a conference paper at ICLR 2024
REFERENCES
Aida Amini. Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorsk, Yejin Choi, and Hannanch Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation- formalisms. arXiv preprint arXiv: 11119905.333 /3319, 2019.
Simran Arora, Brandon Yang, Sabri Eyuboglu. Avanika Narayan, Andrew Hojel, Immanuel Trummer, and Christopher Ré. Language models enable simple systems for generating structured views of heterogeneous data lakes, 2023.
Jacob Austin, Augustus Odena, Maxwell Nye. Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Lc, and Charles Sutton. Program synthesis with large language models, 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Mclanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12271 2023.
Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptist Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling. February 2023a.
doi: 10.48550/ARXI.2302.0113
Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How large language models while reducing cost and improving performance, 2023b.
Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks, 2022.
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug, ARXIV.OR,, 2023c. doi: 408550/a2xi..334005555
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:220..02311, 2022.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, al. Training verifiers solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
David Dohan, Winnie Xu, Aitor Lewkowycz. Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A. Saurous, Jascha Sohl-dickstein, Kevin Murphy, and Charles Sutton. Language model cascades, 2022.
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models, 2023.
Google. Palm 2 technical report, 2023. URL https: //ai. httpaa/ootteeeett/////eeeoooome palm2techreport.pde.
Daniel Conrad Halbert. Programming by example. University California, Berkeley, 1984.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, al. Training compute-optimal large language models. arXiv preprint 2rXiv:2203.155 2022.
Geunwoo Kim, P. Baldi, and S. McAleer. Language models can solve computer tasks. ARXIVORG.. 2023. doi: 10.48550/a2X1.22333.7744
Michal Kosinski. Theory of mind may have spontancously emerged in large language models. arXiv preprint arXiv:2302. 0000:22322000883, 2023.
10

---

Published as a conference paper at ICLR 2024
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. November 2022. doi: 1048550/ARXI2221111122
Yaobo Liang, Chenfei Wu, Ting Song, Wenshan Wu, Yan Xia, Yu Liu, Yang Ou, Shuai Lu, Lei Ji, Shaoguang Mao, et al. Taskmatrix, ai: Completing tasks by connecting foundation models with millions of apis. arXiv preprint arXiv:2303. 2X11:22333316434,, 2023.
Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale gen- eration: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 158-167, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10. (0.18653/v1/P17-1000 URL https::tpp/..m. / antholo woy..lg.......o............. wwtp/17-g1111111
Ruibo Liu, Jason Wei, Shixiang Shane Gu, Te- Yen Wu, Soroush Vosoughi, Claire Cui, Denny Zhou, and Andrew M. Dai. Mind's eye: Grounded language model reasoning through simulation. In The Eleventh International Conference on Learning Representations. 2023. URL https:
//openrevieww.c/////////oe net wwwp:/rrmteeteeeeeeeeeaaoaomaama 4rXMRuo
Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language models. arXiv preprint arXiv:2304. 009442, 2023.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint 1731: 2023.
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski. Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz. Maarten Bosma, David Luan, al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114, 2021.
OpenAl. Gpt-4 technical report, 2023.
Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannanch Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. Art: Automatic multi-step reasoning and tool-use for large language models. arXiv preprint 23X1v:2303.09014, 2023.
Aaron Parisi, Yao Zhao, and Noah Fiedel. Talm: Tool augmented language models, 2022.
Cheng Qian, Chi Han, Yi R Fung, Yujia Qin, Zhiyuan Liu, and Heng Ji. Creator: Disentangling abstract and concrete reasonings of large language models through tool creation. arXiv preprint 1331:22005.14318 2023.
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang. Chaojun Xiao, Chi Han, et al. Tool learning with foundation models. arXiv preprint arXiv:2304.08354. 2023.
Timo Schick, Jane Dwivedi-Wo Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools, 2023.
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint 233.5 2023.
Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang. Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. Language models are mul- tilingual chain-of-thoughh reasoners. arXiv preprint arXiv:2210.03057 2022.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro. Aditya Gupta, Adrià Carriga-Alonss et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206. 04615, 2022.
11

---

Published as a conference paper at ICLR 2024
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoe Le. Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXtv:2210.09261, 2022.
Petar Veličković and Charles Blundell. Neural algorithmic reasoning. Patterns, 2(7):100273, 2021.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305. 16291, 2023.
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint 1 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint wrXiv:2201.11900 2022.
Tongshuang Wu, Michael Terry, and Carrie Jun Cai. Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1-22, 2022.
Heming Xia, Tao Ge, Si-Qing Chen, Furu Wei, and Zhifang Sui. Speculative decoding: Lossless speedup of autoregressive translation, 2023. URL https: wwwppeeeeww.eeeeforrrrreeeeeeeee http://wwww.e.eeoooeeeeeeeeeeeee H H-VlwsYvVi
Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303.113888 2023.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference Learning Representations, 2023. URL https:
/ hwrpenreview.nee/EEeee
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXhv:2205.0106 2022.
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoe Le, and Ed Chi. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.106625 2022.
Zilliz. Gptcache. https: // /ttpt/ttttttt///////////tteht com 2 httl:iiltcc/ccccaeaae 2023.
12

---

Published as a conference paper at ICLR 2024
A ILLUSTRATION OF THE DISPATCHER
stream of instances from (potentially) different tasks
Moke new one
Tool Maker
Does the tool exist?
Task instance
Dispatcher
Yes Reuse the old one
Tool Cache
Tool User
Figure 4: An illustration of the Dispatcher that enables functional cache mechanism. In online setting where task instances arrive sequentially, the dispatcher, a lightweight model, assesses each incoming instance. If suitable tool already exists in the cache tackle the task, the dispatcher selects this tool and forwards the task instance to the tool user for resolution. If no suitable tool is found, the dispatcher routes the task instance the tool maker to create new tool that can be used by tool user later.
B BROADER IMPACT AND LIMITATIONS
This paper explores the potential of enabling Large Language Models (LLMs) to create their own tools, thus allowing them greater autonomy developing their ecosystem. While this avenue of research is promising, it also raises important ethical, safety, and control considerations that need to be carefully addressed.
One of the most significant impacts of our work lies in the potential for LLMs to grow and achieve unprecedented capabilities automatically. This could significantly enhance the range and complexity of tasks these models can handle, potentially revolutionizing fields such customer service, tech- nical support, and even areas of research and development. It could lead to more efficient use of computational resources and reduction in human intervention, especially for routine repetitive tasks.
However, this newfound autonomy of LLMs a double-edged sword. As we endow LLMs with the ability to generate their own tools, we also create a scenario where the quality of the tools they develop may always meet the standards expectations set by human developers. Without proper safeguards, there's 's risk that these models could generate solutions that are suboptimal, incorrect, or even potentially harmful. Furthermore, as LLMs become more autonomous, the potential for loss of control increases. If these models are widely used without appropriate regulation, there could be unforeseen consequences, potentially even leading to scenarios where humans lose control over the AI systems.
In this study, we have not addressed these control and safety issues in depth, and our work has some limitations. Our proposed framework. LLM As Tool Maker, while effective in the tested scenarios, still its early stages of development. is crucial to note that the real-world performance and safety of the system may vary based the complexity and nature of the tasks it is applied to. Additionally, the evaluation and validation of the tools created by the tool maker in a real-world setting is challenge that needs to be addressed.
13

---

Published as a conference paper at ICLR 2024
C
LATM PROMPTS
Tool Maker Prompt
Please write generic Python function to solve this type of problems using only standard python libraries. The output of the function can later be converted the answer (option for multiple choice question). All the function should wrapped by python
Tool Verifier Prompt
Write unit tests to verify the correctness of the function on the questions above using the following format: ""python
(parse the question into the arguments of the function]
(call the function and save the return value in a variable 4 named "ret")
(for multiple choice question, parse the options) (convert the return value "ret" the answer (if the question is multiple choice question, convert to an option) and save it in variable named "ans", otherwise) assert ans == the provided answer (if the question is multiple choice question, assert ans option))
Tool Wrapper Prompt
Success! The function is correct. We will need to summarize
+ the function and use cases up for further use. Please extract the information from the history in the following
format:
Here is a function to solve class of problems:
""python
(the function, including necessary imports)
Use cases:
Question: (question (including options))
Solution:
"""yyhhon
(parse the question into the arguments of the function]
(call the function and save the return value in variable
4 named "ret"}
(for multiple choice question, parse the options) (convert the return value "ret" to the answer (if the multiple choice question, convert to an option) and save it in variable named "ans", otherwise)
4 question
Do this for all the questions in the verification step.
14

---

Published as a conference paper at ICLR 2024
Dispatcher Prompt
Here are several functions that can be used to solve some task:
4
Task: Logical_dedccioo..f/eeeoooo.t
API: find_orded (objects, constraints);
Finds the order of objects that satisfies given set of + constraints.
objects: A 1ist of unique objects (strings) to be ordered. constraints: A list of lambda functions that represent the constraints the order of objects. Each constraint should take the order of objects as input and return a boolean value (True if the constraint is satisfied, False otherwise).
return: A tuple representing the order of objects that satisfies all the constraints. If no such order exists, the function returns None.
Task: traxxing.suullleoobbeeesseeeeeee
API:
square aquare_dancell (initial_partnera, switches): This function takes an initial list of pairs and a list of switches, and returns a dictionary representing the final state of the pairs after performing the switches.
initial_partners: A list of tuples, where each tuple contains two elements representing pair (e.g., [(*Alice", - "goalkeeper"), ("Bob", "left midfielder"), ...]). The elements can be any type (e.g., strings, integers, etc.).
switches: A list of tuples, where each tuple contains two
+ elements representing a pair of elements from the initial_partners list that will be switched (e.g., ["'Alice", "Claire"), ("Alice", "Bob"), ...)). The elements should match the types used in the initial_partner list.
return: A dictionary representing the final state of the pairs after performing the switches. The keys are the list,
first elements of the pairs in the initial-partned and the values are the correspending second elements after performing the switches (e.g., ("Alice": "right winger", "Bob": "center midfielder", ...)).
Skip other tasks
Here is a innsstiio:nnquusttiiio (nteesttiooinnnncccooooooo \n)nAccoding to the API documents above, you may find some functions that can be used to solve the task, or, sometimes there does not exist proper function to solve the task. Figure out if
there is function to solve the task and reply in the
format: \nTask: ([task)) (reply unknown no function can solve the question)
15

---

Published as a conference paper at ICLR 2024
16

---

Published as a conference paper at ICLR 2024
D WRAPPED TOOLS
Tool for Logical Deduction
Here is function to solve class of problems:
"python
from itertools import permutations
def oider (objects, constraints)
for order permutations (objects)
valid True
for constraint in constraints:
if not constraint (order) : valid False
break
if valid:
return order
Use cases:
Question:
The following paragraphs each describe a set of five objects arranged in fixed order. The statements are logically consistent within each paragraph. On 4 shelf, there are five books: a white book, a green book, a brown book, a gray book, and an orange book. The gray book is to the right of the orange book. The green book is the second from the right. The brown book is to the right of the white book. The brown book is to the left of the orange book.
Options:
(A) The white book is the third from the left
(B) The green book is the third from the left
(C) The brown book is the third from the left
(D) The gray book is the third from the left
(E) The orange book is the third from the left
Solution:
"'pyhhon
objects = ["white", "green", "brown", "gray", "orange"]
constraints
lambda order: order. index ("gray")
4
order. index ("orange"),
lambda order: order. index ("green")
len (order) 2,
lambda order: order. index (ooex(""ss ("brown")
order.index
order, index ("white"),
lambda order: order index ("brown") < order. index ("orange")
ret Find_ (objects, constraints)
options
"A": "white", "B": "green",
"c": "brown"
"D": "gray",
"E"; "orange"
ans [k for k, in options. items () if ret [2] [0]
17
Skip two more questions...

---

Published as a conference paper at ICLR 2024
Tool for Tracking Shuffled Objects
Here is a function to solve class of problems:
""python
def (inaraal
switches):
# Create dictionary to store the current partners current_partnes dict (initial_partners)
Tterate through the switches and update the current partners
for switch in switches:
dancerl, dancer2 switch
partnerl current_partners [dancer]] partner2 current_partnan [dancer2]
# Swap the partners
[danerr))] dancerl] partner2
current_partners [dancer2] partnerl
return current_partner.
Use cases:
Question: Alice, Bob, Claire, Dave, and Eve are the team in soccer match. At the start of the match, they are each assigned to position: Alice is playing goalkeeper, Bob is playing left midfielder, Claire is playing right winger, Dave is playing striker, and is playing center midfielder.
As the game progresses, pairs players occasionally swap positions. First, Alice and Claire trade positions. Then, Alice and Bob trade positions. Then, Dave and Bob trade positions. Then, Bob and Eve trade positions. Finally, + Dave and Eve trade positions. At the end of the match, Eve is playing
options:
(A) goalkeeper
(B) left midfielder
(C) right winger
(D) striker
(E) center midfielder
Answer: (C)
Solution:
""'ython
initial_poestions [["Alice", "goalkeeper"), ("Bob", "left
+ midffelder"), ("Claire", "right winger"), ("Dave", "striker"), ("Eve", "center midfielder"]] switches = ["AAliee", "Claire"), ("Alice", "Bob"), ("Dave", "Bob"), ("Rob", "Eve"), ("Dave", "Eve")
ret square square_dance dance
switches)
options = ["goalkeeper", "left midfielder", "right winger", - "striker", "center midfielder"]
ans = options. index (ret (ret(" + 1 # Convert the return value to an option index (1-based)
Skip two more questiors...
18

---

Published as a conference paper at ICLR 2024
Tool for Dyek Language
Here is a function to solve class of problems:
""pyyhon
def smmpltt__sqqennee (input_str)
stack = []
plosing_map ('C: "), 'D' "),, '<: 12, '0: " result = []
for char in input_strs
if char in closing_map m ()
stack. apack. (char)
elif char in closing_map. values ()
if stack and closing_map [stack [-1]]
char:
stack.pop ()
else:
return "Invalid sequence"
else:
return "Invalid character"
while stack:
result.apeent (closing_map pp [stack[-1]] [-1]]) stack.pop
return " join (result)
Use cases:
Question: Complete the rest of the sequence, making sure that the parentheses are closed properly. Input:
[[[())]<<<<())>>}}}
Answer: 1)
Solution:
"'"yython
input_str = "([[1)]<<<()>>>>>""""""""<<<")""
ret sequueneeeeeeeceeeeeeeeeeeee (input_str) ans ret
skip two more questions...
19

---

Published as a conference paper at ICLR 2024
Tool for Word Sorting
Here is a function to solve class of problems: " "python
def hat__wods_aalhabeee return sorted list)
(wsrd_list):
Use cases:
Question: Sort the following words alphabetically: List: + conference apparition ignore dutton layperson coupe superstitious westward turnoff messenger copra floruit primitive implement
Answer:
apparition conference copra coupe dutton floruit ignore implement layperson messenger primitive superstitious turnoff westward
Solution:
"python
wordsl ["conference", "apparition", "ignore", "dutton", "layperson", "coupe", "superstitious", "westward",
4 "turnoff", "messenger", "copra", "floruit", "primitive", "implement"]
retl sort_words yyserddslppaabetcalllyyyy (wordsl) ansl (retl)
Skip two questions...
20

---

Published as a conference paper at ICLR 2024
Tool for Chinese Remainder Theorem
Here is function to solve class of problems:
"python
def find_number (max_limit, divisors, remainders): for num in range (max_limit 1):
if all ( (num remainder) & divisor -- 0 for divisor, remainder in zip (diviscrs, remainders)): return num
return None
Use cases:
Question:
There is basket of more than 1188877 durians.
If we divide them equally among 41 penguins, we have 17 left; if divide them equally among 107 dinosaurs,
have 42 left; if we divide them equally among 271
elephants, have 260 left. How many durians are in the basket?
Solution:
"python
max_limit = 1188877
divisors [41, 107, 271]
remainders = [17, 42, 260]
ret find number (max limit, divisors, remainders) ans ret
Skip two questions...
21

---

Published as a conference paper at ICLR 2024
Tool for Schedule Meeting
Here is a function to solve class of problems:
"ppython
from datetime import datetime, timedelta
def find earliest_rime slot (a avvaailabbl, b_availability,
meeting_dbration):
a_availability [datetimee strppimm (start, (8H:8M") datetime datetime.strpth strptime (end, (8H:(M')) for start, end in 4 a_availability]
b b_availability
strepttimmsssrpimmeeeee (start, (8H:8K") datetime. strptmmmmsrptimmm. (end, (8H:8M') for start, end in b_availability)
for a_start, a_end in a_availability: for b_start, end in b_availability:
latest_start max b p_start) earliest_end = min b_end)
if earliest_end latest_start >=
+ timedelta (minutes hetprresseeeetinnnuurreeeeeeeeee 8H: 8M))
return _strssssrrtttt +
(latest (latest_start
timedelta meteeessmeeeinn_dudeeeeeeee .strfting (1SH:SM)
return None
Use cases:
Question: and want to schedule a 1-hour meeting together.
A's availability: 12:00 12:30, 13:00 13:30, 14:30 15:30, 17:30 18:00. B's availability: 09:00 11:00,
4
12:00 - 12:330, 13:00 - 13:30, 15:30 16:30, 17:30
4 18:00. What time slot works best? (if multiple, choose
the earliest one)
Answer: No time slot works.
Solution:
"'python
a_availability = [('12:00', (('1:000' '12:30'),
('13:00', (13:30'), ('14:30', '15:30'), ('17:30', '18:00')
4
b_availability = [''09:00', '11:00"), ('12:00', 112:30") [''00b00
('13:00', '13:30') ('15:30', '16:30") ('17:30', 118:00')
meeting__urratio 60
ret find_earliestttimm___lll (a_vaaiiblaa b_availability, meeting duration)
ans = ret if ret else "No time slot works."
skip two more questions...
E DATASET CONSTRUCTION
For the "schedule meeting" task, use the following template generate the dataset:
22

---

Published as a conference paper at ICLR 2024
question formmtt """A and want to schedule (interval) -hour - meeting together.
A's availability: (A (A_aaiilabllity)
B's availability: (B_availability]
what time slot works best? (if multiple, choose the earliest one) "n*
where the interval is randomly sampled from {0.5, 1, 1.5), and the availability of A and B are randomly sampled from 8:00-18:00 with minutes as the granularity. The answer is computed by computing the intersection of the two availability sets and then find the earliest time slot that is at least long the meeting duration. If there is no such time slot, return "No time slot works.".
23


Reviews:
[{"review_id": "9a0m1t49R5", "reviewer": "Reviewer_CctH", "summary": "The authors propose a novel method for prompting large language models that instructs them to create their own tools implemented as Python functions which help them solve various tasks that might otherwise be challenging to solve without any tool use. The idea is to prompt a larger LLM (e.g. GPT-4) to write a Python function for solving some given task (along with simple unit tests and documentation with example usage) and then passing this as context to a smaller LLM (e.g. GPT-3.5) and asking it to solve a problem by using a given tool. The key assumption is that larger LLMs are more capable of writing good tool functions compared to smaller LLMs. However, since larger LLMs are more expensive to run, it is beneficial to have them produce a tool that can then be successfully reused by smaller LLMs which are cheaper to run. Additionally, the authors propose a scheme for using another LLM that can decide, given a task, whether it needs to ask the larger LLM to produce a tool, or if it can reuse some of the already produced tools and pass them to the smaller LLM. The authors evaluate their method against reasoning tasks and show the feasibility of their approach.", "questions": "**Q1**: Why does CoT achieve a score of 0.0 on the \"Chinese Remainder Theorem\" problem and LATM achieves 100.0? Also, how does vanilla GPT fare on that task (and why does it fail if it fails)?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 1, "strengths": "**S1**: The paper presentation is relatively clear and easy to follow. Prior work is adequately referenced.\n\n**S2**: The method is relatively simple which makes it easily applicable. On top of that, the overall approach and distribution of roles between the tool-maker, tool-user, and dispatcher are sensible in this context.\n\n**S3**: The proposed method seems to perform favorably compared to the chain-of-thought baseline across 6 reasoning tasks.", "weaknesses": "**W1**: My key issue with the paper is that I'm not sure how substantial the contribution is. I understand the recent enthusiasm surrounding large language models. Furthermore, I do think there could be some scientific merit to studying questions like \"What can LLMs do?\", \"Where do they fail?\", and \"If we do X, can we make LLMs do Y?\" as these questions could deepen our understanding of LLMs, their capabilities, and potential for making them even more powerful. However, prompting LLMs is relatively easy. Hence, to publish a paper about prompting LLMs I would expect either some extremely non-trivial and surprising method or a very wide analysis across a large number of tasks that could further our understanding of the LLMs' capabilities and limitations. On top of that, what I would really love is to see a paper that is able to provide a compelling explanation of why LLMs are able or not able to do something (e.g. solve the Chinese remainder theorem task). On the other hand, a method that essentially says \"Hey LLM here is a task, can you write a tool that solves it, etc...\" is hardly something I would find surprising, especially given that the authors compare it against only a single baseline (which is equally simple in my opinion) and across only a small handful of relatively simple tasks. PS: I am OK to be challenged on this view (either by the authors and/or by other reviewers and PC members) but this is how I see things. ***TLDR***: In my opinion, the novelty bar for papers that propose methods to prompt LLMs should be much higher and should require much more effort and creativity than is presented in this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uScLUAg4XU", "reviewer": "Reviewer_6T59", "summary": "The paper introduces the \"Large Language Models as Tool Makers (LATM)\" framework, which explores the idea of language models creating their own reusable tools for problem-solving. LATM has the ability to craft tools to address challenges. It comprises two phases: tool making and tool using, with a division of labor between powerful and cost-effective models. Additionally, LATM introduces a functional cache mechanism to reduce serving costs. Experiments on a range of complex reasoning tasks show that LATM achieves high performance while being cost-effective.", "questions": "1. What is the price for the tool maker module?\n2. Can you provide more case studies on how the tool makers conduct the self-verification process?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The high-level idea of this paper is intuitive and makes sense. It introduces an innovative approach for utilizing large language models, allowing them to create reusable tools, resembling human tool-making abilities.\n2. This paper focuses on the practical setting, by strategically allocating the labor between powerful and lightweight models optimize performance while reducing computational costs.\n3. The experiments show that the proposed method achieves better performance.", "weaknesses": "1. The paper states that they use a self-verification process with a few labeled instances. However, it is not clear whether the performance is stable with a different selection of few-shot validation examples.\n2. It is not clear how the tool maker can rectify the incorrect tools made in previous rounds (mentioned in 3.1). More case studies are recommended.\n3. Is it possible to evaluate other tool-making benchmarks such as ToolBench[1], and APIBank[2]? These datasets will be useful in faithfully evaluating the tool-making ability of LLMs.\n\n[1] Qin, Yujia, et al. \"Tool learning with foundation models.\" arXiv preprint arXiv:2304.08354 (2023).\n\n[2] Li, Minghao, et al. \"Api-bank: A benchmark for tool-augmented llms.\" arXiv preprint arXiv:2304.08244 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bNIp5r2l27", "reviewer": "Reviewer_hBc8", "summary": "Recently, LLMs have demonstrated impressive problem-solving ability to address complex tasks. In this paper, authors introduce a *closed-loop* framework, called LLMs As Tool Makers (LATM). More specifically, this paper introduces two phases: tool making and tool using. Tool making uses LLMs to make new tools via a form of Python utility function and tool using means using tools for problem-solving. Experimental results demonstrate that LATM can achieve performance that matches GPT-4.", "questions": "1. This paper mainly utilized the GPT-based models for evaluation, do you try any other open-source LLMs?\n2. In Tool verification, the authors utilize 3 validation samples. So how to obtain these validation samples? Are these samples manually created and Are these validation samples fixed or different when meeting different user instructions? What are the generalizations of these examples in different scenarios?\n3. Based on my observations, the current LATM can only make tools in language form and cannot support multimodal ability, right?", "limitations": "", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper introduces how to use LLMs to make a tool, which sounds interesting. To fulfill this point, this paper proposes two stages: Tool Making and Tool Using. For Tool Making, the authors adopt three sub-stages: Tool Proposing, Tool Verification, and Tool Wrapping to achieve the target.\n2. Experimental results demonstrate the proposed method can improve the performance of LLMs in utilizing tools.", "weaknesses": "1. Tool making is an interesting idea. However, one weakness in this direction is that we could have many external tools in real-world scenarios and the capability of tool-making should be able to explore new tools that do not have before. Therefore, to further improve the feasibility of the proposed method, it will be better to explore the capability of tool-making to make new tools when also considering the existence of external tools.\n2. Compared with Tool making, the design of Tool use is relatively trivial. Many works (e.g., AutoGPT, HuggingGPT) have explored the feasibility of tool use by using LLMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T9c5yFLZVt", "reviewer": "Reviewer_uR2C", "summary": "The authors present a Tool-Making and Tool-Using framework with two separate LLMs and a functional caching mechanism to re-use tools for functionally analogous requests. With the dichotomy of making and using of tools, the former is assigned to heavier models (e.g. GPT-4) to implement python utility functions while the latter can be conducted with cheaper models (e.g. GPT 3.5). This strategy saves costs and, at the same time, doesn't impact performance as much.", "questions": "N/A", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "The paper was well-written and easy to read. The description of the problem and the proposed solution were clearly explained.\n\nThe idea of \"dynamically generating tools from a stream of tasks\" is novel (to the extent of the reviewer's knowledge) and if explored more concretely and evaluated end-to-end can be a considerable contribution to the community with many applied use cases in the industry.", "weaknesses": "The paper seems to be lacking originality and doesn't propose a significant contribution to the LLM domain. Here are the reasons for this conclusion:\n\nGenerating Python programs (utility functions), or Tool-making,  has been suggested before in similar works such as that of Chameleon (Lu et al.) referred to there as the \"Program Generator\" module or the Voyager (Wang et al) or the Creator (Qian et al). Tool-using, as well, is a widely used approach within the realm of LLMs to the extent that OpenAI has released models with function-calling abilities. With these points in mind, the novelty of the proposed work gets limited to using lighter models (cost effectiveness) for tool-using to reduce costs and introducing the function cache (reusibility). We appreciate that the authors have clearly credited the mentioned works with their tool-making ideas, particularly the Chameleon with its Python tools where the \"Program Verifier\" is the couterpart to the proposed \"Tool Verification\" step in the current paper.\n\nThe paper initially gives the impression that the functional cache and the dispatcher to be playing a central role in the proposed LATM. However, the dispatcher was only adopted in the stream of experiments and, even then, in a set of isolated tests. Accordingly it is not clear if the end-to-end performance of the LATM method across heterogenous tasks was evaluated trusting the reliability of the dispatcher and the functional cache to make the right calls.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)