T ool P lanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedback (2024)

Abstract: Recently, tool-augmented LLMs have gained increasing attention. Given an instruction, tool-augmented LLMs can interact with various external tools in multiple rounds and provide a final answer. However, previous LLMs were trained on overly detailed instructions, which included API names or parameters, while real users would not explicitly mention these API details. This leads to a gap between trained LLMs and real-world scenarios. In addition, most works ignore whether the interaction process follows the instruction. To address these issues, we constructed a training dataset called MGToolBench, which contains statement and category-level instructions to better reflect real-world scenarios. In addition, we propose ToolPlanner, a two-stage reinforcement learning framework that utilizes path planning and two feedback mechanisms to enhance the LLM’s task completion and instruction-following capabilities. Experimental results show that ToolPlanner significantly improves the Match Rate, Pass Rate and Win Rate by 26.8%, 20.2%, and 5.6% compared to the SOTA model. Human evaluation verifies that the multi-granularity instructions can better align with users’ usage habits. Our data and code will be released upon acceptance.

## introduction recently, tool-augmented large language models (llms) have shown their remarkable ability in utilizing various external tools, such as hugging-face models @xcite , real-world applications @xcite , and massive apis @xcite . to simulate complex realworld tasks, previous studies have continuously increased the size of the external tool pool and the complexity of task instructions @xcite real user 1. i'm going to disneyland and stay in a nearby hotel, any suggestions? 2. i'm going to new york for a business trip next monday, show me the flight options. ## conclusion in this work, we propose toolplanner, a twostage rl framework that utilizes task completion feedback and instruction-following feedback to enhance llms' reasoning and tool usage abilities. additionally, we constructed a training dataset called mgtoolbench, which uses multi-granularity instructions to simulate the usage habits of real users. experimental results show that toolplanner significantly improves the match rate, pass rate and win rate by 26.8%, 20.2%, and 5.6%. human evaluation verifies that the multi-granularity instruction mechanism can generate instructions that better align with user habits. by addressing the challenges of tool-augmented llms in following user instructions at different granularities, our framework shows strong instruction-following and task-completion abilities. we hope that mgtoolbench can serve as a helpful resource for simulating real-world scenarios and help future research improve the practical application ability of tool-augmented llms.