TPTU: Task Planning and Tool Usage of Large Language Model-based AI
  Agents
Authors: name: Jingqing Ruan; name: Yihong Chen; name: Bin Zhang; name: Zhiwei Xu; name: Tianpeng Bao; name: Guoqing Du; name: Shiwei Shi; name: Hangyu Mao; name: Xingyu Zeng; name: Rui Zhao
Published: 2023-08-07 09:22:03

Abstract: With recent advancements in natural language processing, Large Language
Models (LLMs) have emerged as powerful tools for various real-world
applications. Despite their prowess, the intrinsic generative abilities of LLMs
may prove insufficient for handling complex tasks which necessitate a
combination of task planning and the usage of external tools. In this paper, we
first propose a structured framework tailored for LLM-based AI Agents and
discuss the crucial capabilities necessary for tackling intricate problems.
Within this framework, we design two distinct types of agents (i.e., one-step
agent and sequential agent) to execute the inference process. Subsequently, we
instantiate the framework using various LLMs and evaluate their Task Planning
and Tool Usage (TPTU) abilities on typical tasks. By highlighting key findings
and challenges, our goal is to provide a helpful resource for researchers and
practitioners to leverage the power of LLMs in their AI applications. Our study
emphasizes the substantial potential of these models, while also identifying
areas that need more investigation and improvement.