A gent T uning: Enabling Generalized Agent Abilities for LLM s (2024)

Abstract: Open large language models (LLMs) with great performance in various tasks have significantly advanced the development of LLMs. However, they are far inferior to commercial models such as ChatGPT and GPT-4 when acting as agents to tackle complex tasks in the real world. These agent tasks employ LLMs as the central controller responsible for planning, memorization, and tool utilization, necessitating both fine-grained prompting methods and robust LLMs to achieve satisfactory performance. Though many prompting methods have been proposed to complete particular agent tasks, there is lack of research focusing on improving the agent capabilities of LLMs themselves without compromising their general abilities. In this work, we present AgentTuning, a simple and general method to enhance the agent abilities of LLMs while maintaining their general LLM capabilities. We construct AgentInstruct, a lightweight instruction-tuning dataset containing high-quality interaction trajectories. We employ a hybrid instruction-tuning strategy by combining AgentInstruct with open-source instructions from general domains. AgentTuning is used to instruction-tune the Llama 2 series, resulting in AgentLM. Our evaluations show that AgentTuning enables LLMs’ agent capabilities without compromising general abilities. The AgentLM-70B is comparable to GPT-3.5-turbo on unseen agent tasks, demonstrating generalized agent capabilities. We open source the AgentInstruct and AgentLM-7B, 13B, and 70B models at https://anonymous.4open.science/r/AgentTuning, serving open and powerful alternatives to commercial LLMs for agent tasks.

## introduction an agent refers to an entity capable of perceiving its environment, making decisions, and taking actions @xcite . traditional ai agents have been effective in specialized domains, but often fall short in adaptability and generalization. through alignment training, large language models (llms) @xcite , initially designed for language tasks, have displayed unprecedented capabilities in instruction following (ouyang et al., * az, ml, and rl made equal contributions. held-out tasks 2022), reasoning @xcite , planning, and even tool utilization @xcite . these capabilities make llms an ideal foundation for advancing ai agents toward broad, versatile functionality. recent projects such as au-togpt (richards, 2023), gpt-engineer (osika, 2023), and babyagi (nakajima, 2023) have employed llms as the core controllers, building powerful agents capable of solving complex problems in the real world. however, a recent study @xcite shows that open llms like llama @xcite , 2023a,b) and vicuna @xcite significantly lag behind in agent capabilities in complex, real-world scenarios when compared to gpt-3.5 and @xcite in figure 1 , though they have performed well in traditional nlp tasks and largely advanced the development of llms. the performance gap in agent tasks hampers the advancement of in-depth llm research and community innovation. existing studies on llms as agents have thus far largely focused on designing prompts or a framework for completing one particular agent task @xcite @xcite , rather than fundamentally enhancing the agent capabilities of the llms themselves. in addition, many efforts are dedicated to improving llms in specific aspects, involving fine-tuning the llms using datasets tailored to specific tasks @xcite @xcite . this overemphasis on specialized capabilities comes at the expense of the llms' general abilities and also compromises their generalizability. to fundamentally enable generalized agent abilities for llms, we introduce a simple and general approach agenttuning as shown in figure 2 . agent-tuning consists of two components: a lightweight instruct-tuning dataset agentinstruct and a hybrid instruction-tuning strategy that enhances the agent's capabilities while preserving its generalization ability. as shown in we employ agenttuning to fine-tune the open llama 2 series @xcite , whose performance on agent tasks are significantly worse that gpt-3.5, resulting in the agentlm-7b, 13b and 70b models. our empirical evaluations have the following observations. first, agentlm demonstrates strong performance on both held-in tasks in agentinstruct and unseen held-out agent tasks, suggesting robust generalization on agent capabilities. it also makes agentlm-70b comparable to gpt-3.5 on unseen agent tasks without compromising its performance on general nlp tasks, such as on mmlu, gsm8k, humaneval, and mt-bench. second, our analysis on the ratio of agent data with general data suggests that the general capabilities of llms are crucial for the generalization of agent tasks. training solely on agent data, in fact, leads to a decline in generalization performance. this can be explained by the fact that agent tasks demand that llms exhibit comprehensive abilities such as planning and reasoning. third, our error analysis on llama 2 and agentlm shows that agenttuning significantly reduces instances of basic mistakes such as formatting errors, duplicated generation, and refusal to answer. this suggests that the model inherently possesses the capability to tackle agent tasks, and agenttuning indeed enables the llms' agent abil-task inst. from # inst. # filt. traj. ## the agenttuning approach to date, there is no end-to-end attempt to improve the general agent abilities of llms across heterogeneous tasks. most existing agent studies focused on either prompting one particular llm or compiling a llm-based framework for completing an agent task, such as building a web agent in web-shop @xcite and @xcite . according to @xcite , all open llms are far behind of commercial ones such as gpt-4 and chatgpt in terms of acting as agents though these models, such as llama 2, have demonstrated strong performance across various benchmarks. the goal of this work is to improve the generalized agent abilities of llms while at least maintaining their general llm capacities such as their performance on mmlu, gsm8k, and humaneval. we present agenttuning to achieve this goal, the first step of which is to build the agentinstruct dataset that is used in the second step to instruction tune the llms. we carefully experiment and design these two steps such that the llms obtain good performance in (unseen) generalized agent task types while remaining good in general tasks. ## conclusion in this work, we study how to enable generalized agent abilities for llms, bridging the disparity be-tween open and commercial llms on agent tasks. we present the agenttuning approach to achieve this goal. agenttuning first introduces the agentinstruct dataset covering 1,866 verified agent interaction trajectories and then designs an instructiontuning strategy with the mixture of agentinstruct and general-domain instructions. we generate the open agentlm by employing agenttuning to tune the llama 2 models. agentlm exhibits strong performance on unseen agent tasks while preserving their general abilities on mmlu, gsm8k, hu-maneval, and mt-bench. to date, agentlm-70b is the first open llm that matches gpt-3.5-turbo on agent tasks.