M odel S cope-Agent: Building Your Customizable Agent System with Open-source Large Language Models (2023)

Abstract: Large language models (LLMs) have recently demonstrated remarkable capabilities to comprehend human intentions, engage in reasoning, and design planning-like behavior. To further unleash the power of LLMs to accomplish complex tasks, there is a growing trend to build agent frameworks that equips LLMs, such as ChatGPT, with tool-use abilities to connect with massive external APIs. In this work, we introduce ModelScope-Agent, a general and customizable agent framework for real-world applications, based on open-source LLMs as controllers. It provides a user-friendly system library, with a customizable engine design to support model training on multiple open-source LLMs, while also enabling seamless integration with both model APIs and common APIs in a unified way. To equip the LLMs with tool-use abilities, a comprehensive framework has been proposed spanning tool-use data collection, tool retrieval, tool registration, memory control, customized model training, and evaluation for practical real-world applications. Finally, we showcase ModelScopeGPT, a real-world intelligent assistant of ModelScope Community based on the ModelScope-Agent framework, which is able to connect open-source LLMs with more than 1000 public AI models and localized community knowledge in ModelScope. The ModelScope-Agent online demo, library are now publicly available.

## introduction large language models @xcite @xcite have gradually become common ai assistants that demonstrate great potential in comprehending human intentions, performing complex reasoning tasks, and enabling content creation. despite the rapid advancements of open-source llms, e.g., llama @xcite and chat-glm (thudm, 2023), they still remain limited in performing complex tasks, such as following user instructions to use external tools and capture up-to-date information. to further unleash the power of llms for realworld practical applications, a rising trend of current research @xcite @xcite @xcite begins to enable llms with tool-use abilities towards building an ai agent. these include hug-ginggpt @xcite , visual-chatgpt @xcite and @xcite for connecting with huggingface models, toolalpaca @xcite and toolllama @xcite for using massive common apis such as weather forecast and search engine. these methods either directly rely on closed-source counterparts like chatgpt or focus on certain types of api tools. recently, there have also been public releases of ai agents, such as auto-gpt foot_0 , langchain 5 and transformers agent @xcite , which enable llms, such as chatgpt or gpt-4, to use tools and solve complex ai tasks. however, these agents are mainly built with closed-source llms and how to build a customizable agent system with open-source llms remains largely unexplored. in this work, we present modelscope-agent, a general and customizable agent system for realworld applications, based on open-source llms as controllers. modelscope 6 is a public ml community, that seeks to bring together the most advanced machine learning models from the ai community, and streamlines the process of leveraging ai models in real-world applications. modelscope-agent provides a flexible and user-friendly system library, with a customizable engine design to support model training on multiple open-source llms, while also enabling seamless integration with both model apis and common apis in a unified way. it features an llm-centric system design, which includes open-source llms as core controller, and further interact with a tool-use module and a memory module to accomplish complex tasks. at the core of modelscope-agent , the library supports flexible selection and training on various open-source llms, such as llama @xcite , chatglm (thudm, 2023), chat-plug @xcite and other customized llms in modelscope. for tool use, modelscope-agent provides a default tool library, which supports diverse ai model apis across nlp, cv, audio and multi-model fields, as well as massive common apis such as search engine. it also supports registering new self-defined api plugins and automatic api retrieval from the large tool library. it is easy for users to customize their most appropriate llms, local api tools and functions to develop real-world applications. moreover, a memory module is also introduced to better store and manage the system message, user history, in-context examples, tool message and localized knowledge. to enable the open-source llms to better control the whole agent system, we further propose a comprehensive framework of tool-use data collection, customized model training, evaluation and deployment. notably, we release a comprehensive tool-enhanced dataset msagent-bench, which consists of 598k dialogues with various api categories, multi-turn api calls, api-oriented qa, and api-agnostic instructions in both english and chinese. a simple training strategy of weighted lm, that enhances the training of generation of api name and parameters, is used to better ensure the correctness of api calls. besides, an evaluation framework is also supported in our library to examine the tool-use abilities of the trained models in different aspects. furthermore, we applied modelscope-agent in a real-world application of modelscope community namely modelscopegpt, which is able to connect open-source llms with more than 1000 public ai models and access localized community knowledge in modelscope for community qa. to summarize, modelscope-agent is a general and customizable agent system designed for developers to harness the power of open-source llms. the library targets the following goals: modelscope-agent is in continual development by the engineers at modelscope and is released under an apache 2.0 license. full documentation is available through the project website. ## the modelscope agent modelscope-agent is designed to facilitate developers in building customizable agent systems based on open-source llms. the overall system architecture is shown in figure 1 . it includes open-source llms as controller, a tool-use module and a memory module to interact with. given human instruction, the agent, which adopts the selected llm as the controller, will automatically plan tasks, selectively use tools, leverage knowledge in memory, and finally provide helpful responses to users. ## evaluation our evaluation system, msagent-eval, comprises two modules: an automatic evaluation framework that comprehensively evaluates the api usability of the agents and a human evaluation framework implemented by an agent arena that reflects the preferences of human users. ## usage example of modelscopegpt in this section, we showcase a successful application of modelscope community, mod-elscopegpt foot_5 , based on our modelscope-agent. ## conclusion modelscope-agent aims to facilitate building ai agent applications and research based on opensource llms by providing a general and customizable agent framework covering flexible system design, data collection, model training, evaluation and usage examples in real-world applications. it provides an open-source, community-driven library for ai agent learning and best practices for building an agent system with open-source llms. we hope modelscope-agent can help pave the way towards a new era of ai agent. dataset consisted of 360 conversations with 2059 text snippets as the references to be compared with the agent prediction, which comprise 798 api requsts and 1261 plain text answers according to the previous calling results. we compare the models trained in our proposed modelscopegpt. the automatic evaluation results are shown in