G entopia. AI : A Collaborative Platform for Tool-Augmented LLM s (2023)

Abstract: Augmented Language Models (ALMs) empower large language models with the ability to use tools, transforming them into intelligent agents for real-world interactions. However, most existing frameworks for ALMs, to varying degrees, are deficient in the following critical features: flexible customization, collaborative democratization, and holistic evaluation. This paper proposes Gentopia, a lightweight and extensible framework for ALMs. Gentopia allows the flexible customization of agents through simple configurations, seamlessly integrating various language models, task formats, prompting modules, and plugins into a unified paradigm. Furthermore, we establish Gentpool, a public platform enabling the registration and sharing of user-customized agents. Agents registered in Gentpool are composable such that they can be assembled together for agent collaboration, advancing the democratization of artificial intelligence. To ensure high-quality agents, Gentbench, an integral component of Gentpool, is designed to thoroughly evaluate user-customized agents across diverse aspects such as safety, robustness, efficiency, etc. We release Gentopia on Github and will continuously move forward.

## introduction there is a burgeoning trend in research towards augmenting large language models (llms) with external tools, enabling them to access up-to-date databases @xcite , perform arithmetic operations @xcite , navigate websites @xcite , develop software @xcite , etc. this integration of tools marks a departure from traditional language modeling, heralding a new era of intelligent agents capable of interacting with the real world. several projects and frameworks have been proposed to build tool-augmented language models (alms), or "agents", including au-togpt (richards, 2023), superagi @xcite , hugginggpt @xcite , @xcite , langchain @xcite , semantic kernel @xcite , and minichain @xcite . each of these methods is deficient, to varying degrees, in the following critical features. this paper proposes gentopia, a lightweight and extensible framework for the research on alms. gentopia allows practitioners to customize an agent with a single configuration file, greatly simplifying the process of building, tuning, sharing, and evaluating agents. various language models, task formats, prompting modules, and plugins are integrated into a unified paradigm, without loss of flexibility for agent customization. in addition, we believe the collaboration between usercustomized agents can contribute to the democratization of ai. hence, gentpool, a platform for agent registration and sharing is established. agents registered in gentpool can be hierarchically assembled together, enabling the collaboration of multiple agents. gentpool is accompanied by a unique benchmark, gentbench, that can probe customized agents with a holistic evaluation in terms of safety, robustness, efficiency, multilingual capabilities, etc. notably, it is flexible for users to customize the evaluation by configuration. ## background a variety of agent projects have been proposed, targeting an array of diverse tasks, including automated web navigation @xcite , database management @xcite , automated game playing @xcite , collaborative software development @xcite , etc. meanwhile, researchers are enthusiastically developing generalist agents that can perform well for multiple tasks. autogpt @xcite is the first experimental application for fully autonomous llm agents. superagi @xcite ) provides a more user-friendly interface, improved memory management, optimized token usage, and looping detection heuristics. toolllm @xcite features fine-tuning llms to use massive tools. hugging-gpt @xcite expands the potential of artificial intelligence by linking to extensive ai models hosted on huggingface, thereby supporting a range of ai tasks in diverse domains and modalities, including language, vision, and speech. however, given the unique requirements and customization that each specific domain demands, tools and prompting paradigms developed for a particular task may prove irrelevant or ineffective for others. this poses a significant challenge to the development of a single, all-encompassing agent that performs efficiently across all tasks. consequently, there is a rising need for the collaboration of multiple specialized agents. for example, metagpt @xcite models the entire process of software development with carefully orchestrated standard operating procedures (sops) to generate longer program codes for game development. in our work, gentopia provides smooth support for the composition of agents, which is handy for agent collaboration in different scenarios. ## design and implementation gentopia aims to provide easy assembly, sharing, and interaction of task-specialized agents. a single step to improve agent capability and efficiency gives plural contributions to interacted agents, fostering collective growth toward greater intelligence. ## case study we briefly showcase the process of building an agent, who acts as an experienced and visionary entrepreneur for the users to create business plans in gentopia foot_0 . further, the users can evaluate the created agent and share it publicly into gentpool. ## conclusion this paper introduces gentopia.ai, an opensource platform designed for tool-augmented llms. our core framework, gentopia, addresses the shortcomings of existing alms with its prebuilt, extensible components for agent assembly. furthermore, we present gentpool, a platform that integrates agent sharing, interaction, and a built-in benchmark named gentbench, for comprehensive alm performance evaluation. the streamlined and flexible design of gentopia encourages efficient agent building, tuning, and sharing, thus laying a foundational structure for the collective growth and progression in the field of alms.