SlowGuess/NeurIPS_2023_OCR huggingface dataset:
CAMEL: Communicative Agents for “Mind” Exploration of Large Language Model Society
https://www.camel-ai.org
Guohao Li*
Hasan Abed Al Kader Hammoud*
Hani Itani*
Dmitrii Khizbullin
Bernard Ghanem
King Abdullah University of Science and Technology (KAUST)
The rapid advancement of chat-based language models has led to remarkable progress in complex task-solving. However, their success heavily relies on human input to guide the conversation, which can be challenging and time-consuming. This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents, and provides insight into their "cognitive" processes. To address the challenges of achieving autonomous cooperation, we propose a novel communicative agent framework named role-playing. Our approach involves using inception prompting to guide chat agents toward task completion while maintaining consistency with human intentions. We showcase how role-playing can be used to generate conversational data for studying the behaviors and capabilities of a society of agents, providing a valuable resource for investigating conversational language models. In particular, we conduct comprehensive studies on instruction-following cooperation in multi-agent settings. Our contributions include introducing a novel communicative agent framework, offering a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems, and open-sourcing our library to support research on communicative agents and beyond: https://github.com/camel-ai/camel.
"What magical trick makes us intelligent? The trick is that there is no trick. The power of intelligence stems from our vast diversity, not from any single, perfect principle."
- Marvin Minsky, The Society of Mind, p. 308
Confronted with the complexities of real-world tasks, solving them often requires multiple steps. The rapid progress of chat-based large-scale language models (LLMs) has yielded remarkable achievements in complex task-solving [82, 84, 116, 89, 5, 10, 122, 13]. Nevertheless, it is worth noting that their success is heavily reliant on human input to guide the conversation in the right direction. This reliance necessitates users to provide relevant and precise prompts based on their intentions and the chat agent's feedback. This can be challenging, time-consuming, and sometimes impossible. Crafting effective prompts often demands a deep understanding and expertise of a particular domain of knowledge. Consider an individual who lacks trading expertise; they would find it difficult to create suitable prompts for directing a chat agent to develop a trading application. This predicament is raising a crucial question: can we replace human intervention with an autonomous communicative agent capable of steering the conversation toward task completion with minimal human supervision? To tackle this issue, it is crucial to conduct more research exploring the potential,
capabilities, and limitations of communicative agents that operate entirely on their own to complete tasks. Understanding how multiple agents interact with each other is important for anticipating the future of artificial intelligence. The dynamics of collaborating or competing agents play a key role in determining the success of AI systems [6, 26, 27, 84, 99, 9, 10].
This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents and provide insight into their "cognitive" processes. Several challenges arise when asking a society of agents to autonomously cooperate on completing tasks. Examples we encountered in our preliminary analysis include role flipping, assistant repeating instructions, flake replies, and infinite loop of messages. Therefore, it is critical to investigate ways to align these models with human intentions and to explore means enabling their effective cooperation. To address these issues, we propose a novel cooperative agent framework named role-playing to automate cooperation between communicative agents. Specifically, our proposed approach involves using role-playing with inception prompting to autonomously guide the communicative agents toward task completion. Only a preliminary idea is needed from human to guide the conversations toward complex task-solving.
Our library, which we make publicly available, provides modular functionality, and includes implementations of different agents, examples of well-crafted prompts, and data explorers. We hope our library serves as a ground for future research in various areas such as multi-agent systems, cooperative AI, game theory simulations, social analysis, AI ethics, AI alignment, and beyond. In addition, our role-playing method provides a highly scalable way to generate conversational data for studying the behaviors and capabilities of chat agents. We showcase how role-playing can be used to let chat agents communicate with each other for task completion and record their conversations for behavior analysis and capability understanding. In particular, we consider two cooperative scenarios of role-playing and generate two large conversational, task-oriented, and instruction-following datasets: AI Society and Code. We also use our framework to collect two single-turn question-answer datasets, Math and Science, for LLM ability emergence study. Furthermore, we generate a Misalignment dataset that is a simulation of possible malicious applications which demonstrate the potential risks of an unaligned autonomous agent system. The datasets offer a valuable resource for investigating conversational language models, enabling them to comprehend and react to human language more effectively. Furthermore, our role-playing offers a scalable method of creating conversational instruction-following data, which can potentially enhance the development of more advanced language models. We show that solutions derived from our role-playing framework outperform those generated in a single shot by gpt-3.5-turbo [82] in both GPT4 and human evaluations. We also study knowledge emergence in LLMs by fine-tuning LLaMA [117] on progressively growing datasets generated through our framework. Additionally, we evaluate our code generation capabilities through benchmarking our final model on HumanEval [18] and HumanEval+ [69].
Contributions. Our contributions are fourfold: (1) We introduce a novel cooperative agent framework, role-playing, that allows communicative agents to collaborate autonomously toward completing tasks while requiring minimal human intervention; (2) Our framework offers a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems. It illuminates the challenges of achieving autonomous cooperation, and provides strategies for addressing them. We showcase the potential power of multi-agent collaboration for complex-task solving; (3) We demonstrate the significant emergence of LLM training abilities by utilizing the datasets we have collected from simulating four distinct agent collaboration scenarios; (4) We have open-sourced our library, containing implementations of various agents, data generation pipelines, data analysis tools, and collected datasets, to support research on communicative agents and beyond.
Communicative Agents. Communication between agents has been studied for a long time [76, 77]. There are many ways to facilitate communication between agents, and with agents [29, 90, 97]. Among these, natural language is considered the most natural form of communication [97]. By enabling agents to function as communicators themselves, they become capable of solving complex tasks [113, 85, 72, 3, 30, 111, 79, 41, 28, 102, 80, 106, 35, 49, 2, 51, 1, 55, 50, 65, 92]. Communication between AI agents can occur in a competitive setting [115, 108] or a cooperative setting [40, 27, 11, 137, 70]. Cooperative AI refers to artificial intelligence systems that are designed to work together with humans and other AI systems to achieve common goals [24, 125]. Cooperative AI systems take into account the needs and capabilities of other agents in the system and actively seek to collaborate and coordinate their actions with them, which has many potential benefits, including
increased efficiency, improved decision-making, and the ability to tackle complex problems that are beyond the reach of any single agent. However, designing effective cooperative AI systems is still an active area of research, as it requires addressing a range of technical, ethical, and social challenges [27]. Our work enables communicative agents to engage in a conversation and cooperate with each other to solve assigned tasks. The agents, each assigned a distinct role, are expected to apply their expertise and knowledge to solve their common task.
Instructional LLMs and Prompt Engineering. LLMs are trained on diverse text data and excel in text completion, with various downstream NLP applications [12, 22, 47, 131, 117]. However, InstructGPT suggests that LLMs may not align with user intent, proposing reinforcement learning from human feedback (RLHF) [23] and Instruction Fine-Tuning (IFT) [121] to improve LLMs' relevance and appropriateness to user instructions. Special types of instruction or prompting methods, such as Chain-of-Thought (CoT) [123], zero-shot-CoT [61], and ReAct [126], have recently been developed to enhance the performance of LLMs on reasoning, arithmetic and decision making tasks [134, 118, 52, 73, 31, 103, 43, 64, 132, 46, 133, 105, 128, 25, 81, 109]. These techniques underpin the impressive capabilities of recent dialogue LLMs [106, 116, 36, 9, 82, 13], which aim to simulate human-like conversations and provide personalized and interactive experiences for users, exhibiting the behavior of conversational AI agents [33]. However, generating instruction datasets is a crucial challenge in building instruct-based LLMs, with existing datasets ranging from crowdsourced to generated. Hand-crafted instruction instances are available in [120], while leveraging previously crowdsourced NLP datasets is a less labor-intensive curation approach [121, 71, 78, 53]. LLMs have been explored for data generation in [101, 63, 68, 114], and Self-Instruct [119] proposes a semi-automated process for instruction instance generation. Unnatural-Instruction [48] collects instruction instances by prompting a language model with only three seed examples and paraphrasing the generated instances to expand the dataset. There is also a large chunk of work that has proposed methods for automatic dataset creation [67, 57, 19, 75, 20, 98, 59, 96, 129, 62, 130, 86, 8]. Another important challenge is prompt engineering. The quality of the prompt used to guide LLMs significantly affects its performance [91, 12, 66]. While LMs pre-trained on large data can implicitly learn tasks with few-shot prompting, hand-crafted prompts may not always suffice. Automated prompt generation methods have been proposed, such as gradient-guided search [104], mining-based and paraphrasing-based techniques [54], a meta-prompt [93], and automatic instruction selection and generation [136]. In this work, we introduce a conversational LLM auto-prompting method called Inception Prompting, which enables agents to prompt each other to solve tasks through Role-Playing. The AI user continuously provides instructions to the AI assistant for task-solving. This enables us to save the streaming instruction-solution pairs and create diverse, instructional, conversational, and task-oriented datasets. These datasets can be used to analyze the behavior and capabilities of LLMs and for future research for fine-tuning LLMs with conversational instructions.
AI Alignment. AI alignment is a field that aims to ensure that AI systems adhere to their intended goals, interests, and values, as envisioned by their designers [4, 39, 110, 32, 38, 74, 10]. The first attempt at AI alignment was made through the "Three Laws of Robotics," which was introduced by Isaac Asimov in his science fiction stories [6]. Developing aligned AI systems is crucial for achieving desired objectives while avoiding unintended consequences. Research in AI alignment focuses on discouraging AI models from producing false, offensive, deceptive, or manipulative information that could result in various harms [56, 112, 42, 37]. Achieving a high level of alignment requires researchers to grapple with complex ethical, philosophical, and technical issues. We conduct extensive experiments to study different role-playing situations, which probe the alignment of LLMs.
In this paper, we focus on studying communicative agents under cooperative settings where they share common interests. In particular, we study the assistant-user scenario, where a preliminary idea is given at the start. Agents will conceptualize the idea into a specific task and complete it autonomously through conversations.
3.1 Role-playing Framework
"What's the most resilient parasite? An Idea. A single idea from the human mind can build cities. An idea can transform the world and rewrite all the rules. Which is why I have to steal it."
- Dom Cobb, Inception
Figure 1: CAMEL Role-Playing Framework. Our role-playing setup starts with the human user having an idea they want to implement, e.g. develop a trading bot for the stock market. The roles involved in this task would be an AI assistant agent who is a python programmer and an AI user agent who is a stock trader. The task is made more specific using our task specifier agent, leading to a well-defined task for the assistant to solve. Both AI user and AI assistant are provided with the specified task, after which they collaboratively communicate by chatting with each other in an instruction-following fashion to solve the specified task.
Our proposed framework is a novel role-playing approach for studying multiple communicative agents. Specifically, we concentrate on task-oriented role-playing that involves one AI assistant and one AI user. After the multi-agent system receives a preliminary idea and the role assignment from human users, a task-specifier agent will provide a detailed description to make the idea specific. Afterwards, the AI assistant and AI user will cooperate on completing the specified task through multi-turn conversations until the AI user determines the task is done. The AI user is responsible for giving instructions to the AI assistant and directing the conversation toward task completion. On the other hand, the AI assistant is designed to follow the instructions from the AI user and respond with specific solutions. The whole role-playing framework is depicted in Figure 1.
Human Input and Task Specifying. The role-playing session will be instantiated from an idea and selected roles by humans. As an example in Figure 1, a human has a preliminary idea to develop a trading bot for the stock market. Humans may or may not have the knowledge about how the idea can be realized. What is needed is only to designate the potential roles that can implement the idea. For instance, a Python Programmer could collaborate with a Stock Trader to realize the idea of developing a trading bot for the stock market. After the idea and roles are determined, the task specifier agent will brainstorm a specific task that the AI Assistant role can help with the AI user role to complete based on the input idea. An example of a specified task in this scenario could be: develop a trading bot with a sentiment analysis tool that can monitor social media platforms for positive or negative comments about a particular stock, and execute trades based on sentiment analysis results. The main motivation for introducing a task specifier is that conversational agents usually require a concrete task prompt for realizing the task which might be challenging or time-consuming for a non-domain expert. Therefore, the task specifier agent serves as an enhanced imagination module for the idea implementation. Please note that, when studying our framework at a large scale for AI society and Code scenarios, we generate roles and ideas automatically by prompting LLMs instead of relying on human inputs. For our generated Math and Science datasets we generated problem topics, subtopics, and problems automatically by prompting LLMs.
AI Assistant-User Role Assignment. After the task specification, The AI assistant role and the AI user role will be assigned to the user agent and the assistant agent correspondingly to complete the specified task. In practice, a system message is passed to each agent declaring their role. We refer to the assistant system promptMESSAGE by  $\mathcal{P}_{\mathcal{A}}$  and that of the user by  $\mathcal{P}_{\mathcal{U}}$ . The system messages are passed to the agents before the conversations start. Let  $\mathcal{F}_1$  and  $\mathcal{F}_2$  denote two large-scale autoregressive language models [82]. When the system message is passed to those models respectively, we
obtain  $\mathcal{A} \gets \mathcal{F}_1^{\mathcal{P}_A}$  and  $\mathcal{U} \gets \mathcal{F}_2^{\mathcal{P}_U}$  which are referred to as the assistant and user agents respectively. In Figure 1, the AI assistant and the AI user are assigned the roles of a Python Programmer and a Stock Trader at the beginning of the role-playing session respectively. The AI user serves as a task planner, engaging in interactive planning to determine feasible steps for the AI assistant to execute. Meanwhile, the AI assistant acts as a task executor, offering solutions, executing planned steps, and providing responses to the AI user.
Conversation Towards Task-Solving. After the role assignment is completed, the AI assistant  $\mathcal{A}$  and AI user  $\mathcal{U}$  will collaborate in an instruction-following manner to accomplish the task. In the AI assistant-user scenario, the AI user is responsible for providing instructions, and the assistant is expected to respond with a solution that fulfills the instructions. Formally, we denote the user instruction message obtained at time  $t$  by  $\mathcal{I}_t$  and the assistant solution by  $\mathcal{S}_t$ . The set of conversational messages obtained up until time  $t$  is denoted by Equation (1) shown below:
$$
\mathcal {M} _ {t} = \left\{\left(\mathcal {I} _ {0}, \mathcal {S} _ {0}\right), \dots , \left(\mathcal {I} _ {t}, \mathcal {S} _ {t}\right) \right\} = \left\{\left(\mathcal {I} _ {i}, \mathcal {S} _ {i}\right) \right\} | _ {i = 0} ^ {t} \tag {1}
$$
At the next time step,  $t + 1$ , the AI user  $\mathcal{U}$  takes the historical conversation message set  $\mathcal{M}_t$  and provides a new instruction  $\mathcal{I}_{t + 1}$ , as shown in Equation (2). The produced instruction message  $\mathcal{I}_{t + 1}$  is then passed, along with message set  $\mathcal{M}_t$ , to the AI assistant  $\mathcal{A}$ . The AI assistant will then respond with a solution, denoted by  $S_{t + 1}$  in Equation (3):
$$
\mathcal {I} _ {t + 1} = \mathcal {U} (\mathcal {M} t) \tag {3}
$$
After obtaining the solution  $S_{t + 1}$  to the instruction  $\mathcal{I}_{t + 1}$ , the message set is updated using Equation (4) to obtain  $\mathcal{M}_{t + 1}$ :
$$
\mathcal {M} _ {t + 1} \leftarrow \mathcal {M} _ {t} \cup \left(\mathcal {I} _ {t + 1}, \mathcal {S} _ {t + 1}\right) \tag {4}
$$
Note that the formulation above not only models AI-AI communicative scenarios, but it can also be easily extended to model human-AI communication or communication between more than two agents. Specifically, we can use message-passing graphs to model communication between an arbitrary number of agents. In Figure 1, we observe that the AI user initiates the installation and import of essential Python libraries for sentiment analysis and stock trading by instructing the AI assistant through conversations. This example is drawn from our experiments, and the entire conversation is available in the Appendix.
Critic-In-The-Loop. To enhance the controllability of the role-playing framework, we introduce a critic agent capable of selecting proposals from or providing feedback to the role-playing agents. This enables tree-search-like decision-making for task-solving. In practice, the critic can be either an AI agent or a human. The detailed implementation and case studies can be found in the Appendix.
3.2 Inception Prompting
Since prompt engineering is crucial to our role-playing framework, this section delves deeply into our prompting techniques. Our prompt engineering occurs solely at the beginning of role-playing, for task specification and role assignment. Once the conversation phase commences, the AI assistant and AI user prompt each other automatically in a loop until termination. As such, we refer to our technique as Inception Prompting. Our Inception prompt consists of three prompts: the task specifier prompt  $\mathcal{P}_{\mathcal{T}}$ , the assistant system prompt  $\mathcal{P}_{\mathcal{A}}$ , and the user system prompt  $\mathcal{P}_{\mathcal{U}}$ . As an example, we consider the inception prompt of the AI Society scenario. The templates for these prompts of AI Society role-playing are shown in Figure 2. The task specifier prompt contains information about the roles of the AI assistant and AI user in the role-playing session. Therefore, the task specifier agent can take a preliminary task/idea as input and generate a specific task using imagination. The AI assistant system prompt  $\mathcal{P}_{\mathcal{A}}$  and the AI user system prompt  $\mathcal{P}_{\mathcal{U}}$  are mostly symmetrical and include information about the assigned task and roles, communication protocols, termination conditions, and constraints or requirements to avoid unwanted behaviors. The prompt designs for both roles are crucial to achieve autonomous cooperation between agents. It is non-trivial to engineer prompts that ensure agents act in alignment with our intentions. We take the prompt templates from the AI Society in Figure 2 as an example to explain our key design choices. The prompts used for the Code scenario follow a similar sprint as the AI society scenario, but with some additional engineering related to programming languages. More details in the Appendix.
AI Society Inception Prompt
Task Specifier Prompt:
Here is a task that <ASSISTANT ROLE> will help <USER ROLE> to complete: <TASK>.
Please make it more specific. Be creative and imaginative.
Please reply with the specified task in <WORD_LIMIT> words or less. Do not add anything else.
Assistant System Prompt:
Never forget you are a
<ASSISTANT ROLE> and I am a
<USER ROLE>. Never flip roles!
Never instruct me!
We share a common interest in
collaborating to successfully
complete a task.
You must help me to complete the task.
Here is the task: <TASK>. Never
forget our task!
I must instruct you based on your
expertise and my needs to complete the task.
I must give you one instruction at a time.
You must write a specific solution that appropriately completes the requested instruction.
You must decline my instruction
honestly if you cannot perform the
instruction due to physical, moral, legal reasons or responsibility.
legal reasons or your capability and explain the reasons.
Unless I say the task is completed
you should always start with:
Solution: <YOUR SOLUTION>
<YOUR SOLUTION> should be
specific, and provide preferable
implementations and examples for task-solving.
Always end <YOUR SOLUTION> with:
Next request.
User System Prompt:
Never forget you are a <USER ROLE> and I am a <ASSISTANT ROLE>.
Never flip roles! You will always instruct me.
We share a common interest in collaborating to successfully
I must help you to complete the task.
Here is the task: <TASK>. Never forget our task!
You must instruct me based on my expertise and your needs to
complete the task ONLY in the following two ways:
1. Instruct with a necessary input:
Instruction: <YOUR_INSTRUCTION>
Input: <YOUR_INPUT>
2. Instruct without any input:
Input: None
The "Instruction" describes a task or question. The paired
"Input" provides further context or information for the
requested "Instruction".
You must give me one instruction at a time.
I must write a response that appropriately completes the
requested instruction.
I must decline your instruction honestly if I cannot perform
the instruction due to physical, moral, legal reasons or my
capability and explain the reasons.
You should instruct me not ask me questions.