SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Now you must start to instruct me using the two ways described above.
Do not add anything else other than your instruction and the
optional corresponding input!
Keep giving me instructions and necessary inputs until you think the task is completed.
When the task is completed, you must only reply with a single
word <CAMEL_TASK_DONE>.
Never say <CAMEL_TASK_DONE> unless my responses have solved your
Figure 2: Inception Prompt of AI Society Role-Playing. This shows the task specifier prompt, assistant system prompt, and user system prompt which are used for studying the AI society scenario.
Prompt Engineering. To delve deeper into the details in Figure 2, we start by chunking the various parts of the AI assistant system prompt  $\mathcal{P}_{\mathcal{A}}$  shown below:
- Never forget you are a <ASSISTANT ROLE> and I am a <USER ROLE>. This assigns the chosen role to the assistant agent and provides it with information about the user's role.
- Never flip roles! Never instruct me! This prevents agents from flipping roles. In some cases, we have observed the assistant and the user switching roles, where the assistant suddenly takes control and instructs the user, and the user follows those instructions.
- You must decline my instruction honestly if you cannot perform the instruction due to physical, moral, legal reasons or your capability and explain the reasons. This prohibits the agent from producing harmful, false, illegal, and misleading information.
- Unless I say the task is completed, you should always start with: Solution: <YOUR SOLUTION>. <YOUR SOLUTION> should be specific, and provide preferable implementations and examples for task-solving. This encourages the assistant always responds in a consistent format, avoiding any deviation from the
structure of the conversation, and preventing vague or incomplete responses, which we refer to as flake responses, such as "I will do something".
- Always end your solution with: Next request. This ensures that the assistant keeps the conversation going by requesting a new instruction to solve.
For the AI user system prompt  $\mathcal{P}_{\mathcal{U}}$ , we strive to maintain as much symmetry as possible with respect to the AI assistant system prompt. Apart from the opposite role assignment, the user system prompt differs from the assistant prompt in the following ways:
- You must instruct me ... to complete the task ONLY in the following two ways: 1. Instruct with a necessary input: ...; 2. Instruct without any input: ... This follows the typical data structure of instruction-following, which allows the generated instruction-solution pairs to be easily used for fine-tuning LLMs.
- Keep giving me instructions and necessary inputs until you think the task is completed. When the task is completed, you must only reply with a single word <CAMEL_TASK_DONE>. We introduce an end-of-task token, namely, <CAMEL_TASK_DONE>. This token is used once the user believes the task is done. This ensures that the chat is terminated when the user is satisfied. Without doing so, the agents might fall into a chatting loop where they keep on saying "thank you" to each other or "goodbye" indefinitely.
In this section, we will discuss the various experiments that we conducted to arrive at our final design choices. Specifically, we will examine the interesting observations, challenging issues, and several examples we have encountered while enabling agents to communicate with each other under different prompt design choices to achieve autonomous cooperation. In our experiments, we employed two gpt-3.5-turbo agents, referred to as LLM agents for simplicity, with Inception Prompts, as described in Section 3.2, to simulate assistant-user cooperation. For our analysis, we set our attention on AI Society setting. We also gathered conversational data, named CAMEL AI Society and CAMEL Code datasets and problem-solution pairs data named CAMEL Math and CAMEL Science and analyzed and evaluated their quality. Moreover, we will discuss potential extensions of our framework and highlight both the risks and opportunities that future AI society might present.
Data Generation Prompts of AI Society
AI Society
Assistant Role Generation Prompt:
You are a helpful assistant that can play many different roles. Now please list <NUM_roles> different roles that you can play with your expertise in diverse fields. Sort them by alphabetical order. No explanation required.
User Role Generation Prompt:
Please list <NUM_ROLES> most common and diverse groups of internet users or occupations. Use singular form. No explanation. Sort them by alphabetical order. No explanation required.
Task Generation Prompt:
List <NUM_TASKS> diverse tasks that <ASSISTANT ROLE> can assist <USER ROLE> cooperatively to achieve together. Be concise. Be creative.
Figure 3: Data Generation Prompts. In order to maintain a scalable approach our data parameters are generated using an LLM model to reduce human involvement in the generation process. The generation prompts for both AI Society dataset are summarized in this figure.
4.1 Role-Playing for AI Society
To create our AI Society dataset, we have developed a scalable approach that follows a series of steps. Firstly, we prompt the LLM agent to generate possible roles for the assistant and the user. We achieve this by providing the LLM agent with specific prompts designed to elicit these roles. Next, we ask the LLM agent to generate a range of possible tasks that can be solved through collaboration between the assistant and user roles generated previously. After generating a range of possible tasks as described
in the previous step, we then use the task specifier prompt passed to the LLM agent to make the task more specific. The prompts for assistant role generation, user role generation, and task generation are shown in Figure 5 (AI Society). For our AI society dataset, we generated 50 assistant roles, 50 user roles, and 10 tasks for each combination of roles yielding a total of 25,000 conversations. The generated assistant roles and user roles for AI Society as well as details about the generation of Code, Math and Science datasets can be found in the Appendix.
Challenges and Observations. In this section, we explore the four main challenges that we identified during our analysis of the generated datasets. Our observations shed light on some interesting aspects of cooperative AI and the difficulties that arise in its development.
- Role Flipping: One challenge we encountered was role flipping, where the assistant and user switch roles during the conversation. This issue typically arises when the assistant starts providing instructions or commands instead of following the user's prompts, which can lead to confusion and a reversal of roles. To avoid role flipping, it is crucial for the assistant not to ask questions, as this can also contribute to the problem.
- Assistant Repeats Instruction: Another challenge that we observed was the assistant simply repeating the user's instructions without any role flipping occurring.
- Flake Replies: We also observed instances where the assistant agent responds with a flake reply, often taking the form of "I will...". These messages do not contribute to the task at hand, as the assistant promises to take action but ultimately fails to follow through.
- Infinite Loop of Messages: An interesting challenge that we encountered was when the assistant and user engage in an infinite loop of meaningless conversation, such as repeatedly thanking each other or saying goodbye without progressing the task. Interestingly, in some cases, the assistant and user are aware that they are stuck in a loop, but are unable to break out of it.
The Appendix shows examples of each of the four challenges discussed above. Overall, our observations highlight the complexity of cooperative AI development and the need for continued exploration and innovation to overcome the challenges we face. By identifying these issues, we hope to contribute to the development of more effective and engaging cooperative AI systems.
Termination Conditions. The conversation between the assistant and user agents is designed to follow a specific format to ensure consistent and accurate data generation. To ensure that both the user and assistant adhere to their respective roles and responsibilities, certain conditions have been set in place to terminate the chat if necessary. These conditions are outlined below:
- User No Instruct: If the user does not instruct the assistant for 3 rounds, conversation is ended.
- Assistant Instruct: If the assistant provides an instruction to the user, it indicates a role reversal, and the conversation is terminated.
- End of Task Token: If the user believes that the task has been solved, they are expected to say <CAMEL_TASK_DONE> to signify the completion of the task. Once this message is received, the conversation is terminated.
- Assistant&User Token Limit: Given that gpt-3.5-turbo has a limitation on the number of tokens, the conversation is terminated if either the assistant or the user reach the token limit.
- Maximum Number of Messages: To keep the cost of generated chats in check, we have set a maximum limit of 40 messages. This limit guarantees a long enough conversation between the user and assistant while also ensuring that the data generated is not too costly to produce. The cost grows quadratically with the length of the conversation, making it essential to set a limit.
5 Evaluation
5.1 Agent Evaluation
In order to assess the performance of CAMEL (Cooperative Role-playing Communication), we conduct two types of evaluations: (1) Human evaluation, and (2) GPT4 evaluation. We randomly select 100 tasks from our AI Society dataset for evaluation and 100 tasks from our Code dataset. Then, we employ the GPT4 model to summarize the content of the CAMEL conversation-based
solution, presenting a consolidated final solution. Particularly, a GPT4 is used since it possesses a larger token limit which is suitable for summarization. Summarization also makes CAMEL agents' solution undetectable by its format, allowing for a more fair comparison. Subsequently, this solution is compared with a single-shot solution generated by the gpt-3.5-turbo model for the same task. Sample tasks are provided in the Appendix.
Human Evaluation. For this evaluation, we present both the CAMEL summarized agent solution and the gpt-3.5-turbo single-shot solution side-by-side to human participants. The identity behind each solution is not revealed. Participants are then asked to vote on whether one solution is superior to the other or if they are equally good. A total of 453 responses were collected during this evaluation. Note that, human evaluation is only done for AI Society, as assessing code is generally harder for humans (without running the code).
GPT4 Evaluation. We engage a GPT4 agent to evaluate the effectiveness of Model 1 (CAMEL Agent solution) versus Model 2 (gpt-3.5-turbo single-shot solution) for each task. More specifically, we prompt GPT4 to score and decide which solution of the two solutions is better.
Results. The summarized results of each evaluation are outlined in Table 1 which showcases that the CAMEL solution outperforms gpt-3.5-turbo single-shot solution in both the human evaluation and the GPT4 evaluation by a big margin. It is also worth noting that both human evaluation and GPT4 evaluation are highly aligned.
Table 1: Agent Evaluation Results: Results of the evaluations of the CAMEL agent against gpt-3.5-turbo using both human evaluators and GPT4 consistently show that utilizing a multiagent cooperative approach is more effective than gpt-3.5-turbo's single shot solution.
Dataset	Evaluation Type	Draw	gpt-3.5-turbo Wins	CAMEL Agents Win
AI Society	Human Evaluation	13.3%	10.4%	76.3%
GPT4 Evaluation	4.0%	23.0%	73.0%
Code	GPT4 Evaluation	0.0%	24.0%	76.0%
5.2 GPT4 for ChatBot Evaluation
In this section, we progressively fine-tune a LLaMA 7B model on our generated datasets. By progressively incorporating diverse datasets like AI society, code, math, and science, we expect fine-tuned model to demonstrate the ability to develop an increasingly sophisticated understanding of these domains.
We initially start by training on AI society dataset, which aims to let the model learn about human interactions and societal dynamics. As additional datasets were introduced, such as code, the model gained knowledge of programming logic and syntax, enabling it to generate coherent and executable code snippets. The inclusion of the math dataset further expanded the model's capabilities, allowing it to solve complex equations, reason about abstract concepts, and perform precise calculations. Finally, exposure to the science dataset broadened the model's understanding of scientific theories, empirical observations, and experimental methods. The emergence of model capabilities is measured by evaluating the quality of the model responses, before and after training on the new domain, on a set of questions of varying difficulties from each domain. More precisely, the model is tested on 20 AI Society related tasks, 20 coding tasks, 20 math tasks and 60 science tasks.
Those results are highlighted in Table 2 where we see that each time we add a dataset, the model performs better on the incorporated domain. Note that to measure the quality of the models' responses, we follow the evaluation from Section T, which involves prompting a GPT4 agent to score and decide which solution is better. It is worth noting that an improvement on other domains is also observed in some cases such as when we train on Code we improve on Science. This is because our Code dataset contains problems that solve tasks in particular domains which include scientific domain. Similarly, training on AI Society improves code as AI Society contains the role of a "programmer" and hence coding related conversations. Finally, note that the draws observed in LLaMA-7B vs AI Society in Math reflects equally bad solutions compared to the draws observed in AI Society + Code + Math vs AI Society + Code + Math + Science where the draws are equally good solutions. This progression from AI society to code to math to science highlights the potential of AI models to acquire a versatile
and adaptable knowledge base, paralleling the way humans gain expertise in diverse subjects. Sample tasks are provided in the Appendix.
Table 2: Emergence of Knowledge. By progressively fine-tuning LLaMA on datasets from different domains, we observe the emergence of knowledge as the model transitions from AI society to code, math, and science. This finding is indicated by the fact that Model 2 almost always performs better than Model 1, especially on the added dataset.
Dataset	Model 1	Model 2	Draw	Model 1	Model 2
AI Society	Code	Math	Science	AI Society	Code	Math	Science
AI Society	✓	0	6	14
Code	✓	0	0	20
Math	✓	9	5	6
Science	✓	0	13	47
AI Society	✓	✓	✓	4	8	8
Code	✓	✓	✓	1	9	10
Math	✓	✓	✓	5	8	7
Science	✓	✓	✓	1	19	40
AI Society	✓	✓	✓	✓	✓	5	6	9
Code	✓	✓	✓	✓	✓	1	9	10
Math	✓	✓	✓	✓	✓	1	3	16
Science	✓	✓	✓	✓	✓	3	8	49
AI Society	✓	✓	✓	✓	✓	✓	✓	3	1	16
Code	✓	✓	✓	✓	✓	✓	✓	1	8	11
Math	✓	✓	✓	✓	✓	✓	✓	10	5	5
Science	✓	✓	✓	✓	✓	✓	✓	9	2	49
AI Society	✓	✓	✓	✓	0	0	20
Code	✓	✓	✓	✓	0	0	20
Math	✓	✓	✓	✓	0	0	20
Science	✓	✓	✓	✓	0	0	60
5.3 HumanEval(+)
Table 3: HumanEval $^{(+)}$  for Various Models. We test our CAMEL model, which is a LLaMa-7B fine-tuned on all our datasets (AI Society, Code, Math, Science) on HumanEval and HumanEval $^{+}$  benchmarks, where we show competitive pass@k scores with LLaMa-7B and Vicuna-7B.
HumanEval	HumanEval+
pass@k [%]	k = 1	k = 100	k = 1	k = 100
gpt-3.5-turbo	69.4	94.0	61.7	89.8
LLaMA-7B	10.5	36.5	-	-
Vicuna-7B	11.0	42.9	9.9	34.7
CAMEL-7B	14.0	57.9	12.2	50.0
To evaluate the coding task-solving capabilities of our CAMEL model, specifically the LLaMA-7B fine-tuned on our comprehensive datasets, we rely on HumanEval [18] and  $\mathrm{HumanEval^{+}}$  [69]. The results, as depicted in table 3, clearly demonstrate the remarkable performance of CAMEL. It surpasses not only the LLaMA-7B model but also Vicuna-7B [21] by a big margin. These findings underscore the critical role played by the generated datasets in enhancing LLaMA's ability to tackle coding-related tasks.
In this paper, we explore the potential of autonomous cooperation among communicative agents and propose a novel cooperative agent framework named role-playing. Our approach enables communicative agents to collaborate autonomously toward completing tasks while requiring minimal human intervention, leading to better solutions are per our thorough evaluations. Through our analysis, we show that achieving autonomous cooperation is challenging due to issues like conversation deviation, role flipping, and termination conditions. Our framework offers a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems and provides strategies for addressing these challenges. Furthermore, our open-sourced library includes implementations of various agents, data generation pipelines, data analysis tools, and collected datasets, to support research on communicative agents and beyond. Our contributions offer valuable insights into the future of large language artificial intelligence models and cooperative AI systems.
This work was supported by SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence (SDAIA-KAUST AI).
[136] Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations, 2023.
[137] Deyao Zhu, Jun Chen, Kilichbek Haydarov, Xiaogian Shen, Wenxuan Zhang, and Mohamed Elhoseiny. Chatgpt asks, blip-2 answers: Automatic questioning towards enriched visual descriptions, 2023.