Paper: AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors (ICLR-2024)

Reviewer comment (Experiments):
The experimentation on the AgentVerse framework is promising but quantitative results are seriously lacking in this paper. The case studies provide interesting insight into how the AgentVerse framework deals with problems but is not yet significant enough. For instance, Table 1 and Table 2 provide the only quantitative results in the paper, including the appendix.

Author response:
We appreciate your feedback regarding the quantitative experiments of our research. Our paper includes five benchmark datasets, a number we consider substantial for evaluating multi-agent systems. These benchmarks were chosen to be diverse and evaluate different aspects of our framework. 

Furthermore, the emergent behaviors and communication patterns we have shown in the case studies are non-trivial. **These elements, though less tangible than numerical benchmark results, are central to advancing the understanding and capabilities of multi-agent systems**. The case studies in our work are designed to mirror real-world challenges more closely than conventional benchmark datasets and, therefore, **provide valuable insights into the practical applicability of AgentVerse**. We kindly request the consideration of these qualitative aspects as equally, if not more, important than quantitative experiments. The nuanced understanding they provide into agent interaction dynamics is vital for the advancement of multi-agent systems.

Reviewer comment (Experiments):
There is a question about the necessity of framing this as a multi-agent problem. Can sufficient prompting of the agent provide enough context that multiple agents are not necessary (beyond the solo and CoT baselines)?

Author response:
We believe that while sophisticated prompting can enhance single-agent capabilities, the multi-agent framework offers distinct advantages:

1. **Efficiency of Multi-Agent System:** An individual agent, even with sophisticated prompting, may face limitations in efficiency when tackling complex tasks. Multi-agent systems excel in this aspect, especially in real-world scenarios involving intricate multi-tool utilization (Section 3.3) or collaborative goals in gaming environments (Section 4). By coordinating and collaborating, multiple agents can operate more efficiently than a single agent in these contexts since they have a clear division of labor, and less distracting information in their memory.
2. **Complementarity of Single and Multi-Agent Approaches:** Advanced prompting techniques can indeed create more capable agents. This does not contradict with multi-agent system approach. Multi-agent system can also benefit from more capable agents. This synergy allows for leveraging the strengths of single-agent efficiency and multi-agent collaboration. 
3. **Exploring Multi-Agent Dynamics:** Our research delves into some of the intricate properties of multi-agent communication, shedding light on how heterogeneous agents can collaborate effectively. This exploration reveals the potential of multi-agent systems, a domain where single-agent frameworks fall short. The dynamics within multi-agent interactions are not just an extension of single-agent capabilities but constitute a distinct and valuable area of research. Our work represents a significant step in demonstrating the potential and worth of multi-agent studies based on LLMs.

Reviewer comment (Evaluation):
Do the authors have any additional quantitative results, with included uncertainty, to support their claims beyond Table 1 and 2?

Author response:
We additionally run the experiments of GPT-3.5-turbo for 3 times, and report the averaged performance and the standard deviation on each dataset:
|                        | CoT             | Solo            | Group           |
| ---------------------- | --------------- | --------------- | --------------- |
| Conversation           | $83.7_{\pm1.0}$ | $84.0_{\pm1.7}$ | $86.8_{\pm1.2}$ |
| Creative Writing       | $79.5_{\pm0.2}$ | $92.4_{\pm0.2}$ | $90.2_{\pm0.4}$ |
| Mathematical Reasoning | $81.2_{\pm0.6}$ | $83.2_{\pm1.5}$ | $82.8_{\pm0}$   |
| Coding                 | $72.4_{\pm0.3}$ | $73.4_{\pm1.9}$ | $74.8_{\pm0.7}$ |

Reviewer comment (Novelty):
The motivation of this paper seems unclear. It does not clearly show the necessity of AGENTVERSE compared to existing Agent models.

Author response:
**Q1: Motivation of our paper.**

A1: The evolution of AI research is progressively shifting from the development of individual agents to fostering their ability to collaborate effectively. Our research is driven by a vision to **explore and enhance this collaborative potential among AI agents**.

The core of our motivation lies in addressing the complexity of **enabling multiple agents to communicate, coordinate, and collaborate efficiently**. The current LLMs shaped by techniques such as SFT and RLHF are tailored to function as 'assistants,' primarily responding to human queries. However, there is a distinct gap in their ability to act as 'agents' that autonomously interact with each other and navigate diverse environments, whether virtual or physical.

To bridge this gap, it's crucial to design and implement a nuanced communication mechanism that facilitates effective interaction among LLM agents. In response to this need, we have developed AgentVerse. Our framework not only demonstrates effectiveness in various benchmark datasets but also showcases its applicability in a range of **real-world scenarios**. Moreover, our observations of **emergent behaviors** within AgentVerse underscore the untapped potential of LLM agents in multi-agent systems, paving the way for groundbreaking advancements in AI collaboration.

Reviewer comment (Experiments):
There is no comparison between the framework proposed in this paper and the existing LLM-based agent framework should be listed, such as whether it generates agent dynamically, or the number of agents is limited, etc. Multiple dimensions of indicators should be investigated in detail to explain the advantages of your proposed framework.

Author response:
**Q2: Comparison with other frameworks**

A2: We have prepared a comparative table detailing how AgentVerse and other frameworks differ.

Comparison Table: 

- Multiple Agent
- Expert Recruitment
- Autonomous Interaction Among Agents
    - Collaborative Decision-Making
    - Distributed Action Execution
    - Interact with external world

|  |  |  | autonomous interaction | autonomous interaction | autonomous interaction |
| --- | --- | --- | --- | --- | --- |
|  | Single/Multiple Agent | Expert Recruitment | Collaborative Decision-Making (Multi-agent Communication) | Distributed Action Execution | Interact with external world  |
| Camel | Multiple | x | ✓ | x | x |
| AutoGPT | Single | x | x | x | ✓ (human, environment) |
| XAgent | Multiple | x | x | x | ✓ (human, environment) |
| METAGPT | Multiple | x | ✓ | x | ✓ (human) |
| AutoGen | Multiple | x | ✓ | ✓ | ✓ (human, environment) |
| AutoAgents | Multiple | ✓ | ✓ | x | ✓ (human) |
| AgentVerse | Multiple | ✓ | ✓ | ✓ | ✓ (human, environment) |

Reviewer comment (Novelty):
In related work section, it does not summarize and classify the related work of LLM-based automotive agents since these two years. LLMs have been widely used as core controllers for automotive agents that can comply with specific objectives, such as Auto-GPT and MetaGPT, etc.

Author response:
**Q3: Missing related work**

A3: Thank you for pointing out the omission of recent developments in LLM-based automotive agents in our related work section. We will add a paragraph discussing the related work on autonomous agents with notable examples like Auto-GPT, MetaGPT, and others that have emerged as key players in the field.

Reviewer comment (Reproducibility):
While the paper mentions the release of the AGENTVERSE codebase, it does not provide more specific information about how and where readers can access it. It would be beneficial to provide clear repository details or a direct link.

Author response:
**Q4: AgentVerse codebase**

A4: We have uploaded an anonymized version of our codebase, you can refer to [https://anonymous.4open.science/r/AgentVerse-EFC23/](https://anonymous.4open.science/r/AgentVerse-EFC23/README.md). In this code, we further extend AgentVerse to multi-agent simulation. You can refer to the task solving part of our code.

Reviewer comment (Experiments):
When the number of tasks is 10, AGENTVERSE is superior to single ReAct agent in the experiment of tool utilization capabilities, how about the performance when the number of tasks increases or the tasks are heterogeneous?

Author response:
**Q6: Tool utilization performance with more and heterogeneous tasks**

A6: We acknowledge the existence of datasets like ToolBench (https://arxiv.org/abs/2307.16789) for complex multi-tool tasks. However, their lack of a robust evaluation method, primarily relying on LLMs, limits their reliability. Our preliminary experiments with ToolBench also faced technical challenges: most of the APIs they provide have failed. The problem is still not perfectly solved even after contacting the authors. Furthermore, other available tool datasets such as APIBench (https://arxiv.org/abs/2305.15334), either only involve single tool or are overly simplistic for our multi-agent tool utilization context.

We invite you to review Appendix B, which compares the performances of a single ReAct agent and AgentVerse. This comparison clearly demonstrates AgentVerse's superior performance over the single-agent approach. We did not do any cherry picking. Based on these findings, we are confident in AgentVerse’s ability to outperform single-agent setups in complex tool utilization scenarios.

Reviewer comment (Evaluation):
What does the data with respect to coding capabilities in Table 2 mean?

Author response:
**Q7: Table 2 meaning**

A7: It shows the Pass@1 of GPT-3.5-turbo and GPT-4 on Humaneval dataset. For both models, multi-agent setting consistently outperforms solo and CoT settings.

Reviewer comment (Evaluation):
Although not a significant weakness, performance drivers in quantitative experiments could use additional elaboration, vs. the terse tabular summary.

Author response:
A3: Our framework's performance is driven by several key factors: 

1. **Role Assignment**: Assigning expert identities to agents enhances accuracy, as supported by related work (e.g., https://arxiv.org/abs/2305.14930) and evidenced in our consulting case (Figure 2). We've also added an experiment highlighting this, showing improved results in GPT-3.5-Turbo CoT tasks with role assignment. We run each setting for 3 runs and report the averaged result and the standard deviation:

|  | w/o role | w/ role |
| --- | --- | --- |
| Math Reasoning | $72.4_{\pm 0.3}$ | $73.7_{\pm 0.5}$ |
| Coding | $81.2_{\pm 0.6}$ | $82.4_{\pm 0.7}$ |

2. **Environmental Feedback**: Utilizing environmental feedback, like coverage tests in Commongen and unit tests in Humaneval, significantly boosts performance. This is explained in Appendix A, illustrating the impact on datasets where such feedback is applicable.
3. **LLM Communication Capabilities**: As noted in Section 3.1, the ability of LLMs like GPT-3.5-turbo and GPT-4 to handle contradictions during decision-making stages is crucial. GPT-4's superior handling of contradictions indicates the importance of robust communication capabilities for maximizing the benefits of a multi-agent system.

Reviewer comment (Novelty):
Complimentary and similar approaches are emerging rapidly, and some quantitatively outperform this method. I think this paper presents a distinct approach, and the results might be enhanced by contemplating the integration of other methods. As just one example, a very recent preprint using different methods (https://arxiv.org/abs/2310.04406) demonstrates similar or better performance improvements on the code dataset. A discussion of how to merge other ideas into this approach might be helpful. *I have no affiliation with the authors of the referenced work, and have no knowledge of that work beyond having recently run across it on arXiv.

Author response:
A1: We acknowledge the value of incorporating recent advancements into AgentVerse. We think that many of these studies, focused on enhancing individual agents’ reasoning, complement our work. Importantly, they can be seamlessly integrated into AgentVerse. For example, the method you mentioned (https://arxiv.org/abs/2310.04406) is a tree-of-thought-like method that significantly enhances single-agent reasoning. The improved reasoning abilities of individual agents, when combined with our framework's collaborative dynamics, may lead to a more robust and effective problem-solving system. We view such advancements as complementary, enabling potential synergistic improvements to our system, rather than as alternative approaches.

Reviewer comment (Presentation):
Can you comment on the extensibility of your approach, for instance in the potential future work section?

Author response:
A2: As the capability of single agent evolves, we believe that exploring how to enable these agents to move beyond working in isolation will become an important research topic. AgentVerse, with its modular design, facilitates experimentation with various agent teams and tasks. In our paper's Appendix E, we discuss potential extensions, and here we would like to discuss how AgentVerse can be extended to this research. 

1. **More Capable Agents and More Challenging Scenarios.** We can easily incorporate more capable agents from recent research, and apply them to challenging scenarios, such as more sophisticated embodied agent scenarios. Our experiments in the Minecraft environment, a sandbox game, demonstrate its applicability for agents in virtual world. This success suggests that extending AgentVerse to other embodied simulations, or even real-world physical scenarios, is not only feasible but a promising avenue for future exploration. Minecraft serves as a proof of concept, showing how AgentVerse's principles can be adapted to environments where agents interact in more physically grounded settings.
2. **Enhanced Communication.** There are several topics that can be further explored in multi-agent communication. For example, the communication structure. Though we only design two straightforward and intuitive communication structures, i.e., horizontal and vertical, further exploration can be made based on our code and experiments by customizing and replacing the module. Also, the research on communication language, protocol, etc. can all be considered within AgentVerse.
3. **Leverage Emergent Behaviors and Mitigate Safety Issues.** The emergent behaviors observed in our tool utilization and Minecraft scenarios provide valuable insights. How can we harness the beneficial emergent behaviors will be a research topic for multi-agent research. Also, the harmful behaviors we observed also indicates the necessity to study the agent safety within multi-agent context. It may be a new scenario that needs to be considered in alignment.

Reviewer comment (Evaluation):
Evaluation Module: It seems that the evaluation module is purely an LLM-based module, leveraging a multi-dimension scoring system like self-refine (https://arxiv.org/abs/2303.17651). I have some concerns about the module: All the feedbacks are generated by LLMs, instead of interactions with environments (e.g., runtime error, etc.). The evaluation scores generated by the module are also unreliable as well, especially when the module is following a zero-shot format. The quality of the evaluation scores will impact the framework largely.

Author response:
**Q1: Concerns about the evaluation module relying solely on LLMs**

A1: We agree with you that relying solely on LLMs is unreliable. We've incorporated external feedback as a key component in our evaluation module across various scenarios, enhancing reliability and effectiveness. Specifically:

- **Minecraft Environment:** Evaluator receives in-game environmental factors and player states, ensuring real-time, context-specific assessment.
- **Humaneval Benchmark:** Evaluator leverages the results from model-generated unit tests to validate outcomes, as detailed in Appendix A.
- **Commongen Benchmark:** Evaluator uses coverage test results to gauge the effectiveness of generated responses, also elaborated in Appendix A.
- **Tool Usage Scenarios**: Evaluator integrates outcomes from different agents to deliver accurate and comprehensive feedback or reports to users.

Reviewer comment (Experiments):
The framework lacks some details or implementation recommendations (How does the number of agents influence the performance? How do the tools influence the final problem-solving? How many turns on average are needed to solve the problems in the tasks displayed in the paper?).

Author response:
**Q4: Lack of detail in experiments**

A4: Thanks for your suggestion. We have carried out some additional experiments to enrich our presentation. Here we list how the number of agents affects the performance on the quantitative experiments for GPT-3.5-turbo. We run each setting on Humaneval (coding) and MGSM (math) for 3 runs, and report the averaged performance.

(A brief recap, Solo means 1 role assigner agent + 1 decision-making agent + 1 evaluation agent, Group is the same except that there are multiple decision-making agent)

|                        | CoT              | Solo             | Group-2          | Group-3          | Group-4          |
| ---------------------- | ---------------- | ---------------- | ---------------- | ---------------- | ---------------- |
| Mathematical Reasoning | $81.2_{\pm 0.6}$ | $83.2_{\pm 1.5}$ | $82.3_{\pm 0.2}$ | $82.8_{\pm 0}$   | $81.6_{\pm 1.3}$ |
| Programming            | $72.4_{\pm 0.3}$ | $73.4_{\pm 1.9}$ | $74.8_{\pm 0.7}$ | $74.6_{\pm 1.1}$ | $74.8_{\pm 0.6}$ |
| Average Performance    | $76.8_{\pm 0.2}$           | $78.3_{\pm 1.0}$           | $78.5_{\pm 0.5}$           | $78.7_{\pm 0.5}$           | $78.2_{\pm 0.9}$           |

where `Group-x` indicates `x` decision making agent. Generally, using AgentVerse framework with one to three decision making agents give satisfying results. The averaged performance gets highest when there are 3 decision making agents. While these datasets primarily test specific agent abilities, not fully utilizing the diversity of a multi-agent setup, we still observe an upward trend in average performance with an increase in agents. The diminishing returns upon further scaling can be attributed to communication inefficiencies, as discussed in Section 3.1.

At the meantime, we highlight that AgentVerse’s true potential is best observed in more complex challenges. Our case studies, tool utilization experiments, and Minecraft game-playing scenarios are prime examples where the multi-agent framework's capabilities are more pronounced and beneficial.

Reviewer comment (Evaluation):
Some basic explanations/analyses are missing in the paper as well. (1) In Table 1, why do agents based on GPT-3.5 have better performance in a solo setting than group? Why is the situation inverse for GPT-4-based agents? How do the agents' abilities matter in the framework? (2) It seems that the Group setting performance usually has a small gain compared with CoT and Solo. I am just wondering whether Group can be easily surpassed by the other techniques, like self-refine, etc.

Author response:
Q5: Missing analyses and explanations

A5: In Section 3.1, we discuss the performance variance between Group and Solo settings for GPT-3.5-turbo and GPT-4. We also examine the conflict resolution capabilities of LLMs in multi-agent systems. Further detailed explanations on these phenomena will be added to enhance understanding.

Reviewer comment (Evaluation):
The entire framework is too long and complicated. The prompts include module prompts, interaction history, and agent prompts, which are a large number of tokens to call the OpenAI API. The cost is a huge concern here to use the framework for evaluation. Furthermore, the framework is also much too inefficiency compared with the performance gain led by the multi-agent interactions.

Author response:
**Q3: Framework complexity and cost**

A3: We acknowledge the increased complexity and cost of our multi-agent system compared to single-agent systems. This complexity is inherent in our research question: "As single-agent capabilities continue to evolve, how can we enable these agents to collaborate beyond solitary operation, especially in complex real-world challenges?" Drawing parallels with human society, where collaboration leads to greater efficiency and the ability to tackle more complex tasks, our work delves into the potential and challenges of multi-agent collaborations. Indeed, we have observed inefficiencies in agent collaboration, especially agent communication inefficiency. However, as one of the pioneering studies in autonomous multi-agent systems, we explore the potential of collaborative agents, and we demonstrate it with several promising cases achieved with our framework. We acknowledge that communication challenges exist, but we argue that by highlighting these challenges, we can pave the way for future research to develop more efficient multi-agent system with advanced communication capabilities. **In light of this, we think the complexity and cost of the framework is acceptable.**

If you also think this direction is worth exploring and interesting (as suggested by your strength comments), you might agree that identifying the challenging issues and providing the right direction for further research is as important as proposing incremental methods to optimize an autonomous multi-agent system. This approach allows us to highlight the great potential, as we as some limitations of agents in collaborative settings.

Reviewer comment (Novelty):
Compared with the single agent works, what is the main motivation of using multiple agents to solve the problem? Some self-refine/self-ensemble works seem to have better performance compared with multi-agents, like (self-consistency (https://arxiv.org/abs/2203.11171), reflexion (https://arxiv.org/abs/2303.11366), self-debug (https://arxiv.org/abs/2304.05128)).

Author response:
**Q2: Motivation for using multiple agents**

A2: As single-agent capabilities continue to evolve, a key area of research is enabling these agents to collaborate beyond solitary operation, particularly in complex real-world challenges. Our work focuses on this collaborative potential among AI agents.

In single-agent research, the focus often lies on benchmark performance (similar to our Sections 3.1 and 3.2). However, the essence of multi-agent study extends beyond these metrics, delving into the autonomous behaviors and communication patterns within agent dialogues—crucial yet often overlooked aspects of agent interaction.

Therefore, we do not stop at reporting higher metrics than single agent on benchmarks, we further explore the emergent behaviors. While less tangible than numerical benchmark results, **analyzing and leveraging these emergent phenomena is central to advancing multi-agent systems**. Current LLM agents are typically viewed as 'assistants' (e.g., AutoGPT, OpenAI Assistant), performing tasks as directed by humans on their own, their potential as autonomous 'agents' interacting and collaborating with one another remains largely untapped. 

Our research, particularly in Sections 3.3 (Tool Using) and 4 (Minecraft Game Playing), demonstrates the capabilities of LLM agents in autonomous coordination and collaborative task accomplishment. These scenarios highlight the need to consider LLM agents not just as assistants but as interactive participants in a multi-agent environment.

Thus, while multi-agent systems may not always outperform single agents in certain benchmarks, their unique aspects, like emergent behaviors and communication patterns, are valuable research areas in their own right.