Paper: AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation (ICLR-2024)

Reviewer comment (Novelty):
These areas of improvement highlight a need for a more detailed, research-focused approach, and suggest that the work may find a more fitting audience at a venue like the System Demonstration track at ACL, where applied tools and frameworks are showcased and appreciated.

Author response:
AutoGen is motivated by the emerging need of multi-agent systems based on LLM, tools, and human input, and the challenges around building such systems. Given the focus on facilitating the development of multi-agent systems, we selected “infrastructure, software libraries, hardware, etc.” as the primary area of this submission.

Our contributions/novelty lies in: 
(1) the concepts/techniques we proposed for easily creating and orchestrating multi-agent systems, e.g., conversable agent, conversation programming, and conversation-centric way of building multi-agent systems. 
(2) the experiments we conducted to quantify the value of multi-agent settings.
(3) Additionally, we open-source the library to encourage additional research and development to explore and accelerate progress in this emerging area.

Reviewer comment (Experiments):
Is there any scenario in which the single agents outperform the multi-agent counterparts? For example, the Alfworld may be as easy as possible for a single agent to solve; maybe the assistant agent and executor agent can be the same one which generates the formatted [think + act] steps at the same time, which makes the multi-agent problem into a prompt template design problem.

Author response:
In our experiments, we observed that the multiagent settings are comparable to or better than the single-agent counterparts. The cases where multi-agent settings do not provide additional value are typically cases where the tasks are easier and can be solved reliably with a single agent.

For example, we see that on ALFWorld (Figure 4.c), a two-agent setup is comparable to a single-agent (with ReAct). However, a three-agent setup (with a grounding agent) outperforms both the React agent and the 2-agent setup. We note the performance of multi-agent settings will vary depending on the agent design. Hence, it is conceivable that a badly designed multi-agent solution could underperform a single agent.

Reviewer comment (Experiments):
Why do only two methods have the whole dataset results in A1 of Figure 4?

Author response:
Note that the whole MATH dataset includes 5000 questions. Running each method on the dataset costs between $500 and $800. Some methods (that use ChatGPT Plus) require substantial manual effort and are also restricted by message and token hourly rates - see footnote 4 on page 7.

Our experiments using the level-5 problems (the smaller dataset)  indicated a significant performance gap between the methods AutoGen, LangChain, and ReAct and that GPT-4 performs slightly better. As such, we prioritized comparing AutoGen to GPT-4 on the larger (5000 questions) dataset.

Reviewer comment (Experiments):
In the scenario of the A6 of Figure 3, where multi-agent players are playing chess, how do you make sure every agent strictly follows the chess rules?

Author response:
To ensure adherence to chess rules in our multi-agent chess game, we have introduced a 'Board Agent' that leverages the Python chess library for move validation.
This agent operates by extracting the UCI (Universal Chess Interface) move from each player's response at every turn. It then verifies the legality of this move using the chess library. Should a move be deemed illegal, the Board Agent issues an error message to the respective player, requesting a new legal move. For an illustrative example of the Board Agent's functionality, both with and without its intervention, please refer to Figure 16 in Appendix D, located on page 31 of our submission.

Reviewer comment (Novelty):
Not much takeaway in terms of scientific learning. I don’t find as a reader what lessons or results we can get from the paper. The main message is that we have a tool that can help developing multi-agent conversation. In my personal opinion, developing it based on a mature LLM agent is neither time-consuming nor scientifically challenging.

Author response:
## (Re Weakness 1) On scientific contribution and practical utility 
> Comment: Not much takeaway in terms of scientific learning. I don’t find as a reader what lessons or results we can get from the paper. The main message is that we have a tool that can help developing multi-agent conversation.

In addition to displaying engineering excellence, we believe AutoGen furthers science on multi-agents in the context of LLMs in many ways. 
- The concepts of conversable agents and conversation programming are novel and help systematically understand and make progress on the nascent topics of multi-agent for LLM applications. For example, as highlighted by reviewer SJWG, , “Another useful insight of this paper is to divide the main application workflow into small multi-agent conversations…”
- Empirical evaluation in Section 3 and in Appendix D shows AutoGen's advantages over existing frameworks and also produces some new discoveries. For example, through the experiments introduced in A4, we discover that “a multi-agent design is helpful in boosting performance in coding tasks that need safeguards”. We provide further elaboration on the takeaways from this and other applications in Appendix D (Application Details). 
- The proposed concepts and abstractions combined with our OSS implementation enable other researchers to explore multi-agents and further science. E.g, as mentioned in A4 of Section 3 in the main paper, through AutoGen OptiGuide’s implementation was reduced from over 430 lines to just 100 lines. This is a clear testament to the framework's ability to facilitate more efficient scientific experiments.

> Comment: The main message is that we have a tool that can help developing multi-agent conversation. In my personal opinion, developing it based on a mature LLM agent is neither time-consuming nor scientifically challenging.

Although the reviewer does not personally perceive the framework’s value in supporting the development of multi-agent systems, the framework is well-received by the open-source community in general:  as of 20th November 2023, our OSS implementation 
- has been downloaded over 130,000 times within less than 2 months (we omit package name to remain anonymous.); 
- has been used to build many applications from the open-source community (used in more than 200 open-source projects). For example, GPT academic, an LLM-based service for assisting various academic-related activities such as scientific writing, research paper reading, etc,; and 
- has been awarded one of the top 100 open-source projects 2022-2023 by the International Open Benchmark Council under the following evaluation criteria: 
    - The key milestones that lay the foundation for open source movement or development; 
   - The original or pioneering open source works; 
   - The open source works that play a significant role in promoting the development of software and hardware; 
   - The open source works are widely used or cited by industry or academia.

These are all evidence of our framework's practical utility for AI applications and experiments. Although we totally understand and value the reviewer's personal preference, we hope the reviewer can take the general impact and utility this work could bring into consideration.

Reviewer comment (Evaluation):
The results are not convincing. I found the comparison of results is not rigorously evaluated and not convincing. For example, the paper shows that by repeating the question again to LLM may potentially get better results. But the results are not convincing due to few sample they use nor making a lot of sense.

Author response:
## (Re weakness 2) On evaluation and results

Thank you for your feedback. We understand the importance of rigorous evaluation in assessing the effectiveness of our framework.
To ensure a comprehensive understanding, our study encompasses a blend of qualitative and quantitative evaluations across a variety of applications. These evaluations are designed to provide a holistic view of the framework's performance and utility.

> I found the comparison of results is not rigorously evaluated and not convincing.

Since different evaluation types may yield varying insights, we would like to ask the reviewer which application and evaluation this comment is referring to here. 

To further clarify, as mentioned in the paper, the evaluation of all four applications (A1-A4) with quantitative evaluation results followed established practices from existing literature. E.g.,

- For A1, our evaluation used both the full MATH dataset [1], which includes 5000 problem instances, and a sub-sampled set including 120 problems. 
- For A2, our evaluation was performed on all the problem instances from the Natural Questions benchmark [2], which is a benchmark for Question Answering Research and includes 6775 question instances. We followed the evaluation procedure established in [3]. 
- For A3, our evaluation is performed on the ALFWorld dataset [4], following the evaluation procedure used in the ReAct [5] paper.
- For A4, our evaluation is performed following the evaluation procedure established in the original OptiGuide paper [6]. 
 
In addition to quantitative evaluation results, we also include a rich set of qualitative studies for A1-A4 (included in Appendix D Application Details).

A5 and A6 are innovative applications for demonstration purposes (as there is no relevant benchmark for these two tasks) and thus are primarily supported by qualitative.

[1] Hendrycks, Dan, et al. "Measuring mathematical problem solving with the math dataset." NeurIPS 2021.

[2] Kwiatkowski, Tom, et al. "Natural questions: a benchmark for question answering research." ACL 2019.

[3] Adlakha, Vaibhav, et al. "Evaluating correctness and faithfulness of instruction-following models for question answering." arXiv preprint arXiv:2307.16877 (2023).

[4] Shridhar, Mohit, et al. "Alfworld: Aligning text and embodied environments for interactive learning." ICLR 2021

[5] Yao, Shunyu, et al. "React: Synergizing reasoning and acting in language models." ICLR 2023.

[6] Li, Beibin, et al. "Large language models for supply chain optimization." arXiv preprint arXiv:2307.03875 (2023).

Reviewer comment (Evaluation):
Too much brag about their system but little evidence is shown to support it. For example, in the introduction, they aim to design a “capable, reusable, customizable, and effective” system. I didn’t see support towards it.

Author response:
## (Re weakness 3) On the support of claims 

We presented numerous applications developed with the proposed framework and provided very comprehensive evaluations of most of the applications in both Section 2 and Appendix D (from page 19 to page 43). Those are all evidence supporting the advantages of the proposed framework.  Regarding the example mentioned by the review on “capable, reusable, customizable, and effective”, we do provide support for this claim:

- Capable: In the second paragraph of Section 2.1, we introduced 'Agent capabilities powered by LLMs, humans, and tools', elaborating the supported agent capabilities.
- Customizable: Section 2.1 discusses 'Agent customization and cooperation'. In the applications presented in Section 3, we developed agents customized from the built-in agent. This includes the Retrieval-augmented User Proxy agent in Application 2 and the ALF World Executor agent in Application 3.
- Reusable: The built-in agent, Assistant, is reused in Applications 1 (including two scenarios), 2 (including two scenarios), and 4.
- Effective: In Applications 1-4, we demonstrate how AutoGen enables the development of multi-agent systems that are effective in solving tasks such as math problem-solving, retrieval augmented question answering, decision-making, and coding with safety constraints.

Reviewer comment (Experiments):
Can you specify why the interactive retrival can lead to better results? I am curious about what the result will look like if I try different variations of questions? As with interactive, I can think of the improvement is due to more trials other than better problem understanding.

Author response:
## Re question on interactive retrieval 
We thank the reviewer for the insightful question! We agree with the reviewer’s point that the improved performance with interactive retrieval ultimately stems from the additional trials requesting context. However, achieving this wisely and robustly is highly non-trivial and does require a proper understanding of the problem status.
 
To better understand the challenge here, let’s consider a naive approach: this approach performs multiple trials of retrieval right at the beginning when the question is asked. This gives us more trials but does not require problem understanding at all. However, this naive approach has one obvious limitation: it is hard to decide how many up-front trials one should try to include. Including context from too many trials can lead to the conversation exceeding the context length and may incur unnecessarily high inference cost. When the multiple up-front trials are conducted separately, one also needs a way to select the answer from it, which is also non-trivial. Conversely, too few trials may not provide adequate context. This latter case with a single trial is investigated in our ablation study in A2, with results illustrated in Figure 4(b) and Figure 10 in Appendix D. Regarding the former case, since it relies on the number of trials as a hyperparameter and an answer selection method, it is easy to construct scenarios that have undesirable results, e.g., failure due to context overflow or incurring a very high cost.  Interactive retrieval, on the other hand, can be considered an online approach that naturally remedies this tension: It does not rely on a pre-specified trial number but makes the “UPDATE CONTEXT” request when necessary (based on LLM’s understanding of whether the question can or cannot be answered without further context) as the conversation proceed until the question is considered answered.

We would like to hear if your comments are addressed, and the question is answered, and we would be happy to provide further elaboration if needed.

Reviewer comment (Experiments):
The paper does not address the issue of context length, which may become too long as the number of conversation turns increases. This could affect the performance and efficiency of the LLMs and the agents.

Author response:
Thanks to the extensibility of our framework, we are able to work out two solutions to address this potential issue. 
We are able to implement a `CompressibleAgent` under the proposed AutoGen framework that could compress long context when needed. Please find [a notebook example demonstrating how to use CompressibleAgent in this anonymous link](https://anonymous.4open.science/r/autogen-iclr2024/notebook/agentchat_compression.ipynb).

As mentioned in the expanded discussion section in Appendix B, we acknowledge the existence of scenarios/cases where other libraries/packages could help. The context length problem could be one such case. AutoGen has been integrated with MemGPT, which is a recent work that could teach LLMs to manage memory for unbounded context. Please find a notebook example [demonstrating how AutoGen can be used together with MemGPT](https://anonymous.4open.science/r/autogen-iclr2024/notebook/memgpt_coder_autogen.ipynb).

Reviewer comment (Experiments):
It would be better to consider the cost issue, which is important for practical applications. The experiments are conducted on GPT-4 and GPT-3.5, which are expensive and not widely accessible. How would the framework perform on open-source LLMs with lower capacity?

Author response:
In general, open-source LLMs with lower capacity than GPT-3.5/4 would lead to lower performance if not meticulously utilized. Fortunately, our multi-agent framework provides the flexibility to use those open-source models in strategic ways.  In one of our follow-up work (we do not disclose the title of work due to anonymity concerns), we built a two-agent system using the built-in AssistantAgent and UserProxyAgent in AutoGen and evaluated the system on tasks [1] that require coding and external API calls. In this application, if we directly replace GPT-3.5/4 with LLAMA-2-13B-chat in the AssistantAgent, the system’s performance indeed drops by a large margin. However, we also show that by including multiple AssistantAgent backed by GPT models or LLAMA-2-13B-chat into a multi-agent system, we can actually reduce the dollar cost of the system while improving the success rate. 

----

| AssistantAgent     | Success Rate (averaged over 300 queries) | Average Cost 
|---------|----------|----------|
| LLAMA-2-13B-chat  | 13.33% | 0.00
| GPT-3.5-turbo | 34.11% | $0.41
| GPT-4   | 77.22% | $13.93
| GPT-4 +  GPT-3.5-turbo +  LLAMA-2-13B-chat  | 83.22% |  $8.98

[1] ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

Reviewer comment (Novelty):
In my opinion, AutoGen appears to be an extension of CAMEL, both supporting agent role-playing and agent conversations. The authors discuss the related work of CAMEL in the appendix, and highlight two distinct advantages of AutoGen: 1) its capability for tool usage and 2) dynamic conversation. However, it deserves to give an in-depth discussion about CAMEL and AutoGen about their differences in the introduction section. Actually, I do not think the tool-usage is a big challenge if using the GPT4. It is expected to discuss the challenge from static conversation (of CAMEL) to dynamic conversation (of AutoGen).

Author response:
We thank the reviewer for the constructive suggestion and this insightful question.

We would like to first clarify that CAMEL has a different focus and positioning from AutoGen: CAMEL is primarily positioned as a framework for studying the cooperative behaviors of agents of different roles, not an infrastructure to support the development of LLM applications as the case in AutoGen. This difference can be better understood from the two frameworks’ different behaviors in solving a task. E.g., under the task “*Design a custom game using pygame*” [demonstrated in CAMEL’s official GitHub repo](https://colab.research.google.com/drive/1AzP33O8rnMW__7ocWJhVBXjKziJXPtim?usp=sharing#scrollTo=-IhYiAWDme66), we compared AutoGen vs. CAMEL and summarized in results in this [anonymized document](https://anonymous.4open.science/r/autogen-iclr2024/notebook/alternatives/README.md).  From the comparison, we can see that CAMEL is primarily simulating a conversation between an AI agent with the role “*Computer Programmer*", and an agent with the role “*gamer*”, but is not actually creating a meaningful game; while AutoGen is able to actually create games with pygame and save the created game into a file such that it can be directly executed and played. 

One more fundamental distinction, which poses profound technical challenges, is AutoGen’s general support for multi-agent systems with an agent number N > 2. CAMEL’s inception prompting based role-playing framework currently primarily supports systems with two AI agents with potentially a critic in the loop. There is no general support for systems with more than 2 agents or with other conversation patterns.  

Note that moving from 2 to N (N > 2) in an effective way that could support LLM applications is highly non-trivial and is technically challenging: When N = 2, the communication between the agents is straightforward. Supporting N > 2, in general, requires careful abstraction and implementation so that the framework can (1) possess the flexibility to meet various application needs (there is hardly a one-fit-all pattern), and (2) support conversation patterns that can make meaningful progress in task completion. AutoGen is so far the only framework that realizes both objectives decently well. 

We will clarify the major differences and challenges in the main paper accordingly in a future draft.

Reviewer comment (Experiments):
Please refer to Weakness (2). How would the framework perform on open-source LLMs with lower capacity?

Author response:
In general, open-source LLMs with lower capacity than GPT-3.5/4 would lead to lower performance if not meticulously utilized. Fortunately, our multi-agent framework provides the flexibility to use those open-source models in strategic ways.  In one of our follow-up work (we do not disclose the title of work due to anonymity concerns), we built a two-agent system using the built-in AssistantAgent and UserProxyAgent in AutoGen and evaluated the system on tasks [1] that require coding and external API calls. In this application, if we directly replace GPT-3.5/4 with LLAMA-2-13B-chat in the AssistantAgent, the system’s performance indeed drops by a large margin. However, we also show that by including multiple AssistantAgent backed by GPT models or LLAMA-2-13B-chat into a multi-agent system, we can actually reduce the dollar cost of the system while improving the success rate. 

----

| AssistantAgent     | Success Rate (averaged over 300 queries) | Average Cost 
|---------|----------|----------|
| LLAMA-2-13B-chat  | 13.33% | 0.00
| GPT-3.5-turbo | 34.11% | $0.41
| GPT-4   | 77.22% | $13.93
| GPT-4 +  GPT-3.5-turbo +  LLAMA-2-13B-chat  | 83.22% |  $8.98

[1] ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

Reviewer comment (Novelty):
Please refer to Weakness (3). What is the technique challenge for introducing dynamic conversation compared with AutoGen?

Author response:
We thank the reviewer for the constructive suggestion and this insightful question.

We would like to first clarify that CAMEL has a different focus and positioning from AutoGen: CAMEL is primarily positioned as a framework for studying the cooperative behaviors of agents of different roles, not an infrastructure to support the development of LLM applications as the case in AutoGen. This difference can be better understood from the two frameworks’ different behaviors in solving a task. E.g., under the task “*Design a custom game using pygame*” [demonstrated in CAMEL’s official GitHub repo](https://colab.research.google.com/drive/1AzP33O8rnMW__7ocWJhVBXjKziJXPtim?usp=sharing#scrollTo=-IhYiAWDme66), we compared AutoGen vs. CAMEL and summarized in results in this [anonymized document](https://anonymous.4open.science/r/autogen-iclr2024/notebook/alternatives/README.md).  From the comparison, we can see that CAMEL is primarily simulating a conversation between an AI agent with the role “*Computer Programmer*", and an agent with the role “*gamer*”, but is not actually creating a meaningful game; while AutoGen is able to actually create games with pygame and save the created game into a file such that it can be directly executed and played. 

One more fundamental distinction, which poses profound technical challenges, is AutoGen’s general support for multi-agent systems with an agent number N > 2. CAMEL’s inception prompting based role-playing framework currently primarily supports systems with two AI agents with potentially a critic in the loop. There is no general support for systems with more than 2 agents or with other conversation patterns.  

Note that moving from 2 to N (N > 2) in an effective way that could support LLM applications is highly non-trivial and is technically challenging: When N = 2, the communication between the agents is straightforward. Supporting N > 2, in general, requires careful abstraction and implementation so that the framework can (1) possess the flexibility to meet various application needs (there is hardly a one-fit-all pattern), and (2) support conversation patterns that can make meaningful progress in task completion. AutoGen is so far the only framework that realizes both objectives decently well. 

We will clarify the major differences and challenges in the main paper accordingly in a future draft.

Reviewer comment (Evaluation):
In Figure 4(c), the performance of ReAct on Best of 3 is better than AutoGen (2 agents). I am curious as to what do you think would the reason for that?

Author response:
We thank the reviewer for the comments and the question. The difference between the performance of ReAct and AutoGen (2 agents) is actually quite marginal. The marginal difference could be caused by randomness from LLM output generation.