CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation

## Abstract

Abstract In this work, we address the cooperation problem among large language model (LLM) based embodied agents, where agents must cooperate to achieve a common goal.
Previous methods often execute actions extemporaneously and incoherently, without long-term strategic and cooperative planning, leading to redundant steps, failures, and even serious repercussions in complex tasks like search-and-rescue missions where discussion and cooperative plan are crucial.
To solve this issue, we propose Cooperative Plan Optimization (CaPo) to enhance the cooperation efficiency of LLM-based embodied agents. Inspired by human cooperation schemes, CaPo improves cooperation efficiency with two phases: 1) meta-plan generation, and 2) progress-adaptive meta-plan and execution.
In the first phase, all agents analyze the task, discuss, and cooperatively create a meta-plan that decomposes the task into subtasks with detailed steps, ensuring a long-term strategic and coherent plan for efficient coordination.
In the second phase, agents execute tasks according to the meta-plan and dynamically adjust it based on their latest progress (e.g., discovering a target object) through multi-turn discussions.
This progress-based adaptation eliminates redundant actions, improving the overall cooperation efficiency of agents. Experimental results on the ThreeDworld Multi-Agent Transport and Communicative Watch-And-Help tasks demonstrate CaPo’s much higher task completion rate and efficiency compared with state-of-the-arts.

## 1 Introduction

Large Language Models (LLMs) have demonstrated remarkable capabilities in understanding and generating human language, complex reasoning, and planning, achieving impressive performance (OpenAI, ; Touvron et al., ).
These advancements empower LLM-based embodied agents to autonomously make plans (Li et al., ; Padmakumar et al., ; Zhu et al., ; Wang et al., ; Wu et al., ; Huang et al., ) and perform reasoning (Du et al., ; Hao et al., ; Zhou et al., ; Huang et al., )
by using human language to assist people in daily activities, such as housework and daily chores.
The next milestone for agents is to cooperate with others to achieve joint tasks. This is crucial not only for efficiently performing simple tasks but also for tackling complex ones that cannot be completed in isolation due to their inherent complexity or the dynamic nature of the environment (Zhang et al., ; Guo et al., ; Mandi et al., ; Zhang et al., ).

Figure: Figure 1: Procedure example of task accomplishment of CoELA (Zhang et al., ) and our CaPo. In CoELA, after each action execution, Alice and Bob communicate to decide next action which is a greedy single-step plan and suboptimal. For example, they do not use wood basket which can contain sever objects, and both extemporaneously move a single item to the target bed without a long-term strategic and collaborative plan. Differently, in CaPo, Alice and Bob first discuss to make a long-term meta-plan for strategical cooperation in which Alice is arranged to move several target items into a wood basket, and Bob moves the remaining target items and also searches the unknown objects. Then during execution phase, both follow the meta-plan to accomplish task, and dynamically adapt the meta-plan the latest task progress, ensuring its effectiveness and efficiency in coordinating agents.
Refer to caption: x1.png

Notably, the cooperation among LLM-based embodied agents is rarely investigated despite being highly desired. Conventional works often focus on adopting reinforcement learning (RL) (Jiang & Lu, ; Liu et al., ; Wang et al., ) to explore the dynamics of cooperative behavior among non-LLM-based agents. In spite of their promising performance in certain scenarios, RL-based cooperation methods exhibit limited adaptability across different tasks (Dittadi et al., ; Cobbe et al., ), since they are often not trained on large-scale data and lack sufficient generalization ability. To solve this issue, in this work, we are particularly interested in the problem of “how to develop an effective collaboration framework for LLM-based agents”, since LLMs have revealed strong reasoning, planning, and communication ability across different tasks and thus are regarded as good agents’ brains.

Among the limited related works, CoELA (Zhang et al., ) proposes an LLM-based multi-agent cooperation framework in which after each action execution, agents communicate to devise a single-step plan for the next action.
Despite its significant advancements, CoELA’s short-term, single-step planning, which lacks consideration for long-term strategic collaboration, often results in extemporaneous and incoherent actions among agents, leading to several potential issues. Firstly, without a long-term coherent collaboration plan, it leads to numerous redundant action steps and increased costs, since agents’ movement is not easy and is indeed expensive in the physical world. For instance, as shown in Fig. 1, for the object transport task, agent Alice and Bob do not use the wood basket which can contain several objects, and extemporaneously move their nearest target objects one by one, leading to inferior efficiency.
Moreover, complex tasks are difficult to accomplish without thorough discussion and long-term collaboration, especially in (embodied) environments where each agent has only partial observations.
Finally, without a long-term cooperative plan, agents’ extemporaneous actions can result in mistakes with severe consequences. For instance, in search-and-rescue missions, poor coordination can have dire outcomes, such as endangering human lives due to the complex nature of these operations.

Contributions. To address the above issues, we propose a novel and effective Cooperative Plan Optimization (CaPo) framework that uses LLMs’ strong reasoning and planning ability to enhance the cooperation efficiency of LLM-based embodied agents. Inspired by human cooperation schemes (Tuomela, ; Thürmer et al., ), CaPo engages agents in multi-turn discussions to create and update a long-term strategic and coherent meta-plan, providing step-by-step guidance to coordinate agents and efficiently complete tasks.

Specifically, to accomplish a task, CaPo consists of two phases: 1) meta-plan generation, providing long-term strategical and coherent guidance for coordinating agents, and 2) progress-adaptive meta-plan and execution, dynamically adapting the meta-plan to agents’ latest progress.
In the first phase, agents analyze the task and discuss with other agents for collecting relevant information. Next, one agent is responsible for making a meta-plan which decomposes the task into subtasks with detailed accomplishment steps like agent allocations, and then collects the feedback from other agents for further meta-plan refinement. The steps of meta-plan generation and refinement will continue until all agents reach a consensus or the communication cost is exhausted. This approach ensures the thorough discussion and analysis of all agents, helping to make a long-term strategical and coherent meta-plan for efficiently coordinating all agents. For example, as illustrated in Fig. 1, in the object transport task, agents Alice and Bob are strategically assigned to different subtasks.

In the second phase, as shown in Fig. 1, agents follow the meta-plan from the first phase, and focus on their assigned subtasks. As progress is made, agents may complete subtasks or make important observations, such as Alice in Fig. 1 discovering the object “iPhone” which is Bob’s target. Accordingly, agents dynamically adapt meta-plan to the latest task progress through multi-turn discussions, allowing Alice to handle the object “iPhone” and complete the task efficiently. This progress-adaptive approach ensures that the meta-plan remains effective in coordinating all agents, thereby enhancing cooperation efficiency.

Finally, experimental results demonstrate that CaPo significantly improves task completion rates and efficiency compared to state-of-the-art (SoTA) methods on the widely used ThreeDworld Multi-Agent Transport task (Zhang et al., ) (object transport task) and the Communicative Watch-And-Help task (Zhang et al., ) (household chore task). For instance, on the ThreeDworld Multi-Agent Transport task, CaPo surpasses the SoTA CoELA by 16.7% and 4.7% in completion rate with GPT-3.5 and GPT-4 based agents, respectively

## 2 Related Work

LLM-based Agents.
LLM-based agents (Hong et al., ; Wang et al., ; Shen et al., ; Liu et al., ) are designed to autonomously perceive environments, execute actions, accumulate knowledge, and evolve themselves, with rich real world knowledge and complex reasoning capability inherited from LLMs. Notable agents like AutoGPT (Richards & et al, ), BabyAGI (Nakajima, ), and AgentGPT (Reworkd, ) showcase remarkable proficiency in decision-making and complex reasoning.
In the embodied environment, LLM-based agents have shown superior capacity in strategic planning (Li et al., ; Padmakumar et al., ; Wu et al., ; Huang et al., ). Specifically, LLM-planner (Song et al., ) harness LLMs to do few-shot planning for embodied agents. PET (Wu et al., ) translates a task description with LLMs into a list of high-level sub-tasks. TaPA wu2023embodied enables the agent to generate executable plans by aligning LLMs with visual perception models. Another line of research focuses on harnessing LLMs’s reasoning capabilities in embodied tasks (Zhou et al., ; Huang et al., ). ELLM (Du et al., ) utilizes LLMs to set pretraining goals in RL, guiding agents towards the goal without human involvement.

Multi-Agent Cooperation.
Multi-agent cooperation and communication have been studied for decades to improve communication efficiency (Jiang & Lu, ; Li et al., )
and planning (Torreno et al., ; Zhang et al., ). Within the domain of embodied intelligence,
ProAgent (Zhang et al., ) harnesses LLMs to develop proactive agents that dynamically adjust their behavior to foster better cooperation with teammates.
RoCo (Mandi et al., ) introduce a multi-robot collaboration framework that employs LLMs for both high-level communication and low-level path planning. (Guo et al., ) proposed a prompt-based organizational framework for LLM agents to reduce communication costs and boost team efficiency.
CoELA (Zhang et al., ) enables agents to plan, communicate, and collaborate effectively, but its plan is one-step plan and is short-term. Despite these advancements, these methods focus on short-term planning and do not involve sufficient agent discussion, while ours seeks to a long-term strategical and coherent plan via agent’s thoughtful discussions for efficient multi-agent cooperation.

Optimization with LLMs.
With the advancement of prompting techniques, LLMs have shown remarkable performance across various domains (Wei et al., ; Kojima et al., ; Wang et al., ; Zhou et al., ; Madaan et al., ). Their ability to understand natural language lays out a new possibility for optimization. (Yang et al., ) first proposed to leverage LLMs as optimizer, where the optimization task is described in natural language.
OPT2I (Mañas et al., ) aims to enhance prompt-image consistency in text-to-image models by iteratively generating revised prompts with LLMs to maximize the consistency score. VislingInstruct (Zhu et al., ) proposes optimizing multi-modal instruction for multi-modal language models in a zero-shot manner.
DyLAN (Liu et al., ) is particularly relevant to our work. DyLAN (Liu et al., ) enables agents to interact for multiple rounds in a dynamic architecture to optimize the selection of agent.
In contrast, our work investigates cooperative plan optimization via multi-turn discussion between agents.

## 3 Preliminaries

We follow previous work (Zhang et al., ; Gong et al., ) and formulate the embodied multi-agent cooperation task as an decentralized partially observable Markov decision process (DEC-POMDP) (Bernstein et al., ; Spaan et al., ), which is defined as $<n,\mathcal{S},\mathcal{O},\mathcal{A},P,r,\gamma>$. Here, $n$ represents the number of agents; $\mathcal{S}$ is the finite state space; $\mathcal{O}$ denotes the observation space; $\mathcal{A}$ is a finite joint action space of all agents; $P:\mathcal{S}\times\mathcal{A}\times\mathcal{S}\rightarrow[0,1]$ denotes the transition probability function; $r=\mathcal{S}\times\mathcal{A}\rightarrow\mathbb{R}$ denotes the reward function; $\gamma\in[0,1]$ denotes the discount factor. In this framework, at time step $t\in\mathbb{N}$, each agent $i$ observes the environment’s state $s_{t}\in\mathcal{S}$, and receives an observation set $\mathcal{O}_{i}$. $\mathcal{O}_{i}$ consists of a world observation $\mathcal{O}_{i}^{w}$, which the agent gathers through its sensors, or a communication message observation $\mathcal{O}_{i}^{c}$ from other teammate agents. Agent $i$ takes actions from its action space $\mathcal{A}_{i}$, which includes a finite set of world action $\mathcal{A}_{i}^{w}$, e.g., grasping a target object, or a finite set of messaging action $\mathcal{A}_{i}^{c}$. Then agents receive a shared reward $r_{t}=r(s_{t},a_{t})$, where $a_{t}\in\mathcal{A}$ denotes the joint actions of agents, and observe a new state $s_{t+1}$ with probability $P(s_{t+1}|s_{t},a_{t})$.
We formulate the problem with two decentralized intelligent embodied agents working together to complete a long-horizon rearrangement task (Zhang et al., ; Batra et al., ) in a multi-room indoor environment.
During the task, agents can execute multiple kinds of actions, such as navigation, interaction, and communication by sending messages.

## 4 Cooperative Plan Optimization

We first introduce the overall framework of CooperAtive Plan Optimization (CaPo) for LLM-based embodied agents in Sec. 4.1.
We then respectively elaborate on the two key phases of CaPo, i.e., meta-plan generation and progress-adaptive meta-plan and execution, in Sec. 4.2 and Sec. 4.3.

Figure: Figure 2: Overview of the CooperAtive Plan Optimization (CaPo) framework for embodied multi-agent cooperation. CaPo consists of two key phases: 1) meta-plan Generation: All agents collaboratively formulate a meta-plan before taking any actions through multi-turn discussions. One agent serves as meta-plan designer, responsible for creating the meta-plan, while all other agents serve as meta-plan evaluators, providing critical feedback about meta-plan. 2) Progressive-adaptive meta-plan and Execution: As new progress is made, agents adopt a progress-adaptive planning module to adapt the meta-plan to the latest task progress, ensuring the effectiveness of meta-plan.
Refer to caption: x2.png

### 4.1 Overall Framework of CaPo

CaPo aims to enhance cooperation efficiency of LLM-based embodied agents. Its key idea is to create a long-term meta-plan for strategically and coherently coordinating agents to complete a rearragement task.
Accordingly, agents follow the meta-plan to complete task step by step, and dynamically adapt the meta-plan to their latest progress, thereby avoiding redundant work allocation and improving overall cooperation efficiency.

The overall pipeline of CaPo is in Fig. 2.
Each agent contains several modules, including 1) a perception module, 2) a memory module, 3) a communication module, 4) a cooperative planning module, 5) a progress-adaptive planning module, 6) a plan parsing module, and 7) an execution module. For each agent, the perception module gathers observations from environment, including messages from other agents and relevant scene information from the RGB-D image. The memory module dynamically stores the shared task, dialogue history between agents, agent progress, teammate progress, and action history, all formatted as text descriptions. Additionally, the semantic map from the perception module is also stored.
The communication module retrieves relevant information from the memory module and uses an LLM to generate messages that are sent to other agents. The cooperative planning module either generates the meta-plan or provides feedback on the meta-plan, as seen with Alice and Bob in Fig. 2, respectively. The progress-adaptive planning module adapts the meta-plan to agents’ latest task progress. The plan parsing module, powered by the LLM, determines which sub-plan to execute based on relevant information from the memory module and available actions for the current state. Finally, the execution module converts high-level sub-plans from the plan parsing module into primitive actions for execution.

To complete a task cooperatively and efficiently, inspired by humans collaboration (Tuomela, ; Thürmer et al., ), CaPo first analyzes the task at hand to create a long-term meta-plan before agents take any actions. All agents participate in this plan-making process, either generating meta-plan or providing feedback. The meta-plan is then dynamically refined based on the latest agent progress to ensure its effectiveness in coordinating agents.
To this end, it contains two key phases, including 1) meta-plan generation, and 2) progress-adaptive meta-plan and execution.
In the meta-plan generation phase, given a task, multiple embodied agents first gather relevant information such as object locations. Then, they discuss together to create a meta-plan that decomposes the task into subtasks and consider agent situation (e.g., agent and object locations) to assign agents to different subtasks with accomplishment steps. In the progress-adaptive meta-plan and execution phase, agents dynamically align the meta-plan with their latest progress. This is achieved through multi-turn discussion triggered by clear task progress, such as discovering target objects or successfully completing subtasks. In the following, we will elaborate on these two phases in turn.

### 4.2 meta-plan Generation

To generate the long-term meta-plan which coordinates all agents to accomplish tasks efficiently, CaPo introduces two key steps, including 1) meta-plan initialization where one agent initializes a meta-plan according to the task description and existing information, and 2) meta-plan evaluation and optimization where all agents evaluate the meta-plan and provide feedback to improve the plan.

Meta-plan Initialization. At the beginning of a task, the task description is provided to all agents, e.g, Transport 2 apples and 3 bananas to the bed.
One agent, e.g., Alice in Fig. 2, is randomly selected as the meta-plan designer, and creates the meta-plan through a *cooperative planning module*. Note that the meta-plan here, as illustrated in Fig. 3, differs from the short-term or unorganized plans used in previous work (Zhang et al., ; ; Mandi et al., ).
Specifically, the cooperative planning module is equipped with a pre-trained LLM, and leverage the LLM to generate the meta-plan.
The prompting for the LLM is organized as follows:

Here, <Task Desc>, <Instruct Head>, and <Meta-plan> are three placeholders for the task description, instruction head, and generated meta-plan.
The task description provides background descriptions about the task, while the instruction head introduces additional constraints into the generation of meta-plan, such as the format of meta-plan and available actions to generate a clear and executable plan.
Detailed prompt design is shown in Fig. 9 of Appendix.

Meta-plan Evaluation and Optimization.
The meta-plan generated by a single agent is often biased by that agent’s partial observations, resulting in a suboptimal plan that fails to coordinate all agents effectively. To address this issue, CaPo involves all agents in a multi-turn discussion to optimize the meta-plan. Specifically, the meta-plan designer (e.g., Alice in Fig. 3) broadcasts the meta-plan to all teammate agents, while teammate agents (e.g., Bob in Fig. 3) serve as meta-plan evaluators, providing feedback about the meta-plan.
Since teammate agents have different partial observations of the environment, they provide the meta-plan designer with better situational awareness to help generate a more efficient and effective meta-plan.
This optimization process continues until all agents reach a consensus, i.e., the evaluator agents are satisfied with the meta-plan, or the communication budget (e.g., maximum discussion round) is exhausted. Indeed, Fig. 8 in Sec. 5.2 analyzes the convergence analysis of the meta-plan optimization process, and shows that typically agents would reach a consensus within three rounds of discussion.

Figure: Figure 3: Examples of the evaluation and optimization process of meta-plan via multi-turn discussion between agents. The discussion is triggered by new progress, i.e., Alice founds new object ’purse’. Here, Alice acts as the meta-plan designer, while Bob serves as the meta-plan evaluator. The example is derived from the transporting task of TDW-MAT.
Refer to caption: x3.png

As shown in Fig. 2, each agent is equipped with a communication module powered by a pretrained LLM to facilitate multi-turn discussions. Specifically, the communication module first retrieves relevant information from the memory, e.g., meta-plan, agent state, and previous dialogue history among agents, then prompts the LLM to generate the message to send via the following prompt:

The tags <Meta-plan>, <Agent State>, and <Dialog History> act as placeholders for inserting the meta-plan, the agent’s state, and the dialogue history between agents. The tag <Instruct Head> differs for the meta-plan designer and evaluator: the former instructs the LLM to generate messages asking teammates for their opinions, while the latter focuses on providing feedback on the meta-plan. After receiving feedback from the teammate agents, the meta-plan creator reinitiates the process to generate a new meta-plan. Fig. 3 illustrates the evaluation and optimization process of a meta-plan through multi-turn discussions among agents. It is evident that the optimized meta-plan effectively integrates partially observed information from all agents, resulting in improved coordination and efficiency. Detailed prompt designs for the communication module can be found in Fig. 10 and 11 in the Appendix.

### 4.3 Progress-Adaptive meta-plan & \& & Execution

The optimized meta-plan acts as a high-level guide, assigning subtasks to each agent and coordinating them for efficient task completion. However, due to dynamic environmental changes and task progress updates, the meta-plan can become outdated during execution. As illustrated in Fig. 4, agents may encounter significant progress, such as discovering target objects or completing subtasks, necessitating adjustments to the meta-plan. In such cases, the previous plan becomes less effective or invalid for coordinating the agents.

To address this, we design a *progress-adaptive planning module* for CaPo for adapting the meta-plan to the agents’ latest progress. This module follows a similar process as described in Sec. 4.2—meta-plan initialization, evaluation, and optimization—but with modified prompting strategies for the LLMs. Whenever an agent makes new progress, the meta-plan designer promptly generates an updated meta-plan, followed by a multi-turn discussion among all agents to further optimize it. The LLM prompting strategies for the progress-adaptive planning module are structured as follows:

Here we introduce two placeholders, <Agent Progress> and <Teammate Progress>, to capture the task progress of agents and enable the LLM to generate progress-aware responses, such as meta-plans or communication messages. Agents engage in discussions to optimize the meta-plan until a consensus is reached or communication resources are exhausted (e.g., after three discussion rounds). Detailed prompt designs for the LLMs—responsible for generating the meta-plan and facilitating messages for both the meta-plan designer and evaluator—are provided in Fig. 11$\sim$12.

Figure: Figure 4: Two types of new progress during task execution. (a) Discover a new object poundcake. (b) complete a subtask.
Refer to caption: x4.png

Once the meta-plan or progress-adaptive meta-plan is established, each agent autonomously transforms the plan into executable actions via a plan parsing module and an execution module. The plan parsing module generates the latest sub-plan by retrieving relevant information from the memory module and converting it into text descriptions, and then compiles an Action List of all available high-level sub-plans. We implement the plan parsing module as a pretrained LLM, and prompt it with a concatenation of Instruct Head, Task Description, meta-plan, Action History, Agent Progress, and Action List to choose the most suitable sub-plan. See Fig. 13 in Appendix for more prompt details. Given the sub-plan, we adopt a similar execution module as in (Zhang et al., ) to generate primitive actions for executing the sub-plan.

## 5 Experiments

Benchmarks.
We follow CoELA, and adopt the ThreeDworld Multi-Agent Transport (TDW-MAT) task (Zhang et al., ), and the Communicative Watch-And-Help (C-WAH) task (Zhang et al., ) to test our CaPo.
TDW-MAT is built on the general purpose virtual world simulation platform TDW platform (Gan et al., ), and
requires agents to move objects by their hands or containers which can contain several objects for efficient moving to the destination. Moreover, agents can receive ego-centric 512$\times$512 RGB-D images as observation and can communicate with others.
The test set of TDW-MAT consists 24 episodes, which evenly divided into food and stuff tasks.
In C-WAH, agents are requested to complete five types of household activities, represented as various predicates with specific counts that must be satisfied. The test set contains 10 episodes, including both symbolic and visual observation settings. More details about TDW-MAT and C-WAH environments are provided in Appendix B.1 and B.2, respectively.

Metrics. On TDW-MAT, we adopt Transport Rate, i.e., the fraction of subtasks completed within 3000 time steps (a.k.a. frames), as performance metric. Note, one action step may last multiple time steps, e.g., resetting arms. On C-WAH, Average Steps to complete all tasks is used as the metric to evaluate cooperation efficiency.

Implementation. Following CoELA, we test two settings on TDW-MAT task: 1) a real-world setting where the perception module is instantiated as Mask-RCNN (He et al., ) that is trained using collected scene images (Zhang et al., ), and 2) an oracle setting with segmentation ground-truth. We use GPT-3.5-turbo and GPT-4 from the OpenAI API (OpenAI, ), and LLAMA-2-13B-CHAT (Touvron et al., ), as LLMs in embodied agents. We set default parameters for LLMs: temperature of 0.7, a maximum of 256 output tokens, and top-p sampling with $p=1$.
Our code will be made publicly available.

**Table 1: Comparison of average Transport Rate (TR, %) of all baselines on the TDW-MAT w/o and w/ Oracle Perception task. Each task requires agents to move two kinds of items, including Food and Stuff. RHP^∗ uses a single agent while all others adopt two agents. The subscript value like ${+9.6\%}$ in 57${}_{\ +9.6\%}$ denotes the relative improvement when comparing with baseline CoELA.**
|  | Classic Agents | GPT-3.5 Agents | LLAMA-2 Agents |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | RHP^∗ | RHP | CoELA | ProAgent | RoCo | CaPo${}_{\text{(ours)}}$ | CoELA | ProAgent | RoCo | CaPo${}_{\text{(ours)}}$ |
| w/o Oracle Perception |  |  |  |  |  |  |  |  |  |  |
| Food ($\uparrow$) | 49 | 67 | 67 | 68${}_{\ +1.5\%}$ | 64${}_{\ -4.5\%}$ | 70${}_{\ +4.5\%}$ | 57 | 60${}_{\ +5.3\%}$ | 59${}_{\ +3.5\%}$ | 66${}_{\ +15.8\%}$ |
| Stuff ($\uparrow$) | 36 | 54 | 39 | 44+12.8% | 47+20.1% | 45+15.4% | 48 | 55${}_{\ +14.6\%}$ | 51${}_{\ +6.3\%}$ | 56${}_{\ +16.7\%}$ |
| Avg. ($\uparrow$) | 43 | 61 | 52 | 56${}_{\ +7.7\%}$ | 55${}_{\ +5.6\%}$ | 57${}_{\ +9.6\%}$ | 53 | 57${}_{\ +7.5\%}$ | 55${}_{\ +3.8\%}$ | 61${}_{\ +15.1\%}$ |
| w/ Oracle Perception |  |  |  |  |  |  |  |  |  |  |
| Food ($\uparrow$) | 52 | 76 | 72 | 80${}_{\ +11.1\%}$ | 74${}_{\ +2.8\%}$ | 85${}_{\ +18.1\%}$ | 60 | 64${}_{\ +6.7\%}$ | 62${}_{\ +3.3\%}$ | 66${}_{\ +10.0\%}$ |
| Stuff ($\uparrow$) | 49 | 74 | 73 | 76${}_{\ +4.1\%}$ | 80${}_{\ +9.6\%}$ | 84${}_{\ +15.1\%}$ | 63 | 62${}_{\ -1.6\%}$ | 69${}_{\ +9.5\%}$ | 76${}_{\ +20.6\%}$ |
| Avg. ($\uparrow$) | 50 | 75 | 72 | 78${}_{\ +8.3\%}$ | 77${}_{\ +6.9\%}$ | 84${}_{\ +16.7\%}$ | 62 | 63${}_{\ +1.6\%}$ | 65${}_{\ +4.8\%}$ | 71${}_{\ +14.5\%}$ |

**Table 2: Comparison of Average Step (AS) of all compared methods on the C-WAH task. Symbolic obs and visual obs denote symbolic observation and visual observation settings, respectively. MHP^∗ uses a single agent while all others adopt two agents.**
|  | Classic Agents | GPT-4 Agents |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | MHP^∗ | MHP | CoELA | ProAgent | RoCo | CaPo${}_{\text{(ours)}}$ |
| Symbolic Obs ($\downarrow$) | 111 | 75 | 57 | 62${}_{\ -8.8\%}$ | 57${}_{\ +0.0\%}$ | 51${}_{\ +10.5\%}$ |
| Visual Obs ($\downarrow$) | 141 | 103 | 92 | 90${}_{\ +2.2\%}$ | 89${}_{\ +3.2\%}$ | 83${}_{\ +9.8\%}$ |

Baselines. We adopt two types of methods as our baseline: 1) classical agents, including MCTS-based Hierarchical Planner (MHP) (Puig et al., ) and Rule-based Hierarchical Planner (RHP) (Gan et al., ). 2) LLM-driven agents, including CoELA Zhang et al. (), ProAgent (Zhang et al., ), and RoCo (Mandi et al., ). CoELA Zhang et al. () features a modular framework for multi-agent planning, communication, and complete long-horizon tasks, but generate independent short-term plan for each agent. ProAgent (Zhang et al., ), and RoCo (Mandi et al., ) generate joint plans for cooperative agents, and introduce a reflection loop or environment feedback for plan validation. See more details in Appendix A.1.

### 5.1 Main results

Performance comparison. We follow CoELA to test two-agent cooperation setting, and compare with classical methods like MHP and RHP, and LLM-driven methods CoELA, ProAgent, and RoCo.

Table 1 summarizes the performance of all compared methods under the two settings of the TDW-MAT task, and shows several observations. 1) Compared with the single-agent baseline RHP, CaPo and all two-agent baselines consistently make significant improvements, showing the effectiveness of multi-agent cooperation in embodied tasks. 2) In multi-agent comparisons, our CaPo outperforms LLM-driven methods by a remarkable margin, e.g., respectively making 16.7% and 8.4% improvement over CoELA and RoCo under the oracle perception setting. 3) CaPo with different LLMs as agent brain exhibits consistent superior performance across all settings. Indeed, CaPo with LLAMA-2 achieves comparable performance with CoELA with GPT-3.5-turbo under oracle perception setting. The improvement of CaPo is derived from its meta-plan and progress-adaptive meta-plan, which both provide strategical and coherent guidance for agent cooperation, thereby improving cooperation performance.

Figure: Figure 5: Comparison of Transport Rate (%) of CoELA and CaPo using GPT-3.5 under different time steps.
Refer to caption: x5.png

Figure: Figure 7: Examples of cooperative behaviors introduced by meta-plan. Guided by meta-plan, agents show clear work and task allocation, thereby improving cooperative efficiency.
Refer to caption: x7.png

Table 2 reports the performance of all methods on the C-WAH task, and shows similar and consistent
observations to those on the TDW-MAT task. Specifically, with GPT-4 agents, our CaPo respectively makes 9.8%, 7.6% and 6.6% relative improvement on CoELA, ProAgent and RoCo, consistently highlighting the superiority of CaPo in enhancing multi-agent cooperation.

Efficiency comparison. To demonstrate the cooperation efficiency, we compare the transport rates of CaPo and CoELA with different time steps on TDW-MAT. As shown in Fig. 6, with the same time step budget and GPT-3.5 agents, CaPo consistently outperforms CoELA across various time steps, indicating its superiority to coordinate agents effectively. The improvement is particularly clear in scenarios of small time steps. For example, given 1,000 time steps, CaPo doubles the transport rate of CoELA by improving 10% to 23%. This shows that with limited time or resources, a cooperative meta-plan can significantly improve cooperation efficiency.

Qualitative Analysis. Here we investigate the agents’ behavior in CaPo with GPT-4 on the C-WAH task.
In the meta-plan generation phase, as shown in Fig. 3, agents ask questions, provide feedback, and collaboratively refine the initial meta-plan. Moreover, in this phase, Fig. 7 shows that with meta-plan as guidance, two agents, Alice and Bob, have clear work/labor allocation to complete tasks, thereby avoiding redundant steps and improving cooperation efficiency.
For the progress-adaptive meta-plan and execution phase, Fig. 6 also shows that when agent Alice achieves progress, e.g., discovering three target objects, both agents will accordingly discuss to adapt the meta-plan, e.g., grasping 1 wine and 1 cupcake by Alice. This ongoing adaptation of the meta-plan provides strategic, coherent, and timely guidance, facilitating efficient coordination among agents and ultimately enhancing multi-agent cooperation.

### 5.2 Ablation Study

Effects of each component in CaPo.
Here we examine the effects of two key components: 1) meta-plan generation, which includes meta-plan initialization, evaluation, and optimization, and 2) the progress-adaptive meta-plan. To evaluate their impact, we first remove both components from CaPo, resulting in CaPo1.
As shown in Table 4, CaPo2, which includes meta-plan initialization but freezes the meta-plan during subsequent procedures, improves upon CaPo1 by approximately 1% across three metrics, demonstrating the value of meta-plan initialization. Similarly, CaPo3, which incorporates the full meta-plan generation process, outperforms CaPo2 by a significant margin, highlighting the benefits of meta-plan evaluation and optimization. Finally, CaPo achieves a 7% improvement over CaPo3, showcasing the effectiveness of the progress-adaptive meta-plan. These results underscore the importance of each component in the CaPo framework.

**Table 3: Effects of the components in CaPo using GPT-3.5 on TDW-MAT task. We report the transport Rate (TR, %). MP” denote ‘Meta Plan” and Progress-Adaptive Meta Plan”, respectively.**
| Method | Food ($\uparrow$) | Stuff ($\uparrow$) | Avg. ($\uparrow$) |
| --- | --- | --- | --- |
| CaPo1 (No MP + No Pro. MP) | 72 | 75 | 73 |
| CaPo2 (MP Initialization + No Pro. MP) | 73 | 76 | 74 |
| CaPo3 (MP Generation + No Pro. MP) | 74 | 80 | 77 |
| CaPo (MP Generation + Pro. MP) | 85 | 84 | 84 |

Effects of agent number.
Table 4 investigates the effects of agent number in CaPo using GPT-4 on the C-WAH task, where “CaPo $\times$ C” denotes using C GPT-4 agents. We can observe that increasing agent number to 3 significantly reduces the overall time step number required to complete tasks. This improvement also shows the effectiveness of our CaPo on multiple agent cooperation. However, increasing the number of agents to four results in only minor or degraded improvements.
This is because for simple tasks, agents are too much and suffer from highly-frequent agent dispatch, leading to inferior collaboration efficiency. For instance, setting up a dining table does not require four waiters, as a maximum of two agents is sufficient.

Figure: Figure 8: Percentage (%) of discussion rounds needed for agents to reach consensus on a meta-plan, based on results from TDW-MAT.
Refer to caption: x8.png

Progress in meta-plan adaptation. Fig. 4 in Sec. 4.3 shows two clear task progress examples: 1) discovering a target object and 2) completing a subtask, both of which can trigger agents to adapt the meta-plan to their latest task progress.
Such progress is crucial, as agents need to continually refine the meta-plan to complete tasks efficiently and maximize cooperation. Conversely, actions without significant progress, such as entering a new room, do not prompt agents to adjust the current (progress-adaptive) meta-plan. This is because it would be unnecessary, and updating the meta-plan involves communication, which incurs additional time overhead.

Convergence analysis of agent discussion.
Here we investigate the convergence of agent discussions, specifically focusing on how many rounds are required for agents to reach a consensus on the meta-plan. In the TDW-MAT environment, we set the maximum number of discussion rounds—referred to as the discussion budget—at three. As shown in Fig. 8, agents reach consensus on the new meta-plan within three rounds in most cases, with 78.9% achieving consensus in the “Stuff” tasks. This meta-plan, which incorporates the states of all agents, enables more efficient cooperation in task completion. Furthermore, by limiting the number of discussion rounds, CaPo strikes a balance between discussion effectiveness and budget, preventing unnecessary or prolonged discussions.

## 6 Conclusion

In this work, we introduce Cooperative Plan Optimization (CaPo) to enhance cooperation efficiency of LLM-driven embodied agents. CaPo first proposes to create a strategic and coherent meta-plan through multi-turn agents discussion before executing any actions. CaPo first proposes creating a strategic and coherent meta-plan through multi-turn discussions among agents before executing any actions. This meta-plan serves as an action guide to efficiently coordinate multiple agents in completing tasks. During the execution phase, agents dynamically adapt the meta-plan to their latest task progress, maintaining the effectiveness of the meta-plan in coordinating agents to complete tasks efficiently. Experimental results on TDW-MAT and C-WAH tasks show the higher task completion rates and efficiency of CaPo compared to state-of-the-arts.

While CaPo significantly improves multi-agent cooperation efficiency, it has limitations, specifically its heavy reliance on LLMs for reasoning and planning during meta-plan generation and adaptation. As shown in Table 1, agents using stronger LLMs like GPT-3.5 outperform those using weaker ones like LLAMA-2. This dependency is a common challenge for LLM-based frameworks like CoELA.

## Reproducibility Statement

We provide detailed descriptions of the two aforementioned embodied environments in Sec.B, covering task settings, as well as the observation and action spaces of the agents. Additionally, we present the detailed prompt designs used in our LLMs in Sec.C of the Appendix. Furthermore, we include a section in Appendix Sec. A.3 to demonstrate the reproducibility of our experimental results on the TDW-MAT environments.

## Appendix A Additional Experiments and Discussion

### A.1 Baseline models

We adopt two types of methods as our baseline, including classical agents and LLM-driven multi-agents. (1) The classical agents include MCTS-based Hierarchical Planner (MHP) (Puig et al., ) which is a hierarchical planner originating from the original Watch-And-Help Challeng, and Rule-based Hierarchical Planner (RHP) (Gan et al., ) derived from a strong baseline in the ThreeDWorld Transport Challenge.
(2) LLM-driven agents consist of CoELA Zhang et al. (), ProAgent (Zhang et al., ), and RoCo (Mandi et al., ). Cooperative Embodied Language Agent (CoELA) (Zhang et al., ) can plan, communicate, and collaborate with other agents to complete long-horizon tasks, but generate independent short-term plan for each agent. In addition, we also introduce two more baselines – ProAgent (Zhang et al., ) and RoCo (Mandi et al., ), and implement them on TDW-MAT and C-WAH using source codes. These two baselines generate joint plans for cooperative agents, and introduce a reflection loop or environment feedback for plan validation.

### A.2 Results with GPT-4

Here we further provide results on TDW-MAT task using GPT-4 in Table 5. We can observe that our proposed method CaPo achieves consistently better performance that baseline methods across all tasks, including food and stuff transportation. This also demonstrate the effectiveness of our method on improving multi-agent cooperation.

**Table 5: Transport Rate (TR) comparison on the TDW-MAT task using GPT-4. RHP^∗ uses a single agent while all others adopt two agents. Results with oracle perception are reported.**
| Method | Food ($\uparrow$) | Stuff ($\uparrow$) | Avg. ($\uparrow$) |
| --- | --- | --- | --- |
| RHP^∗ | 52 | 49 | 50 |
| RHP | 76 | 74 | 75 |
| CoELA | 87 | 83 | 85 |
| CaPo | 90 | 87 | 89 |

### A.3 Reproducibility of results

LLM-driven reasoning and planning tend to be stochastic, requiring multiple runs to assess stability. To verify the stability and reproducibility of our method, we conducted three runs on TDW-MAT with oracle perception and GPT-3.5 agents. As shown in Table 6, the results exhibit minor variance across runs, demonstrating the stability and reproducibility of our method.

**Table 6: Transport Rate (TR) comparison on the TDW-MAT task using GPT-3.5 and oracle perception. We perform 3 runs with random seeds and report mean and variance.**
| Runs | Food ($\uparrow$) | Stuff ($\uparrow$) | Avg. ($\uparrow$) |
| --- | --- | --- | --- |
| 1 | 0.85 | 0.84 | 0.84 |
| 2 | 0.84 | 0.81 | 0.82 |
| 3 | 0.85 | 0.83 | 0.84 |
| Average | 0.84 (0.006) | 0.82 (0.015) | 0.83 (0.012) |

## Appendix B Additional Environment Details

We evaluate our method and all baseline methods in two simulated environments: ThreeDWorld Multi-Agent Transport (TDW-MAT) and Communicative Watch-And-Help (C-WAH). We follow CoELA Zhang et al. () and list detailed introductions to these environments below.

### B.1 ThreeDWorld Multi-Agent Transport

#### Tasks.

TDW-MAT consists of two types of tasks, food-transporting task and stuff-transporting task. The food-transporting task has 6 types of targets
(apple, banana, orange, bread, loaf bread, and burger) and 3 containers (bowl, plate, and tea tray). In contrast, the stuff-transporting task has 6 different types of targets(calculator, mouse, pen, lighter, purse, and iPhone) and 3 containers (plastic basket, wood basket, and wicker basket). In each task, there are 10 target objects and 2 to 5 containers in total. Additionally, there are 4 types of rooms:
living room, office, kitchen, and bedroom, and objects are placed in these rooms consistent with
common sense. The agents are tasked with transporting as many target objects as possible to the goal position using containers as tools. Each container can carry up to three objects, while without a container, an agent can transport only two objects at a time. The agents must transport as many target objects as possible within 3000 frames.

#### Observation Space

The embodied agent receives a variety of observations, with the primary ones being an egocentric RGB image and a depth image. Additionally, there are several auxiliary observations. The observation space includes:

- •
RGB image: This is an egocentric image captured by a forward-facing camera, with a resolution of $512\times 512$ and a field of view of 90 degrees.
- •
Depth image: This image shares the same camera intrinsic parameters as the RGB image.
- •
Oracle Perception (optional): An image where each object ID is represented by a distinct color, using the same camera intrinsic parameters as the RGB image.
- •
Agent position and rotation: The position and rotation of the agent within the simulation environment.
- •
Messages: Communications sent by all agents.
- •
Held objects: Information about the objects currently held by the agent.
- •
Opponent held objects: Information about objects held by another agent, if the agent is within view.

#### Action Space

In TDW-MAT, agents can perform 7 distinct types of actions to interact with the environment or communicate with each other. Each action spans multiple frames, and the detailed action space is outlined below:

- •
Move forward: The agent advances by $0.5$m.
- •
Turn left: The agent rotates left by $15$ degrees.
- •
Turn right: The agent rotates right by $15$ degrees.
- •
Grasp: The agent grasps an object, successfully performing this action only when in close proximity to the object. The object can be either a target or a container.
- •
Put In: The agent places a target into a container, an action that is possible only when the agent is holding a target in one hand and a container in the other.
- •
Drop: The agent releases the objects held in hand.
- •
Send message: The agent sends a message to other agents, with a limit of 500 characters per frame.

### B.2 Communicative Watch-And-Help

Communicative Watch-And-Help (C-WAH) builds upon the Watch-And-Help challenge Puig et al. () by incorporating the ability for agents to send messages to one another. Sending messages, like other actions, consumes one timestep and is subject to a maximum length constraint.

**Table 7: Task description in C-WAH. The tasks are divided into five types, each containing several predicates.**
| Task Name | Predicate Set |
| --- | --- |
| Prepare afternoon tea | ON(cupcake,coffeetable), ON(pudding,coffeetable),<br>ON(apple,coffeetable), ON(juice,coffeetable),<br>ON(wine,coffeetable) |
| Wash dishes | IN(plate,dishwasher), IN(fork,dishwasher) |
| Prepare a meal | ON(coffeepot,dinnertable),ON(cupcake,dinnertable),<br>ON(pancake,dinnertable), ON(poundcake,dinnertable),<br>ON(pudding,dinnertable), ON(apple,dinnertable),<br>ON(juice,dinnertable), ON(wine,dinnertable) |
| Put groceries | IN(cupcake,fridge), IN(pancake,fridge),<br>IN(poundcake,fridge), IN(pudding,fridge),<br>IN(apple,fridge), IN(juice,fridge),<br>IN(wine,fridge) |
| Set up a dinner table | ON(plate,dinnertable), ON(fork,dinnertable) |

#### Tasks

The Communicative Watch-And-Help (C-WAH) framework includes five types of tasks: Prepare afternoon tea, Wash dishes, Prepare a meal, Put groceries, and Set up a dinner table. These tasks encompass various household activities, each consisting of several subtasks described by predicates in the ”ON/IN(x, y)” format, such as ”Put x ON/IN y”. Detailed descriptions of the tasks are provided in Table 7.

The primary objective is to complete all given subtasks within 250 timesteps, with each task containing between 3 to 5 subtasks.

#### Observation Space

C-WAH offers two modes of observation: Symbolic Observation and Visual Observation.

In Symbolic Observation, following the original Watch-And-Help challenge, an agent can access comprehensive object information within the same room, including location, status, name, and relationships.

In Visual Observation, agents receive egocentric RGB and depth images along with auxiliary observations. The detailed observation space includes:

- •
RGB image: An egocentric image from a forward-facing camera, with a resolution of $256\times 512$ and a field of view of 60 degrees.
- •
Depth image: An image with the same camera intrinsic parameters as the RGB image.
- •
Oracle Perception: An image where each object ID is mapped to a color, sharing the same camera intrinsic parameters as the RGB image.
- •
Agent position: The agent’s position within the simulation world.
- •
Messages: Communications sent by all agents.
- •
Held objects: Information about the objects currently held by the agent.
- •
Opponent held objects: Information about objects held by another agent, if visible.

#### Action Space

The action space in C-WAH closely resembles that of the original Watch-And-Help Challenge, with the addition of the send message action. The detailed action space includes:

- •
Walk towards: Move towards an object in the same room or towards a specific room.
- •
Turn left: Rotate left by 30 degrees.
- •
Turn right: Rotate right by 30 degrees.
- •
Grasp: Grasp an object, which can be successfully performed only when the agent is close to the object.
- •
Open: Open a closed container, performable only when the agent is near the container.
- •
Close: Close an open container, performable only when the agent is near the container.
- •
Put: Place held objects into an open container or onto a surface, performable only when the agent is near the target position.
- •
Send message: Communicate with other agents, with a limit of 500 characters per message.

## Appendix C Prompt Template

We list the prompts template for meta plan initialization, communication module of Alice, communication module of Bob, cooperative planning module, and the plan parsing module as follows.

Figure: Figure 9: Prompts for LLM in generating meta plan.
Refer to caption: x9.png

Figure: Figure 10: Prompts for LLM in the communication module of the mentor agent, e.g., Alice.
Refer to caption: x10.png

Figure: Figure 11: Prompts for LLM in the communication module of the teammate agent, e.g., Bob.
Refer to caption: x11.png

Figure: Figure 12: Prompts for LLM in cooperative planning module to generate progress-adaptive meta plan.
Refer to caption: x12.png

Figure: Figure 13: Prompts for LLM in the plan praising module.
Refer to caption: x13.png

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: Please see the weaknesses above.
soundness: 3
strengths: - Introducing a meta-plan mechanism to ensure agents can form and follow a long-term strategic and coherent plan can improve the coordination's efficiency.

- The paper is clearly written and well-organized.

- The ablation study and qualitative analysis provides valuable insights.
summary: The paper introduced Cooperative Plan Optimization into the CoELA framework to improve embodied multi-agent cooperation. A new meta-plan mechanism ensures agents can form and follow a long-term strategic and coherent plan for efficient coordination. Multi-turn discussions following pre-defined templates among agents are triggered whenever new progress is made to adapt the meta plan to progress. The experiments on two embodied multi-agent cooperation benchmarks demonstrate the proposed method achieves better performance than sota.
weaknesses: - Since the paper reuses 1) a perception module, 2) a memory module, 3) a communication module, and 7) an execution module from the CoELA paper, it may be better to put the discussion of these modules into the preliminary sections rather than in the method section to make this paper's contribution more clear.

- The contribution is limited since the only improvement is achieved by introducing a pre-defined communication protocol by prompting into an existing CoELA framework.

- The paper uses the same formalization and experiments set up as prior work CoELA, which is not a bad thing. For the main results in Table 1, the column with RHP*, RHP, and LLAMA-2 CoELA is identical as reported in the CoELA paper but with no appropriate notation. In the meantime, only the CoELA baseline's results are changed to use GPT-3.5 backbones instead of the GPT-4 used in the original paper, which also causes a major performance drop (avg. 71 -> 52; 85->72) compared to the reported results in the original paper. Results with gpt-4 are reported as original in Table 2 and Appendix Table 5, where the performance gain is down to 4 points rather than 13 points. Why the results are organized as such needs further clarification.

- One of the interesting settings of the CoELA paper is that they formalized communication as a step that also comes with a cost so the agents need to balance the cost and gain of communication. They make the findings that it's not effective yet for LLM-based agents to communicate spontaneously, which may be part of the motivation for this paper. In this paper, however, it's not specifically discussed whether this costly communication is still employed, and if so, some analysis around the communication cost and corresponding efficiency achieved may provide valuable insights for the community.

- Is the progress-adaptive planning module always triggered whenever there is new progress? How many rounds of meta-plan updates are there on average and how many steps would be used to conduct this meta-plan discussion? Is the communication module also needed after the progress-adaptive planning module makes the update to share it with the others? What happens if two agents make new progress at the same time and all want to be the meta-plan designer? What will happen if agents can't reach a consensus? Will they just have no communication afterward or try to communicate again after how many steps or stuck at arguing every few steps?

- Minor typos:
  - L70: typo, 'sever' -> 'several'

---

confidence: 5
contribution: 1
initial_rating: 5
presentation: 3
questions: 1. Can the authors elaborate further on their contributions compared to CoELA? The current presentation does not sufficiently highlight the novelty of this work.
2. Would the authors consider conducting additional experiments in different environments to better demonstrate the generalizability and effectiveness of their method?
3. Is it possible to reduce the communication cost by replacing the broadcasting communication mechanism with a different network design?
4. In Figure 8, the authors claim that consensus is typically reached within 3 rounds. However, over 20% of trials still fail to reach consensus after 3 rounds. Is setting the discussion budget to 3 rounds a reasonable choice given this observation?
soundness: 2
strengths: 1. The paper is well-written and easy to follow. The figures and illustrative examples help demonstrate the effectiveness of the proposed progress-adaptive meta-plan.
2. The related work section is sufficiently comprehensive, providing a clear context for the study.
3. Complex ablation studies on each module of CaPo, the effect of agent number, and convergence analysis.
summary: The paper introduces Cooperative Plan Optimization (CaPo) to improve cooperation among LLM-based embodied agents. Existing methods lack strategic planning, leading to inefficiency and failures in complex cooperative tasks. CaPo enhances cooperation through two phases: meta-plan generation and progress-adaptive execution. In the first phase, agents collaboratively analyze the task and create a strategic plan with detailed steps. In the second phase, agents execute and adjust the plan dynamically based on real-time progress, supported by ongoing multi-turn discussions. This progress-based adaptation prevents redundant actions and boosts overall task efficiency. Empirical experiments on the ThreeDworld Multi-Agent Transport and Communicative Watch-And-Help tasks demonstrate CaPo’s superiority in task completion and efficiency.
weaknesses: 1. This work appears to be a straightforward extension of CoELA, with minimal innovation beyond basic prompt engineering for the meta-plan module. The problem setting and agent framework largely mirror those of CoELA. For instance, as noted in Section 4.1, the CaPo agent comprises seven modules: 1) perception, 2) memory, 3) communication, 4) cooperative planning, 5) progress-adaptive planning, 6) plan parsing, and 7) execution. The CoELA agent already includes five of these, lacking only the cooperative planning and progress-adaptive planning modules, while these 2 modules can be easily implemented by prompting LLM.
2. The authors limited their experiments to the TDW-MAT and C-WAH testbeds, which are already used in CoELA. To demonstrate the broader effectiveness of their method, it would be beneficial to include experiments in new and varied environments.
3. The process of updating the meta-plan relies on broadcasting communication among agents, which incurs significant communication costs.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: In the ablation study, the paper mentions the impact of the number of agents, but the game appears to only contain two players, Alice and Bod. I'm confused about the number of agents. Please clarify how you tested different numbers of agents if the main experiments only used two.
soundness: 3
strengths: The paper introduces a novel two-phase planning approach, Cooperative Plan Optimization (CaPo), which addresses the limitations of single-step planning in CoELA. This approach demonstrates its effectiveness through experiments on the TDW-MAT and C-WAH environments.
summary: This paper introduces the Cooperative Plan Optimization (CaPo) framework, which aims to enhance the long-term planning cooperation efficiency among Large Language Model (LLM)-based embodied agents. CaPo includes two-phase: meta-plan generation and progress-adaptive meta-plan and execution. Experimental results on the TDW-MAT and C-WAH tasks demonstrate CaPo's significant improvement over state-of-the-art methods.
weaknesses: The main content of the article appears to be an extension of CoELA[1], enhancing its single-step planning approach by incorporating multi-turn discussions after the execution phase. Meanwhile, several works, for example, [2,3] have already demonstrated multiple rounds of discussions in various contexts, so the novelty and significance of this article's contribution seem somewhat lacking.

Additionally, I confess that I am not familiar with ThreeDworld Multi-Agent Transport (TDW-MAT) task, and the Communicative Watch-And-Help (C-WAH) task, so I am not quite clear on how to assess their complexity. However, for works [4,5,6] based on Minecraft, which typically evaluates dozens or even hundreds of tasks. I am cautiously skeptical about whether considering only 6 or 10 target tasks in food and stuff settings is sufficient to thoroughly validate the capabilities of the proposed framework. How the complexity of TDW-MAT and C-WAH tasks compares to Minecraft tasks? Whether you have plans to evaluate on a larger set of tasks in future work?

[1] Zhang, H., Du, W., Shan, J., Zhou, Q., Du, Y., Tenenbaum, J.B., Shu, T. and Gan, C., 2023. Building cooperative embodied agents modularly with large language models. arXiv preprint arXiv:2307.02485.
[2] Du, Y., Li, S., Torralba, A., Tenenbaum, J.B. and Mordatch, I., 2023. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325.
[3] Wang, Z., Mao, S., Wu, W., Ge, T., Wei, F. and Ji, H., 2023. Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. arXiv preprint arXiv:2307.05300.
[4] Wang, Z., Cai, S., Chen, G., Liu, A., Ma, X. and Liang, Y., 2023. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560.
[5] Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L. and Anandkumar, A., 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291.
[6] Cai, S., Zhang, B., Wang, Z., Ma, X., Liu, A. and Liang, Y., 2023. Groot: Learning to follow instructions by watching gameplay videos. arXiv preprint arXiv:2310.08235.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: What is the standard error for the metrics shown in Table 1? 

What is the runtime of this method at each turn? Can it be run in real-time with humans and how does planner quality change as a function of time?

Nice to have - what are results from playing this method with humans compared to other baselines? What do humans prefer?
soundness: 4
strengths: The authors did a nice job ablating the different modular components of their agent architecture. Oftentimes in the LLM agent literature plans are either made in a single-turn or an agent just refines what to do in its own mind, but introducing multi-turn group communication seems like an effective strategy if one has enough of a communication budget. The architecture itself seems highly reproducible, the benchmarks are challenging, and the impact of which LLM underlies communication is thoughtfully examined.
summary: This paper introduces a method called CaPo, which uses LLM agents having multi-turn conversations with each other to create meta-plans (a collection of subtasks for solving the overarching problem that are assigned to each agent). Not only does it precompute these tasks, it dynamically adjusts them over time as agents perceive new information or interact with objects in ways that impact the preexisting plan. By testing on realistic simulators emulating embodied household tasks, the authors show the efficacy of the approach compared to conventional LLM approaches and classical symbolic planners.
weaknesses: The experiments could benefit more from being evaluated on real humans collaborating with LLM agents on tasks such as Watch and Help. If the ultimate goal of this line of work is to build embodied agents that help humans, understanding how real humans complete the task differently with different agents is key. Extra evaluations could show the same efficiency metrics but also metrics indicating which agents human preferred playing with. If CaPo is very verbose when making plans, humans may not prefer interacting with it despite the efficiency gains compared to a method like CoELA. It would be interesting to dissect this. 

Moreover, an analysis of the runtime cost of CaPo would be nice to see. If a multi-round discussion is needed to reformulate a plan every time new information is discovered, how long does this take? How long can prompts end up being for the LLMs (how large does the memory grow as well) and does the duration of an episode/memory size impact planner quality?

Meta-review: This paper proposes an LLM-based embodied cooperation method that extends prior methods such as CoELA with a meta plan generation and dynamic adjustment, inspired by human cooperation. This is a meaningful contribution to LLM-based multi-agent cooperation. The revision and additional results further strengthen the submission. Overall, the proposed method has demonstrated a significant improvement over existing methods on multiple challenging benchmarks.

Decision: Accept (Poster)