CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation
Authors: Jie Liu
Pan Zhou
Yingjun Du
Ah-Hwee Tan
Cees G. M. Snoek
Jan-Jakob Sonke
Efstratios Gavves
Venue: ICLR 2025 Poster (2025)
Keywords: Embodied AI
multi-agent cooperation
LLM
Primary area: applications to robotics, autonomy, planning
TL;DR: We propose a new framework to improve cooperation efficiency for embodied mutli-agent cooperation.

Abstract: In this work, we address the cooperation problem among large language model (LLM) based embodied agents, where agents must cooperate to achieve a common goal. Previous methods often execute actions extemporaneously and incoherently, without long-term  strategic and cooperative planning, leading to redundant steps, failures, and even serious repercussions in complex tasks like search-and-rescue missions where discussion and cooperative plan are crucial.  To solve this issue, we propose Cooperative Plan Optimization (CaPo) to enhance the cooperation efficiency of LLM-based embodied agents. Inspired by human cooperation schemes, CaPo improves cooperation efficiency with two  phases: 1) meta plan generation, and 2) progress-adaptive meta plan and execution. In the first phase, all agents analyze the task, discuss, and cooperatively create a meta-plan that decomposes the task into subtasks with detailed steps, ensuring a long-term strategic and coherent plan for efficient coordination.  In the second phase, agents execute tasks according to the meta-plan and dynamically adjust it based on their latest progress (e.g., discovering a target object) through multi-turn discussions.  This progress-based adaptation eliminates redundant actions, improving the overall cooperation efficiency of agents. Experimental results on the ThreeDworld Multi-Agent Transport and Communicative Watch-And-Help tasks demonstrate CaPo's much higher task completion rate and efficiency compared with  state-of-the-arts. The code is released at https://github.com/jliu4ai/CaPo.

Published as a conference paper at ICLR 2025
CAPO: COOPERATIVE PLAN OPTIMIZATION FOR EFFI- CIENT EMBODIED MULTI-AGENT COOPERATION
Jie Liu' Pan Zhou?* Yingjun Du Ah-Hwee Tan Jan-Jakol Sonke' Efstratios Gavves Uniiersity of Amsterdam, The Netherlands "Singapore Management University, Singapore 'hhe Netherlands Cancer Institute, The Netherlands *Archimedes/Athena RC, Greece j. j.liu5euva.nl panzhou3 3 @mmail com E. G.vveesuua.. ml
Cees G.M. Snoek'
ABSTRACT
this work, we address the cooperation problem among large language model (LLM) based embodied agents, where agents must cooperate to achieve common goal. Previous methods often execute actions extemporaneously and incoherently, without long-term tingtterm strategic and cooperative planning. leading to redundant steps, failures, and even serious repercussions in complex tasks like search-and-reseue missions where discussion and cooperative plan crucial. To solve this issue, we propose Cooperative Plan Optimization (CaPo) to enhance the cooperation effi- ciency of LLM-based embodied agents. Inspired by human cooperation schemes, CaPo improves cooperation efficiency with two phases: 1) meta-plan generation, and 2) progress-adaptive meta-plan and execution. In the first phase, all agents analyze the task, discuss, and cooperatively create meta-plan that decomposes the task into subtasks with detailed steps, ensuring a long-term strategic and co- herent plan for efficient coordination. In the second phase, agents execute tasks according to the meta-plan and dynamically adjust it based on their latest progress (e.g., discovering target object) through multi-turn discussions. This progress- based adaptation eliminates redundant actions, improving the overall cooperation efficiency of agents. Experimental results on the ThreeD Multi-Agent Trans- port and Communicative Watch-And And-Help tasks demonstrate CaPo's much higher task completion rate and efficiency compared with ofate-of-the-arts. The code is released at ht httpp:: : / //gitttuu wig............ com ji iuuuui4i.. /CaPo.
INTRODUCTION
Large Language Models (LLMs) have demonstrated remarkable capabilities in understanding and generating human language. complex reasoning, and planning, achieving impressive advance- ment (OpenAI, 2024; Touvron et al., 2023). These advancements empower LLM-based embodied agents to autonomously make plans (Li et al., 2023a; Padmakumar et al., 2022; Zhu et al., 2023; Wang et al., 2023a; Wu et al., 2023b; Huang et al., 2022b) and perform reasoning (Du et al., 2023; Hao et al., 2023; Zhou et al., 2024: Huang et al., 2022a) by using human language to assist people in daily activities, such as housework and daily chores. The next milestone for agents is to cooperate with others to achieve joint tasks. This crucial not only for efficiently performing simple tasks but also for tackling complex ones that cannot be completed in isolation due to their inherent complexity or the dynamic nature of the enviromment (Zhang et al., 2023b; Guo et al., 2024; Mandi et al., 2023).
Notably, the cooperation among LLM-based embodied agents is rarely investigated despite being highly desired. Conventional works often focus on adopting reinforcement learning (RL) (Jiang & Lu, 2018; Liu et al., 2021; Wang al., 2021) to explore the dynamics of cooperative behavior among agents. In spite of their promising performance in certain scenarios, RL-based cooperation methods exhibit limited adaptability across different tasks (Dittadi et al., 2021; Cobbe et al., 2019), since they are often not trained on large-scale data and lack sufficient generalization ability. To solve this issue, in this work, we are particularly interested in the problem of "how to
"Corresponding author

---

Published as a conference paper at ICLR 2025
Tusk Description: Transport mouses, pen, iPhone, and parse the bed.
Existing Agent Information: Alice knows the location of mouse, pen; Dob knows the location of purse. They don't 't know the location of iPPoone
Ours (CaPo)
CoELA
Meta Plan Generation
Step 1: Alize piapp mosnc, per, and put them isto far wood basket.
Communication & Exrcution
Bub jaiaps passe Step 2: Alke incopses miese and pes uasta weod haskett the but. lish monquers purse bed, try End ifhene and trrquat it auehe
grosps presss
Communication
banke
AlCC
Prigras-nnaetiee Mea Plan & Evecution
Step Alics jeaqe rasnne, pas, and pat them into fhe wood
Trogres-adaatinn Meta
Alkc Lazipatt pusse the bod, Bob ticepposs mosse the bed.
beaket. asaaps pussc: Step 2: Alise trussport mossa and
pen using wood
trassport passe
innquurt
Communication & Execution
Pregress
New Alice found iPhooo whet
0 updite
4
Alive go plasp par, Bo's grape pen.
Progress-ddaptive Meta
Step I: Allos graps lffure and pirs it word hasdert Dun manqurs punse to the had Step 2: Alles immponns wood busket (comnnning pem. meene, (ffune] the bed. compllesed)
Figure I: Examples of task accomplishment of CoELA (Zhang et al., 2023b) and our CaPo. In CoELA, after each action execution, Alice and Bob communicate to decide next action which is greedy single-step plan and suboptimal. For example, they do not use wood basket which can contain multiple objects, and both extemporaneously move a single item to the target bed without long- term and collaborative plan. In contrast, in CaPo, Alice and Bob first discuss to make a long-term meta-plan for strategical cooperation in which Alice is arranged to move several target items into a wood basket, and Bob moves the remaining target items and also searches the unknown objects. Then during execution phase, both follow the meta-plan to accomplish task, and dynamically adapt the meta-plan to latest task progress, ensuring its effectiveness and efficiency in coordinating agents.
develop an effective collaboration framework for LLM-based agents". since LLMs have revealed strong reasoning and planning capability as good agents' brain across different tasks.
Among the limited related works, CoELA (Zhang et al., 2023b) proposes an LLM-based multi- agent cooperation framework in which after each action execution, agents communicate to devise a single-step plan for the next action. Despite its significant advancements, CoBLA's short-term, single-step planning, which lacks consideration for long-term strategic collaboration, often results in extemporaneous and incoherent actions among agents, leading to several potential issues. D Without long-term coherent collaboration plan, agents are prone to executing numerous redundant actions, resulting in increased costs. This issue is particularly critical in the physical world, where agents' movements are inherently challenging and resource-intensive. For instance, as shown in Fig. 1, for the object transport task, agent Alice and Bob do not use the wood basket which can contain several objects, and extemporancously move their nearest target objects one by one, leading to inferior efficiency. @ Complex tasks are challenging to accomplish without thorough discussion and long-term collaboration, particularly in embodied environments where agents operate with only partial observations. 3 Without long-ten cooperative plan, agents' uncoordinated actions can lead to severe consequences. For example, in search-and-rescue missions, poor coordination may endanger lives due the complexity of such operations.
Contributions. This work introduces Cooperative Plan Optimization (CaPo), a novel framework designed to enhance the cooperation efficiency of LLM-based embodied agents by leveraging their advanced reasoning and planning capabilities. Inspired by human cooperation schemes (Tuomela, 1998; Thürmer al., 2017), CaPo enables agents to collaboratively create and refine long-term, strategic, and coherent meta-plans through multi-turn discussions. These meta-plan serve as ac- tionable guides that facilitate efficient task coordination and completion.
Specifically, CaPo addresses key challenges in multi-agent cooperation through two main phases. The first phase is Meta-Plan Generation, where agents collaboratively analyze the task and ex- change information to generate a meta-plan that decomposes the overall task into detailed subtasks, including agent assignments and execution steps. A designated agent initiates the meta-plan, while others evaluate and refine it through iteratíve feedback, ensuring alignment and optimal planning. The discussion process continues until consensus is reached or the communication budget is ex- hausted. This phase ensures thorough deliberation and creates coherent roadmap for task exe- cution. For example, in the object transport task, agents Alice and Bob are strategically assigned complementary subtasks optimize their coordination (see Fig. 1).
2

---

Published as a conference paper at ICLR 2025
The second phase is Progress-daaptive Meta-Plan and Execution, where agents dynamically adapt the meta-plan based on real-time progress and discoveries during task execution. For instance, if Alice identifies the location of an object that was initially unknown (e.g., an "iPhone"), the agents collaboratively update the meta-plan to incorporate this new information, ensuring timely and ef- ficient task completion. This adaptive mechanism maintains the meta-plan's effectiveness, even in dynamic environments, and prevents inefficiencies or redundant actions.
The proposed approach has been rigorously evaluated on benchmark tasks, including the ThreeDworld Aulii-Agent Transport (TDW-MAT) and Communicative Watch-And-Hlelp (C-WAII) tasks (Zhang et al., 2023b). Experimental results demonstrate that CaPo significantly improves task completion rates and cooperation efficiency compared to state-of-the-ar methods. Notably, on the TDW-MAT task, CaPo achieves completion rate improvement of 16.7% and 4.7% over the SoTA CoELA method with GPT-3.5 and GPT-4 agents, respectively.
2 RELATED WORK
LLM-based Agents. LLM-based agents (Hong et al., 2023; Wang et al., 2024; Shen et al., 2024; Liu et al., 2023a) are designed to autonomously perceive environments, execute actions, and accu- mulate knowledge, with rich real world knowledge and complex reasoning capability inherited from LLMs. Notable agents like AutoGPT (Richards & et al, 2021), BabyAGI (Nakajima, 2023), and AgentGPT (Reworkd, 2023) showcase remarkable proficiency in decision-making making and complex rea- soning. In the embodied environment, LLM-based agents have shown superior capacity in strategic planning (Li et al., 2023a; Padmakumar et al., 2022; Wu et al., 2023b; Huang et al., 2022b). Specifi- cally, L.LM-planner (Song et al., 2023) harness L.LMs to do few-shot planning for embodied agents. PET (Wu et al., 2023a) translates task description with LLMs into list of high-level sub-tasks. TaPA wu2023embodied enables the agent to generate executable plans by aligning LLMs with vi- sual perception models. Another research line focuses on harnessing LLMs's 's reasoning capabilities in embodied tasks (Zhou et al., 2024; Huang et al., 2022a). ELLM (Du et al., 2023) utilizes LLMs to set pretraining goals in RL, guiding agents towards the goal without human involvement.
Multi-Agent Cooperation. Multi-agent cooperation and communication have been studied for decades to improve communication efficiency (Jiang & Lu, 2018; Li et al., 2023b) and plan- ning (Torreno et al., 2017; Zhang et al., 2023a). Within the domain of embodied intelligence, ProAgent (Zhang et al., 2023a) harnesses LLMs to develop proactive agents that dynamically adjust their behavior to foster better cooperation with teammates. RoCo (Mandi et al., 2023) introduce a multi-robot collaboration framework that employs LLMs for both high-level communication and low-level path planning. (Guo et al., 2024) proposed prompt-based organizational framework for LLM agents to reduce communication costs and boost team efficiency. CoELA (Zhang et al., 2023b) enables agents plan, communicate, and collaborate effectively, but its plan is one-step plan and is short-term. Despite these advancements, these methods focus on short-term planning and do not involve sufficient agent discussion, while ours seeks a long-term strategical and coherent plan via agent's 's thoughtful discussions for efficient multi-agent cooperation.
Optimization with LLMs. With the advancement of prompting techniques, LLMs have shown re- markable performance across various domains (Wei et al., 2022; Kojima et al., 2022; Wang et al., 2022: Zhou al., 2022: Madaan et al., 2024). Their ability to understand natural language lays out a new possibility for optimization. (Yang et al., 2023) first proposed to leverage LLMs as optimizer, where the optimization task described in natural language. OPT21 (Mañas et al., 2024) aims to en- hance prompt-image consistency in text-to-image models by iteratively generating revised prompts with LLMs to maximize the consistency score. VislingInstruct (Zhu et al., 2024) proposes optimiz- ing multi-modal instruction for multi-modal language models in zero-shot manner. DyLAN (Liu et al., 2023b) is particularly relevant to our work. DyLAN (Liu et al., 2023b) enables agents to interact for multiple rounds in dynamic architecture to optimize the selection of agent. In contrast, our work investigates cooperative plan optimization via multi-turn discussion between agents.
3 PRELIMINARIES
Task Formulation. We formulate the embodied multi- cooperation task as an decentralized partially observable Markov decision process (DEC-POMDP) (Bernstein al., 2002; Spaan et al.,
3

---

Published as a conference paper at ICLR 2025
Vemory Module
Task Description
Semantic Mip
Metseplan Hiviory
Dislogne History
Agent Progrex
Tesmmate Progress
Action History
Meta-Plan Generation Phase
Environment
Communication Module
Communication Module
Messago
Hi, Bob. Thanks for your advice, I will update plan accoedingly.
111, Alice. should explore the kitchen, please updane the mera plan.
Alice
Bob
Observation
Feedbuck
Meta plan
Meta-plan
Feedback
Perception Module
Comperative Planning Module
Comperative Planning Module
Step 1:
Alice explores <living room.. Dob explores <linchens.
Step exploves <living pocma..
Mieta
Bob explores seffiecs.
Memory Module
Step N: Alice (sub-plan). Beb [sub-plan]
Seep N: Allice plan-p Bob [sub-plon]
Progress-adaptive Meta Plan & Execution Phase
Previous Meta plan
Progress?
Yos
Propres--daptive Planning Module
Environment
Exccution Module
Plan Parsing
Pregress-
Step
Alicn goes groep <hasket- seffiee, put "poni>
Module
-ddapiive
it; Bob contimues exploring <kitchert-
Meta Plan
Step [suboplan). [suh-plan]
Figure 2: Overview of the CooperAtive Plan Optimization (CaPo) framework for embodied multi-agent cooperation. CaPo consists of two key phases: 1) meta-plan Generation: All agents collaboratively formulate a meta-plan before taking any actions through multi-turn discussions. One agent serves as meta-plan designer, responsible for creating the meta-plan, while all other agents serve as meta-plan evaluators, providing critical feedhack about meta-plan. 2) Progressive-adaptive meta-plan and Execution: AS new progress is made, agents adopt poggrsss-adattin planning module to adapt the meta-plan to the latest task progress, ensuring the effectiveness of meta-plan.
>. Here, n represents the number of agents; S is the finite state space; O denotes the observation space: AA is a finite joint action space of all agents; P S A S + [0. 1] denotes the transition probability function;
2006), which is defined as < n, O,
S x A - R denotes the reward function; ? E [0, 1] denotes the discount factor. In this framework, at time step I E N, each agent observes the environment's 's state 8, E S, and receives an observation set O, O consists of world observation OJ, which the agent gathers through its sensors, or a communication message observation Of from other teammate agents. Agent takes actions from its action space A, which includes finite set of world action Ar" e.g., grasping target object, or a finite set of messaging action AÇ. Then agents receive a shared reward r; = r(8t, az). where as E A denotes the joint actions of agents, and observe new state 81+1 with probability P(s++1|st. az). We formulate the problem with two decentralized intelligent embodied agents working together to complete a long-horizon rearrangement task (Zhang et al., 2023b; Batra et al., 2020) in multi-room indoor environment. During the task, agents can execute multiple kinds of actions, such as navigation, interaction, and communication by sending messages.
CoELA Framework. CoELA Zhang et al. (2023b) is pioneering modular framework for embod- ied multi-agent cooperation, which consists of five key modules: (a) Perception, (b) Memory, (c) Communication, (d) Planning, and (e) Execution. For each agent, the (a) Perception Module gathers observations from environment, including messages from other agents and relevant scene informa- tion from the RGB-D image. The (b) Memory Module dynamically stores the shared task, dialogue history between agents, agent progress, team- mate progress, and action history, all formatted as (c) The Communication Module retrieves relevant information from the memory module and uses an LLM to generate messages that are sent to other agents. (d) The Planning Mod- ule driven by LLM decides which plan to take given related information retrieved from the memory module and available actions proposed regarding the current state. Finally, the (e) Execution Mod- ule convert high-level plan into primitive actions executable in the environment. CoELA exhibits great performances in the embodied multi-agent cooperation tasks, making it strong baseline for validating the effectiveness of our model in improving multi-agent cooperation efficiency.
text descriptions.
COOPERATIVE PLAN OPTIMIZATION
We first introduce the overall framework of CooperAtive Plan Optimization (CaPo) for LLM-based embodied agents in Sec. 4. We then respectively elaborate on the two key phases of CaPo, i.c., meta-plan generation and progress-adaptiv meta-plan and execution, in Sec. 4.2 and Sec. 4.3.
4

---

Published as a conference paper at ICLR 2025
4.1 OVERALL FRAMEWORK OF CAPO
CaPo addresses the challenge of enabling two centralized embodied agents to effectively cooperate in shared environment. In this setup, each agent has partial observations and must rely on com- munication and coordination to accomplish complex tasks. The key objective is to achieve strategic and step-by-step collaboration, where both agents contribute to the task's 's completion through effi cient planning and adaptive decision-making As shown in Fig. 2, each agent is equipped with five foundational modules from CoELA: perception, memory, communication, plan- Slan-phrasing, and exe- cution. These modules enable the agents to perceive their environment, store and retrieve relevant information, exchange messages for coordination, generate plans, and execute actions accordingly.
To complete task cooperatively and efficiently, inspired by humans collaboration (Tuomela, 1998; Thürmer et al., 2017). CaPo first analyzes the task at hand to create a long-term meta-plan before agents take any actions. All agents participate in this plan-making process, either generating meta- plan or providing feedback. The meta-plan is then dynamically refined based on the latest agent progress ensure its effectiveness in coordinating agents. To this end, contains two key phases, including 1) meta-plan generation, and 2) progress-adaptive meta-plan and execution. In the meta- plan generation phase, given task, multiple embodied agents first gather relevant information such as object locations. Then, they discuss together to create a meta- meta-plan that decomposes the task into subtasks and consider agent situation (e.g., agent and object locations) to assign agents to differ- ent subtasks with accomplishment steps. In the progress- -araeiiv meta-plan and execution phase, agents dynamically align the meta-plan with their latest progress. This is achieved through multi- turn discussion triggered by clear task progress, such discovering target objects successfully completing subtasks. In the following, we will elaborate on these two phases in turn.
4.2 META-PLAN GENERATION
To generate the long-term meta-plan which coordinates all agents to accomplish tasks efficiently, CaPo introduces two key steps, including 1) meta-plan initialization where one agent initializes a meta-plan according to the task description and existing information, and 2) meta-plan evaluation and optimization where all agents evaluate the meta-plan and provide feedback to improve the plan.
Meta-plan Initialization. At the beginning of a task, the task description is provided to all agents, e.g, Transport apples es and 3 bananas to the bed. One agent, e.g., Al- ice in Fig. 2, is randomly selected the meta-plan designer, and creates the meta-plan through cooperative planning module. Note that the meta-plan here, as illustrated in Fig. 3, differs from the short-term or unorganized plans used in previous work (Zhang et al., 2023b;a; Mandi et al., 2023). Specifically, the cooperative planning module is equipped with a pre-trained LLM, and leverage the LLM to generate the meta-plan. The prompting for the LLM is organized as follows:
Prompt: <Task
Desc>
<instruct Head>
LLM: <Meta-plas
three placeholders for the task description, instruction head, and generated meta-plan. The task description provides back- ground descriptions about the task, while the instruction head introduces additional constraints into the generation of meta-plan, such as the format of meta-plan and available actions generate a clear and executable plan. Detailed prompt design is shown in Fig. 9 of Appendix.
Here, <Task Desc> <Instruct Head> and <Meta-plan>
Meta-plan Evaluation and Optimization. The meta-plan generated by single agent often biased by that agent's partial observations, resulting in a suboptimal plan that fails to coordinate all agents effectively. To address this issue, CaPo involves all agents a multi-turn discussion to optimize the meta-plan. Specifically, the meta-plan designer (c.g.. Alice in Fig. 3) broadcasts the meta-plan to all teammate agents, while teammate agents (e.g., Bob in Fig. 3) serve as meta- plan evaluators, providing feedback about the meta-plan. Since teammate agents have different partial observations of the environment, they provide the meta-plan designer with better situational awareness to help generate more efficient and effective meta-plan. This optimization process continues until all agents reach a consensus, i.e., the evaluator agents are satisfied with the meta- plan, or the communication budget (e.g., maximum discussion round) is exhausted. Indeed, Fig. 8 in Sec. 5.2 analyzes the convergence analysis of the meta-plan optimization process, and shows that typically agents would reach consensus within three rounds of discussion.
5

---

Published as a conference paper at ICLR 2025
Meta Plan Generation
Step 1: Alice grasp "purse" (931981), 1), pur opunse> (931981) into container. Bob explore living room. Step 2: Alice put spest- (14288289), "mouse" (3660150) container. Bob go grasp found objects transport them to bod.
Discussio Round
Message
Alice
have explored Office found porse. am ourrently holding a pen. continue exploring Office guiher the remaining target objects efficienlly. Do you have suggostions for next in the meta plan?
Message to Hi Alice, considering you currenily bolding one object have found container, you can start putting objects into the container. will grasp the mouse in living room. Please updati the action plan accondingly. Satisfied with the meta plan No
Boly
Meta Plan Generation
Disconsion Round 2
Stop I: Alice go gropp container, put holding (1412214 into container, Hob go grup <iPhonc> (111 161655) Step Alice go graup <purse> (931981, <mouse> (3660110) 56660150),00 and into container; Boh put <iPhone- (11176165) into container.
Message Rob
Alice
Hi Bob, I have updased the plan according your suggestions. think could graep the iPhone in the living you are already there. Do you have any other suggestions to improve our meta plan?
Message to Alice
Hi Alice, current plan looks good. But I recommend grasp the <basker--(011122 and then put objecr into io transport more objects efficiently. grasp iPhone the living room Satisfled with metn plan : No
Bob
Meta Plan Generation
Discusiio Round
Step I: Alice grasp the <basket> (601123), put holding (149229) into <basket>(60) 1231; Bob grasp Paoooge 76165), Step 2: Alioz go gropp Spursc> (931981), Smousee> (36601 and put them <basktt2(601 123); Bob continue exploring living room.
Alice
Messape Bob
Hi Rob, I have updated the meta plan according to your suggestions. am satisfied with this meta plan; how do you think about
Message to Allce
Hi Alise, I think the mota plan is on the right track,, let's work logelher to transport as many objects as possible to the bod. Good
progress far". Satisfied with the meta plan Yos
Bob
Figure 3: Examples of the evaluation and optimization process of meta-plan via multi-turn dis- cussion between agents. The discussion is triggered by new progress, i.e., Alice founds new object 'purse'. Here, Alice acts as the meta-plan designer, while Bob serves as the -plan evaluator. The example is derived from the transporting task of TDW-MAT.
As shown in Fig. each agent is equipped with a communication module powered by a pretrained LLM to facilitate multi-turn discussions. Specifically, the communication module first retrieves relevant information from the memory, e.g., meta-plan, agent state, and previous dialogue history among agents, then prompts the LLM to generate the message to send via the following prompt:
Prompt: <Task Desc>
<Instruct Head>
<Meta-plas
<Agent State> +
<Dialog History> \n.
LLM:
<Messages>
The tags <Meta-plan>> <Agent State> and <Dialoq History> act as placeholders for inserting the meta-plan, the agent's state, and the dialogue history between agents. The tag <Instruct Head> differs for the meta-plan designer and evaluator: the former instructs the LLM to generate messages asking teammates for their opinions, while the latter focuses on providing feedback on the meta-plan. After receiving feedback from the teammate agents, the meta-plan creator reinitiates the process to generate a new meta-plan. Fig. 3 illustrates the evaluation and optimization process of a meta-plan through multi-turn discussions among agents. is evident that the optimized meta-plan effectively integrates partially observed information from all agents, resulting in improved coordination and efficiency. Detailed prompt designs for the communication module can be found in Fig. 10 and 11 in the Appendix.
4.3 PROGRESS- META-PLAN & EXECUTION
The optimized meta-plan acts as high-level guide, assigning subtasks to each agent and coordinat- ing them for efficient task completion. However, due to dynamic environmental changes and task progress updates, the meta-plan can become outdated during execution. As illustrated in Fig. 4, agents may encounter significant progress, such as discovering target objects or completing sub- tasks, necessitating adjustments to the meta-plan. In such cases, the previous plan becomes less effective or invalid for coordinating the agents.
To address this, we design -adptiivee planning module for CaPo for adapting the meta-plan to the agents' latest progress. This module follows a similar process as described in Sec. 4.2-meta- meta- plan initialization, evaluation, and optimization- but with modified prompting strategies for the

---

Published as a conference paper at ICLR 2025
Whenever agent makes progress, the meta-plan designer promptly generates updated meta-plan, followed by a multi-turn discussion among all agents to further optimize it. The LLM prompting strategies for the progress pngress-adaptive planning module structured as follows:
LLMs.
Prompt:
<Task Desc>
nntruct
Head>
<Meta-plan>
<Agent Progress>
<Teammate Progress>
<Dialog History> \n.
LLM:
<meta-plan>
<Messages>
Here we introduce two placeholders, <Agent Progress> and <Teammate Progress> to capture the task progress of agents and enable the LLM to generate progress-aware responses, such meta-plans communication messages. Agents engage in discussions optimize the meta- plan until a consensus is reached or communication resources are exhausted (e.g.. after three discus- sion rounds). Detailed prompt designs for the LLMs- for generating the meta-plan and facilitating messages for both the meta-plan designer and evaluator-are provided in Fig. 1~~1 12.
Once the meta-plan progress-adaptive meta-plan is established, each agent autonomously transforms the plan into executable actions via a plan pars- ing module and an execution module. The plan parsing module generates the latest sub-plan by retrieving relevant information from the memory module and converting it into text descriptions, and then compiles Action List of all avail- able high-level sub-plans. We implement the plan parsing module a pretrained LLM, and prompt with a concatenation of rust Head, Task Description, meta-plan, Action History, Agent Progress, and Action List choose the most suitable sub-plan. See Fig. 13 in Appendix for more prompt details. Given the sub-plan, we adopt a similar execution
Subtask: Put poundcake fridge.
(a) Discover turget object
(b) Complete subtask
Figure 4: Two types of new progress dur- ing task execution. (a) Discover
ob-
ject poundeake. (b) complete a subtask.
module as in (Zhang et al., 2023b) to generate primitive actions for executing the sub-plan.
EXPERIMENTS
Benchmarks. We follow CoELA, and adopt the ThreeDworld Multi-Agent Transport (TDW-MAT) task (Zhang et al., 2023b), and the Communicative Watch-And-Hell (C-WAH) task (Zhang et al., 2023b) to test our CaPo. TDW-MAT is built on the general purpose virtual world simulation plat- form TDW platform (Gan et al., 2020), and requires agents move objects by their hands con- tainers which can contain several objects for efficient moving to the destination. Moreover, agents receive ego-centric 512x512 RGB-D images observation and can communicate with others. The test set of TDW-MAT consists 24 episodes, which evenly divided into food and stuff tasks. In C-WAH, agents requested to complete five types of household activities, represented various predicates with specific counts that must be satisfied. The test set contains 10 episodes, including both symbolic and visual observation settings. More details about TDW-MAT and C-WAH environ- ments are provided in Appendix B. and B.2, respectively.
Metrics. On TDW-MAT, adopt Transport Rate, i.e., the fraction of subtasks completed within 3000 time steps ka. frames). as performance metric. Note, one action step may last multiple time steps, e.g., resetting arms. On C-WAH. Average Steps complete all tasks is used as the metric to evaluate cooperation efficiency. Following CoELA (Zhang et al., 2023b), we also report Efficiency Improvement (EI) of cooperating with other agents as AM AM/Mo, where A. the main efficiency metric difference, and Mo denotes the larger of the main efficiency metric for numerical stability.
Implementation. We test two settings on TDW-MAT task: 1) a real-world setting where the per- ception module is instantiated as Mask-RCNN (He et al., 2017) that is trained using collected scene images (Zhang et al., 2023b), and 2) an oracle setting with segmentation ground -tuutth. We use GPT- 3.5-turbo and GPT-4 from the OpenAl API (OpenAI, 2024), and LLAMA-2- 2-1B--111B-CHBB 33BCHAT (Touvron et al., 2023), as LLMs embodied agents. We set default parameters for LLMs: temperature of 0.7, maximum of 256 output tokens, and top-1 sampling. Our code will be made publicly available.

---

Published as a conference paper at ICLR 2025
Classic Agents
GPT-3.5 Agents
LLAMA-2 Agents
GPT-4 Agents
RHP-t
RHPi
CoELA CaPo(uur))
CoBLA CaPo
CoELA ProAgent
RoCo
Cal CaPoue (ours))
w/o Oracle Pereeption
Food (f)
49
67
+5%
67
+33%
70
+311
+.9%
66
117%
82
+38%
82
+27%
83 34%
+-43%
Stuff (1)
36
54
134%
39
188%
45
27%
48
111%
56
122%
61
141%
57
133%
60
3.%
64
1.40%
Avg. (1)
43
61
20%
52 20%
57
| 29%
53
10%
61
19%
71
139%
69
131%
71
1 36%
74
4.11%
w/ Oracle Perception
Food (1)
52
76
+33%
72 +29%
85 +38%
60 +.3%
66 +14%
87
+411%
84
+37%
88 +122%
90
+40%
Stuff (f)
49
74
+344%
73 +.2%%
84
+-30%
63
19%
76
233%
83
441%
85
4-44%
82
4-30%
87
4-38%
Avg. (1)
50
75 +344%
72 +-00%
84
+-38%
62
4-8%6
71
118%
85
4411%
84
4-35%
85
4-38%6
+-399%
Table 1: Comparison of average Transport Rate %) of all baselines on the TDW-MAT w/o and w/ Oracle Perception task. Each task requires agents to move kinds of items, including Food and Stuff, RHP uses a single agent while all others adopt two agents. denotes results quoted from CoELA. The subscript value like +25% in 67 +25% denotes the Efficiency Improvement.
Classic Agents
Heterogeneous Agents
GPT-4 Agents
MHP*
MHP*
MHP+CoELA MHP+CaPo CoELAt
ProAgent
RoCo
CaPo
Symbolic Obs (1)
111
75 +33%
59 +45%
57 +-77%
57 +49%
62
+37%
57 +433%
51 +466%
Visual Obs (1)
141
103
4-26%
94
+34%
90 +.38%
92
+34%
90
+37%
89
+32%
83 +37%
Table 2: Comparison of Average Steps of all methods on the C-WAH task. "Symbolic Obs" and "Visual Obs" denote symbolic and visual observation settings, respectively. MHP" uses a single agent while all others adopt two agents. indicates results quoted from CoELA. The subscript value like +33% in
75 +33% denotes the Efficiency Improvement.
Baselines. We adopt three types of methods as our baseline: 1) classical agents, including MCTS-based Hierarchical Planner (MHP) (Puig et al., 2020) and Rule-based Hierarchical Plan- ner (RHP) (Gan et al., 2022). 2) LLM-driven agents, including CoELA Zhang et al. (2023b), ProAgent (Zhang et al., 2023a), and RoCo (Mandi et al., 2023). CoELA Zhang et al. (2023b) fea- tures a modular framework for multi-agent planning, communication, and complete long-horizon tasks, but generate independent short-term plan for each agent. ProAgent (Zhang et al., 2023a), and RoCo (Mandi et al., 2023) generate joint plans for cooperative agents, and introduce a reflec- tion loop or environment feedback for plan validation. 3) Heterogeneous agents, where agents with different planning and communication mechanisms collaborate within the same environment. See more details in Appendix A.I and A.2.
5.1
MAIN RESULTS
Performance comparison. We follow CoELA to test two-agent cooperation setting, and compare with classical methods like MHP and RHP, and LLM-driven methods CoELA, ProAgent, and RoCo. Table summarizes the performance of all compared methods under the two settings of the TDW- MAT task, and shows several observations. 1) Compared with the single-agent baseline RHP, CaPo and all two-agent baselines consistently make signíficant improvements, showing the effectiveness of multi-agent cooperation in embodied tasks. 2) In multi-agent comparisons, our CaPo outperforms LLM-driven methods by clear margin, e.g., respectively making 4.7% and 5.9% improvement over CoELA and RoCo under the oracle perception setting. 3) CaPo demonstrates consistently superior performance across all settings when paired with different LLMs as the agent brain. Notably, even with a less advanced LLM like GPT-3.5-turbo CaPo achieves a significantly higher performance compared to the baseline CoELA. For example, under the oracle perception setting, CaPo achieves transport rate of 84, outperforming CoELA's 72. The improvement of CaPo is derived from its meta-plan and progress-adaptive meta-plan, which both provide strategical and coherent guidance for agent cooperation, thereby improving cooperation performance.
Table 2 reports the performance of all methods on the C-WAH task. Specifically, with GPT-4 agents, CaPo respectively makes 9.8%, 7.6% and 6.6% relative improvement on CoELA, ProAgent and RoCo. Interestingly, CaPo, when paired with MHP, where the CaPo agent independently creates and updates the meta-plan, outperforms both team of two MHP agents and combination of MHP
8

---

Published as a conference paper at ICLR 2025
CoELA-SoTA CaPo-Ours
Step Alice cupkirs living roors Beb explore offec.
x1.2
84
Step
gropp foond objects
fridgr
Carreat peogress
Next aetinn Alfcet
Next action for Bobs Cnap cxpenke.
72
capokkes copekk
59
x1.9
x2.3
23
10
1000 time steps 2000 times steps 3000 time steps
Progres-atapthee
I: grsaps and eapeake Beb arage cuposke
Figure Comparison of Transport Rate (%) of CoELA and CaPo using GPT-3.5 under different time steps.
Figure 6: Example of progress-ddaptive meta-plan adaptation. New progress: Alice found target objects, wine and 2 cupcakes.
Meta Plan
I: Alice gues gropp pubding and wine: Bob grasp cupcake. Step transport holding objects
Step Alice Boli holding objects table
Alke
Bob
Alire
Go grepp pudling and
Go group cupcake.
Sub
Timmspoet puiding and
Transport cupenike.
Sub
Put puskligg and wine table.
Put cupcake dinner- -table.
Plan
Figure 7: Examples of cooperative behaviors introduced by meta-plan Guided by meta-plan, agents show clear work and task allocation, thereby improving cooperative efficiency.
and CoBLA agents. These results consistently highlight the superiority of CaPo in enhancing multi- agent cooperation. We provide more analysis on heterogeneous agents in Appendix A.2.
Efficiency comparison. To demonstrate the cooperation efficiency, we compare the transport rates of CaPo and CoELA with different time steps on TDW-MAT. As shown in Fig. 5, with the same time step budget and GPT-3.5 agents, CaPo consistently outperforms CoELA across various time steps, indicating its superiority to coordinate agents effectively. The improvement is particularly clear in scenarios of small time steps. For example, given 1,000 time steps, CaPo doubles the transport rate of CoELA by improving 10% to 23%. This shows that with limited time resources, a cooperative meta-plan can significantly improve cooperation efficiency.
Qualitative Analysis. Here we investigate the agents' behavior in CaPo with GPT-4 on the C- WAH task. In the meta-plan generation phase, as shown in Fig. 3, agents ask questions, provide feedback, and collaboratively refine the initial meta-plan. Moreover, in this phase, Fig. shows that with meta-plan as guidance, two agents, Alice and Bob, have clear work/labor allocation to complete tasks, thereby avoiding redundant steps and improving cooperation efficiency. For the progress-adaptive meta-plan and execution phase, Fig. 6 also shows that when agent Alice achieves progress, e.g., discovering three target objects, both agents will accordingly discuss to adapt the meta-plan, e.g., grasping wine and 1 cupcake by Alice. This ongoing adaptation of the meta-plan provides strategic, coherent, and timely guidance, facilitating efficient coordination among agents and ultimately enhancing multi-agent cooperation.
5.2 ABLATION STUDY
Effects of each component CaPo. Here we examine the effects of two key components: 1) meta-plan generation, which includes meta-plan initialization, evaluation, and optimization, and 2) the progress-adaptive meta-plan. To evaluate their impact, we first remove both components from CaPo, resulting in CaPoi. As shown in Table 3, CaPoz, which includes meta-plan initialization freezes the meta-plan during subsequent procedures, improves upon CaPo; by approximately 1% across three metries, demonstrating the value of meta-plan initialization. Similarly, CaPog. which incorporates the full meta- meta-plan generation process, outperforms CaPo, by significant mar- gin, highlighting the benefits of meta-plan evaluation and optimization. Finally, CaPo achieves 7% improvement over CaPoy, showcasing the effectiveness of the progress- adaativ meta-plan. These results underscore the importance of each component in the CaPo framework.

---

Published as a conference paper at ICLR 2025
Method
Food (1)
Stufl (1) Avg. (1)
Method Symbolic Obs
Visual Obs (})
CaPo; (No MP No Pro. MP)
72
75
73
Calox1
106
CaPoz (MP Initialization No Pro. MP)
73
76
74
CaPox2
83
CaPos (MP Generation + No Pro. MP)
74
80
77
CaPox3
46
72
CaPo (MP Generation + Pro. MP)
85
CaPox4
45
74
Table 4:
Benefits of increasing agent numbers in our CaPo using GPT-4 the C-WAH task. Aver- age steps required to complete task reported.
Table 3: Effects of the components in CaPo using GPT-3.5 on TDW-MAT task. We report the transport Rate (TR, %). MP" denote 'Meta Plan" and Progress- Adaptive Meta Plan", respectively.
Effeets agent number. Table 4 investigates the effects of agent number in CaPo using GPT- 4 on the C-WAH task, where "CaPo x C" denotes using C GPT-4 agents. We can observe that increasing agent number to significantly reduces the overall time step number required to complete tasks. This improvement also shows the effectiveness of our CaPo on multiple agent cooperation. However, increasing the number agents to four results in only minor or degraded improvements. This is because for simple tasks, agents are much and suffer from highly-frequet agent dispatch, leading to inferior collaboration efficiency. For instance, setting up dining table does not require four waiters, as maximum of two agents is sufficient.
Progress in meta-plan adaptation. Fig. in Sec. 4.3 shows clear task progress examples: 1) discovering target object and 2) completing a subtask, both of which can trigger agents adapt the meta-plan task progress, Such progress is crucial, as agents need continually refine the meta-plan to complete tasks ef- ficiently and maximize cooperation. Conversely, actions without significant progress, such as entering a new room, do not prompt agents to adjust the current (progress- adaptive) meta-plan. This is because would be unnec- essary, and updating the meta-plan involves communica- tion, which incurs additional time overhead.
Discussion Rounds
>3
their latest
36.4
31.2
32.4
25.6
20.1
20.9
12.3
Food tasks
Stuff tasks
Convergence analysis of agent discussion. Here in- vestigate the convergence of agent discussions, focusing on how many rounds are required for agents reach con- sensus on the meta-plan. In the TDW-MAT environment, limit the maximum number of discussion rounds- (details in Appendix A.6). As shown in Fig. 8, agents reach consensus within three rounds in most cases, with 78.9% achieving consensus the "Stuff" tasks. By capping the discussion rounds, CaPo achieves a balance between discussion effectiveness and the budget, avoiding unnecessary or pro- longed deliberations. Additionally, we provided an communication analysis in Appendix A.7, which demonstrate CaPo achieves good balance between cooperation efficiency and communication cost.
Figure 8: Percentage (%) of dis- cussion rounds needed for agents to reach consensus on meta-plan, based on results from TDW-MAT.
as the discussion budget-to three
6 CONCLUSION AND LIMITATIONS
In this work, we introduce Cooperative Plan Optimization (CaPo) improve the cooperation effi- ciency of LLM--drven embodied agents. CaPo creates strategic and coherent meta-plan through multi-turn agent discussions before any actions are executed. This meta-plan acts as an action guide to effectively coordinate multiple agents in task completion. During execution, agents dynamically adapt the meta-plan based on task progress, ensuring continued effectiveness in coordination. Experimental results TDW-MAT and C-WAH demonstrate that CaPo achieves higher task com- pletion rates and efficiency compared to state-of-the-art methods.
Potential Limitation: While CaPo significantly improves multi-agent cooperation efficiency, it has certain limitations. Specifically, it relies heavily on LLMs for reasoning and planning during meta-plan generation and adaptation. Additionally, CaPo employs pre-defined communication protocol for creating and updating the meta-plan, which may restrict its flexibility in dynamic or heterogeneous environments. Addressing these limitations will be focus of our future work.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was partially funded by Elekta Oncology Systems AB and a RVO public-private Partner- ship grant (PPS2102).
REFERENCES
Dhruv Batra, Angel X. Chang. Sonia Chernova, Andrew J. Davison, Jia Deng, Vladlen Koltun, Sergey Levine, Jitendra Malik, Igor Mordatch, Roozbeh Mottaghi, Manolis Savva, and Hao Su. Rearrangement: A challenge for embodied ai, 2020. URL httpps///aaxiii.//e/eeesessmmast 2011. 01975.
Daniel S Bernstein, Robert Givan, Neil Immerman, and Shlomo Zilberstein. The complexity of decentralized control of markov decision processes. Mathematics of operations research, 27(4): 119-840, 2002.
Shaofei Cai, Bowei Zhang, Zihao Wang, Xiaojian Ma, Anji Liu, and Yitao Liang. Groot: Learning to follow instructions by watching gameplay videos. arXiv preprint arXiv:2310.08235, 2023.
Karl Cobbe, Oleg Klimov, Chris Hesse, Tachoon Kim, and John Schulman. Quantifying generaliza- tion in reinforcement learning. In International conference on machine learning, pp. 1282-1289. PMLR, 2019.
Andrea Dittadi, Frederik Träuble, Manuel Wüthrich, Felix Widmaier, Peter Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf. and Stefan Bauer. The role of pre- trained representations for the ood generalization of reinforcement learning agents. arXiv preprint arXiv:2107.05686. 2021.
Yuging Du, Olivia Watkins, Zihan Wang. Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. Guiding pretraining in reinforcement learning with large language models. In International Conference on Machine Learning, pp. 8657-8677. PMLR, 2023.
Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, et al. Threedworkd: A platform for interactive multi-modal physical simulation. arXiv preprint arXiv:2007.04994 2020.
Chuang Gan, Siyuan Zhou, Jeremy Schwartz, Seth Alter, Abhishek Bhandwaldar, Dan Gutfreund, Daniel LK Yamins, James J DiCarlo, Josh McDermott, Antonio Torralba, al. The threedworld transport challenge: A visually guided task-and and--ond-motion planning benchmark towards physically realistic embodied ai. In 2022 International conference on robotics and automation (ICRA), pp- 8847-8854. IEEF, 2022.
Xudong Guo, Kaixuan Huang, Jiale Liu, Wenhui Fan, Natalia Vélez, Qingyun Wu, Huazheng Wang, Thomas L. Griffiths, and Mengdi Wang- Embodied Ilm agents learn to cooperate in organized teams. arXiv preprint arXiv:2403. 12482, 2024.
Shibo Hao, Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305. 2023.
Kaiming He,, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-enn. In Proceedings the IEEE international conference on computer vision, pp. 2961-2969, 2017.
Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang. Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi- agent collaborative framework. arXiv preprint urXiv:2308. 00352, 2023.
Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models zero-shot planners: Extracting actionable knowledge for embodied agents. International Conference on Machine Learning, pp. 9118-911977 PMLR, 2022a.

---

Published as a conference paper at ICLR 2025
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05660 2022b.
Jiechuan Jiang and Zongqing Lu. Learning attentional communication for multi-agent cooperation. Advances in neural information processing systems, 31, 2018.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199 222133, 2022.
Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín- Martín, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. Behavior- A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In Conference on Robot Learning, pp. 80-93. PMLR, 2023a.
Huao Li, Yu Quan Chong, Simon Stepputtis, Joseph Camphell, Dana Hughes, Michael Lewis, and Katia Sycara. Theory of mind for multi-agent collaboration via large language models. arXiv preprint vrXiv:2310.107000 2023b.
lou-Jen Liu, Unnat Jain, Raymond A Yeh, and Alexander Schwing. Cooperative exploration for multi-agent deep reinforcement learning. In International conference on machine learning, pp- 0826-6836. PMLR, 2021.
Xiao Liu, Hao Yu. Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating 1lms as agents. arXiv preprint arXiv:2308. 036888 2023a.
Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. Dynamic llm-agent network: An Imm- agent collaboration framework with agent team optimization. arXiv preprint arXiv:3310.02170, 2023b.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36, 2024.
Oscar Mañas, Pietro Astolfi, Melissa Hall, Candace Ross, Jack Urbanek, Adina Williams, Aish-
warya Agrawal, Adriana Romero-Soriee Soriano, and Michal Drozdzal. Improving text-to-image sistency via automatic prompt optimization. arXiv preprint arXiv:2403. 17804, 2024.
Zhao Mandi, Shreeya Jain, and Shuran Song. Roco: Dialectic multi-robot collaboration with large language models. arXiv preprint arXiv:2307.047386 2023.
Yohei Nakajima. Babyaig. https://githie
htw.a//aaaaaaaaaaaaaaaaaa...a... babyagi, 2023.
jima/
OpenAI. Gpt-4 technical report, 2024.
Aishwarya Padmakumar, Jesse Thomason, Ayush Shrivastava, Patrick Lange, Anjali Narayan-Chen Spandana Gella, Robinson Piramuthu, Gokhan Tur, and Dilek Hakkani-Tur. Teach: Task-driven embodied agents that chat. In Proceedings of the AAA! Conference on Artificial Intelligence, volume 36, pp. 2017-2025, 2022.
Xavier Puig, Tianmin Shu, Shuang Li, Zilin Wang. Yuan-Hong Liao, Joshua Tenenbaum, Sanja Fidler, and Antonio Torralba. Watch-and-hell A challenge for social perception and human-ai collaboration. arXiv preprint arXiv:2010.0998 2020.
Xavier Puig, Tianmin Shu, Shuang Li, Zilin Wang. Yuan-Hong Liao, Joshua Tenenbaum, Sanja Fidler, and Antonio Torralba. Watch-and-helpp A challenge for social perception and human-ai collaboration. In International Conference on Learning Representations, 2021.
Reworkd. Agentgpt. https: /ttttuu/ com/ rwworkd/
Agent 2023.
Toran Bruce Richards and et al. Auto-gpt: An autonomous gpt-4 experiment. ht githhb..
com/Si f Gravi tas /AttoGTT, 2021.
12

---

Published as a conference paper at ICLR 2025
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugging- gpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36, 2024.
Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVE conference on computer vision and patlern recognition, pp. 10740-10749, 2020.
Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Lim-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2998-3009, 2023.
Matthijs TJ Spaan, Geoffrey J Gordon, and Nikos Vlassis. Decentralized planning under uncertainty for teams of communicating agents. In Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems, pp. 249-24922 256, 2006.
J Lukas Thürmer, Frank Wieber, and Peter M Gollwitzer. Planning and performance in small groups: Collective implementation intentions enhance group goal striving. Frontiers in Psychology, 8:603, 2017.
Alejandro Torreno, Eva Onaindia, Antonín Komenda, and Michal Štolba. Cooperative multi-agent planning: A survey, ACM Computing Surveys (CSUR), 50(6):1-32, 2017.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint urXiv:2307.09288 2023.
Tuomela. Collective goals and cooperation. In Discourse, Interaction and Communication: Pro- ceedings of the Fourth International Colloquium Cognitive Science (ICCS-95), pp. 121-139. Springer, 1998.
Guanzhi Wang, Yugi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXivv2305. 16291 2023a.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 20236. URL https://aaxiii org/ abss/230 11.335.16299 16291.
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):1 2024.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv: 11171, 2022.
Yuanfei Wang, Fangwei Zhong, Jing Xu, and Yizhou Wang. Tom2c: Target-oriented multi-agent communication and cooperation with theory of mind. arXiv preprint arXiv:21/1.09/ 2021.
Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unlcashing the emergent cognitive synergy in large language models; A task-solving agent through multi-persong self-collaboration. arXiv preprint arXiv:2307.05300 2023c.
Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint 232i:2300.00/560 2023d.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi. Quoe V Le, Denny Zhou, et al. Chain-of -h-hn-hf- prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-2444 -4-337,, 2022.
13

---

Published as a conference paper at ICLR 2025
Yue Wu, So Yeon Min, Yonatan Bisk, Ruslan Salakhutdinov, Amos Azaria, Yuanzhi Li, Tom Mitchell, and Shrimai Prabhumoye. Plan, eliminate, and track-language models are good teachers for embodied agents. arXiv preprint wrxhv:2305.02412 2023a.
Zhenyu Wu, Ziwei Wang, Xiuwei Xu, Jiwen Lu, and Haibin Yan. Embodied task planning with large language models. arXiv preprint arXiv:2307.01844 2023b.
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoe V Le. Denny Zhou, and Xinyun Chen. Large language models optimizers. arXiv preprint arXiv:23099034400 2023.
Ceyao Zhang, Kaijie Yang, Siyi Hu, Zihao Wang. Guanghe Li, Yihang Sun, Cheng Zhang, Zhaowei Zhang, Anji Liu, Song-Chun Zhu, al. Proagent: Building proactive cooperative ai with large language models. arXiv preprint arXiv:2308. 1339, 2023a.
Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B Tenenbaum, Tian- min Shu, and Chuang Gan. Building cooperative embodied agents modularly with large language models. arXiv preprint arXiv:2307. 220.:2337.002485, 2023b.
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuur- mans, Claire Cui, Olivier Bousquet, Quoe Le, et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625 2022.
Gengze Zhou, Yicong Hong, and Qi Wu. Navgpt: Explicit reasoning in vision-ami -nd---nd--dd and--adddddaa language naviga- tion with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 7641-7649, 2024.
Dongsheng Zhu, Xunzhu Tang, Weidong Han, Jinghui Lu, Yukun Zhao, Guoliang Xing, Junfeng Wang, and Dawei Yin. Vislinginstruct: Elevating zero-shot learning in muuli--odee language models with autonomous instruction optimization. arXiv preprint urXiv:2402.07398, 2024.
Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, al. Ghost in the minecraft: Generally capable agents for open- world enviroments via large language models with bas- knowledge and memory. arXiv preprint arXiv:2305. 17144, 2023.
14

---

Published as a conference paper at ICLR 2025
Appendix
A ADDITIONAL EXPERIMENTS AND DISCUSSION
A.1 BASELINE MODELS AND OTHER ENVIRONMENTS
We adopt two types of methods as baseline, including classical agents and LLM-driven multi- agents. (1) The classical agents include MCTS-based Hierarchical Planner (MHP) (Puig et al., 2020) which is hierarchical planner originating from the original Watch-And-Help Challeng, and Rule-based Hierarchical Planner (RHP) (Gan et al.. 2022) derived from a strong baseline in the ThreeDWorld Transport Challenge. (2) LLM-driven agents consist of CoELA Zhang al. (2023b), ProAgent (Zhang et al., 2023a), and RoCo (Mandi et al., 2023). Cooperative Embodied Language Agent (CoELA) (Zhang et al., 2023b) can plan, communicate, and collaborate with other agents to complete long-howe ong-horizon tasks, but generate independent short-term plan for each agent. In addition, we also introduce two more baselines - ProAgent (Zhang et al., 2023a) and RoCo (Mandi et al., 2023), and implement them on TDW-MAT and C-WAH using source codes. These two baselines generate joint plans for cooperative agents, and introduce reflection loop or environment feedback for plan validation.
We conducted experiments on TDW-MAT and C-WAH, which are widely recognized benchmarks for embodied multi-agent cooperation. While other environments, such ALFRED (Shridhar al., 2020) and BEHAVIOR-IK (Li et al., 2023a), are notable in this domain, they do not support multi- agent cooperation simulation, and therefore, we did not include them in experiments. Similarly, works based on Minecraft (Wang et al., 2023b; Cai et al., 2023), such as (Wang et al., 2023c;d), focus settings that differ from ours, as our framework emphasizes decentralized cooperation with partial observation. Consequently, Minecraft- baseed environments were also excluded from our experiments.
A.2 EXPERIMENTS HETEROGENEOUS AGENTS' COOPERATION
CaPo introduces pre-defined communication protocol to create and update the meta-plan, which facilitate the multi colperatoorr efficiency. However, it is also very interesting to explore whether CaPo could improve cooperation efficiency when cooperating with other decentralized agents, e.g, RHP and CoELA. Specifically, when cooperating with other agents, the CaPo agent independently creates and update the meta-plan due the lack of multi-round discussion. We conduct experiments on the TDW-MAT tasks using oracle perception, and the results are shown in Table 5.
Runs
Food (f)
Stuff (t)
Avg. (f)
RHP
0.52
0.49
0.50
RHP+RHPt
0.76 (+33%)
0.74(+34%)
0.75(433%)
RHP+COELAt
0.85 (+40%)
0.77 (+35%)
0.81 (+37%)
RHP+CaPo
0.85 (+38%)
0.80 (+37%)
0.82 (+37%)
CaPo+CoELA
0.86 (+36%)
0.83 (+35%)
0.84 (+35%)
Table 5: Results on heterogeneous agents' cooperation. Here we report transport rate (efficiency improvement) on TDW-MAT tasks using oracle perception and GPT-4. denotes results from the CoELA paper. The reported may fluctuate even for same transport rate, due slight variations of the performance of baseline agent.
As shown in the above table, when CaPo cooperates with RHP agents, it achieves higher transport rate compared to both a team of two MHP agents and a combination of MHP and CoELA agents. We attribute this improvement the independently developed meta-plan, which guides the CaPo agent to complete tasks more efficiently and implicitly enhances multi-agent cooperation efficiency.
Interestingly, when our CaPo cooperates with the more advanced agent CoELA, it further improves the transport rate. This highlights the potential of deploying CaPo to collaborate with other decen- tralized agents.
15

---

Published as a conference paper at ICLR 2025
A.3 REPRODUCIBILITY OF RESULTS
LLM-driven reasoning and planning tend to be stochastic, requiring multiple runs to assess stability. To verify the stability and reproducibility of method, we conducted three on TDW-MAT with oracle perception and GPT-3.5 agents. As shown in Table 6, the results exhibit minor variance across runs, demonstrating the stability and reproducibility of our method.
Runs
Food (1)
Stuff (t)
Avg. (1)
0.85
0.84
0.84
2
0.84
0.81
0.82
3
0.85
0.83
0.84
Average
0.84 (0.006)
0.82 (0.015) 0.83 (0.012)
Table 6: Transport Rate (TR) comparison on the TDW-MAT task using GPT-3.5 and oracle perception. We perform 3 runs with random seeds and report mean and variance.
A.4 RESULTS ON ADDITIONAL ENVIRONMENTS
To test the generalization and effectiveness of our model on new environments, we add one more experiments on RoCoBench (Mandi et al., 2023), which is multi-robot collaboration environment. We choose three tasks exhibiting asymmetric observation, which are more relevant to decentralized, partia-observation settings, and the results are shown in the Table Our CaPo framework achieves a higher success rate with fixed steps and requires few steps to complete the task. This further highlights the effectiveness and generalization of our model on embodied multi-agent cooperation.
Sweep Floor
Pack Grocery
Move Rope
Success rate
Step
Success rate
Step
Success rate
Step
RoCo
0.95 + 0.05
7.1
0.44 + 0.06
9.9
0.65 + 0.11 2.5
CaPo (ours)
0.98 + 0.13
6.5
0.63 0.11
7.6
0.74 + 0.17
2.2
Efficiency comparison on RocoBench (Mandi et al., 2023). We report two metrics, success rate the task, and average steps required to complete the task.
Table 7:
A.5 NUMBER OF DISCUSSION ROUND
To generate or update the meta plan, all agents involved in multi-round discussion. We set the discussion budget to 3 rounds to balance the consensus rate and overall efficiency. As shown in the Table A.5, with a 3-round limit, the failure rate decreases to 21.1 %, and overall efficiency improves to 84%. Extending the discussion budget beyond 3 rounds offers only marginal benefits in reducing failed cases (e.g., reduction to 17.8% with 4 rounds) while providing no additional efficiency gains, as efficiency stabilizes at 84%. These results demonstrate that a 3-round discussion budget is a reasonable choice, effectively balancing communication costs and task performance.
Maximal Round
2
4
Rate of failed cases (%)
79.1
53.5
21.1
17.8
15.5
Transport rate (%)
82
84
84
83
Table 8: Failed case rate and transport rate under different maximal rounds of discussion.
A.6 COMMUNICATION COST
For communication cost and efficiency, we want to discuss them from several aspects. Firstly, the task cost mainly includes agent moving cost and communication cost. By comparison, moving cost
16

---

Published as a conference paper at ICLR 2025
often much higher than communication cost, since current robots lack the ability of humans, and their movement is costly especially in complex and large-scale environments like urban areas; while the latter only needs LLM's inference, and massage sending and receiving, and is indeed much easier and efficient. For example, moving from one room to another can often require much more time than LLM's inference.
Moreover, to reduce communication cost, our CaPo adopts two strategies. a) agent's progress- adaptive planning module will evaluate whether meta plan is suitable for current task progress: yes, then all agents will continue their subtasks, and will not communicate, reducing uncommunica- tion costs; b) with the meta plan as a reference, few round communication is sufficient to arrive at a new plan, since task progress is often smoothly achieved and thus adaptation the meta plan need slightly small changes. Indeed, we set the maximum round of communication as 3 for each discus- sion. So these strategies differ from CoELA which needs communication at each iteration, while ours does not.
Method
Rounds Meta-plan Updates
Character
Frames Cost Rounds Discussion per Meta-plan
Time Cost per Meta-plan Update
Time Cost per Task
Cost per Meta-plan
from Communication
Food tasks
14
1,757 chars
frames
2.4 rounds
20.4 see
26.2 min 25.6 min
Stuff tasks
1,462 chars
51 frames
2.1 rounds
22.7 sec
Table 9: Communication cost analysis on the TDW-MAT dataset with oracle perception.
For average rounds of meta-plan updates and average steps for each meta plan discussion, we report the statistics Table 9. Regarding Food tasks on TDW, agents update the meta-plan for average of 14 times per task, with each discussion requiring 2.4 steps and a total L,757 characters for sending (i.e.. 4 frames since frame can send 500 characters in CoELA). While this incurs some time cost, the long-term coherent meta-plan results in a substantial transport rate gain. This demonstrates that creating coherent meta-plan through communication is significantly relying solely on embodied actions.
time-efficient than
A.7 COLLABORATING WITH HUMAN
To evaluate human-agent cooperation, we follow the methodology of CoELA and conduct experi- ments on the C- C-WAH tasks, where the agent Bob (meta-plan evaluator) is controlled by real human participants. We adopt average steps as an efficiency metric and also ask three participants to rate their teammate agent (i.e., CaPo or CoELA) on a 7-point Likert Scale based on three criteria: com- munication effectiveness, helpfulness, and trust. The results are detailed in Table A.7.
Method
Average Steps
Comm Effectiveness
Helpfulness
Trust
CoELA
47 +
5.8 0.3
5.5 0.4
6.0 + 0.1/7
CaPo (ours)
44
6.3 + 0.1
6.1 0.3
6.3 + 0.2
Table 10: Comparison of CoELA and CaPo on Human Collaboration Metrics.
Efficiency (Average steps). Humans collaborating with CaPo generally achieve higher efficiency, completing tasks with fewer steps, i.e., 44 steps. This is because both agents and humans can avoid redundant actions through the guidance of the proposed meta-plan.
Human Preferences. (1) Communication effectiveness represents the effectiveness of communi- cation between agent and human participants, c.g., whether the agent understands the message and provides useful information. In general, CaPo demonstrates clearer and more reliable communi- cation than CoELA (6.3 vs. 5.8). This improvement is attributed to CaPo's multi-round discussion mechanism, which creates a long-term, coherent meta-plan, reducing the need for frequent commu- nication. In contrast, in CoELA, agents and humans have to exchange information more frequently, due to the lack of a coherent plan. Finally, agents didn't show verbose behavior when making constrained the LLM to generate concise plans and messages in
plans (as shown in Figure 3), prompting. (2) Helpfulness indicates whether the agent helps human participants complete the task faster. By multi-round discussion and creating coherent meta-plans, CaPo could help both agent
17

---

Published as a conference paper at ICLR 2025
and human participants reduce redundant actions in the embodied environments, e.g., aimless exploration. (3) Trust measures how human participants trust their agent teammate. Humans show stronger preference for trusting CaPo (6.3 vs. 6.0) during the cooperation. This is because CaPo involves humans in the planning process, seeking their input during multi-round discussions to col- laboratively create the meta-plan. In contrast, CoELA agents more independently and sometimes fail to respond to human messages, which weakens trust.
Through human-agent cooperation experiments, we demonstrate that CaPo enhances communica- tion clarity, actively supports human decision-making, and fosters trust by engaging humans in the planning process. These advantages position CaPo more favorable teammate for human collab- orators, underscoring its potential for real-world cooperative applications.
B ADDITIONAL ENVIRONMENT DETAILS
We evaluate method and all bascline methods in two simulated environments: ThreeDWorld Multi-Agent Transport (TDW-MAT) and Communicative Watch-And-Hell (C-WAH), We follow CoELA Zhang al. (2023b) and list detailed introductions these environments below.
B. THREEDWORLD MULTI-AGENT TRANSPORT
Tasks. TDW-MAT consists of two types of tasks, food-transporting task and stuff-transporting task. The food-transporting task has types targets (apple, banana, orange, bread, loaf bread, and burger) and 3 containers (bowl, plate, and tea tray). In contrast, the tuff-transporting task has different types of turgets(calculator, mouse, pen, lighter, purse, and iPhone) and containers (plastic basket, wood basket, and wicker basket). In each task, there are 10 target objects and to 5 containers in total. Additionally, there are 4 types of rooms: living room, office, kitchen, and bedroom, and objects are placed in these rooms consistent with common sense. The agents are tasked with transporting as many target objects as possible to the goal position using containers as tools, Each container can carry up three objects, while without container, an agent can transport only two objects at a time. The agents must transport many target objects possible within 3000 frames.
Observation Space The embodied agent receives variety of observations, with the primary being an egocentric RGB image and a depth image. Additionally, there are several auxiliary obser- vations. The observation space includes:
RGB image: This is an egocentric image captured by forward-facing camera, with resolution of 512 x 512 and field of view of 90 degrees.
Depth image: This image shares the same camera intrinsic parameters the RGB image.
Oracle Perception (optional): An image where each object ID is represented by distinct color, using the same camera intrinsic parameters as the RGB image.
Agent position and rotation: The position and rotation of the agent within the simulation environment.
Messages: Communications sent by all agents.
Held objects: Information about the objects currently held by the agent.
Opponent held objects: Information about objects held by another agent, the agent is within view.
Action Space In TDW-MAT, agents can perform 7 distinct types of actions to interact with the environment or communicate with each other. Each action spans multiple frames, and the detailed action space is outlined below:
Move forward: The agent advances by 0.5m.
Turn left: The agent rotates left by 15 degrees.
Turn right: The agent rotates right by 15 degrees.
18

---

Prepare afternoon tea Published as a conference paper at ICLR 2025
Grasp: The agent grasps object, successfully performing this action only when in close proximity to the object. The object can be either a target or a container.
Put In: The agent places a target into container, an action that is possible only when the agent is holding a target in one hand and a container in the other.
Drop: The agent releases the objects held in hand.
Send message: The agent sends message to other agents, with limit of 500 characters per frame.
B.2 COMMUNICATIVE WATCH-AND-HER HELP
Communicative Watch-And-Heip (C-WAH) builds upon the Watch-And-Help challenge Puig et al. (2021) by incorporating the ability for agents to send messages to one another. Sending messages, like other actions, consumes one timestep and is subject maximum length constraint.
Task Name
Predicate Set
ONfpuddingg.offtcctabbe
ON(apple,coffeetall (((wine.ccoffeetable)
(fjuuic,.cffeetabll))
Wash dishes
NNplaat,,ishhwasine
(Nffork.dishwasher)
Prepare meal
dinuuddingdinnettablle dinnertable), A'oappple,immnetabble,
ON(juice (inneree.iinnertublee).. (iwweeeddimee
Put groceries
N((upcake..rride IN(pamcake.friddee
IN(poundcake, fridge), IN(pudding.frion fridge), frrpppe..rrddge) Iijuice,fridge). Ir(wiee,fridge)
Set up a dinner table
ON(plate,dimneerablee), oN(fork.dinnertable)
Table 11: Task description in C-WAH. The tasks are divided into five types, each containing several predicates.
Tasks The Communicative Watch-And-Help (C-WAH) framework includes five types tasks: Prepare afternoon tea, Wash dishes, Prepare meal, Put groceries, and Set up dinner table. These tasks encompass various household activities, each consisting of several subtasks described by predicates in the "ON/IN(x, y)" format, such as "Put ON/N y". Detailed descriptions of the tasks are provided in Table 11.
The primary objective is to complete all given subtasks within 250 timesteps, with each task taining between to 5 subtasks.
Observation Space C-WAH offers two modes of observation: Symbolic Observation and Visual Observation.
In Symbolic Observation, following the original Watch- And-Help challenge, an agent can access comprehensive object information within the same room, including location, status, name, and rela- tionships.
In Visual Observation, agents receive egocentric RGB and depth images along with auxiliary obser- vations. The detailed observation space includes:
RGB image: An egocentric image from forward facing camera, with resolution of 256 x 512 and field of view of 60 degrees.
Depth image: An image with the camera intrinsic parameters the RGB image.
19

---

Published as a conference paper at ICLR 2025
Oracle Perception: An image where each object ID is mapped to color, sharing the same camera intrinsic parameters as the RGB image.
Agent position: The agent's position within the simulation world.
Messages: Communications sent by all agents.
Held objects: Information about the objects eurrently held by the agent.
Opponent held objects: Information about objects held by another agent, if visible.
Action Space The action space in C-WAH closely resembles that of the original Watch-And-Hor
Challenge, with the addition of the send message action. The detailed action space includes:
Walk towards: Move towards object in the same towards specific room.
Turn left: Rotate left by 30 degrees.
Turn right: Rotate right by 30 degrees.
Grasp: Grasp an object, which can be successfully performed only when the agent is close to the object.
Open: Open closed container, performable only when the agent is near the container.
Close: Close an open container, performable only when the agent is near the container.
Put: Place held objects into an open container or onto a surface, performable only when
the agent the target position.
Send message: Communicate with other agents, with limit of 500 characters per message.
20

---

Published as a conference paper at ICLR 2025
C PROMPT TEMPLATE
We list the prompts template for meta plan initialization, communication module of Alice, commu- nication module Bob, cooperative planning module, and the plan parsing module as follows.
am Alice. My teammate Bob and want transport as many target objects possible to the bed with the help of containers within 3000 steps. can hold two things at time, and they can be objects or containers. can grasp containers and put objects into them hold more objects time.
Assume that you are an expert plan outline designer. Given our shared goal, please help me generate global meta plan for and Bob during task exccution, guiding and Bob achieve the goal collaboratively as soon as possible, Note that a container can contain three objects, and will be lost once transported to the bed. can only put objects into the container hold after grasping it. All objects are denoted as <name> (id), such as <table> (712). Actions several steps finish. may costly to go another room transport to the bed, use these actions sparingly.
The generated meta plan
following requirements:
1.There allowed actions you
construct the meta plan. "go to':
specified room. 2) "explore': explore room for underlying target objects. 3) "go grasp": : go to grasp a specified target object. 4) 'put": Place an object into a specified container. 5) 'transport":
Transport holding objects or containers to the bed and drop them on the bed.
2.The meta plan should be concise, brief, and reliable.
3.e meta plan must be structured strietly in the three-step format: {Action Plan: Step I: Alice xxx, Bob xxx; Step 2: Alice xxx, Bob xxx; Step Alice xxx, Bob xxx]. Here, 'xx'' represents one or multiple allowed actions. The actions in Step are of the highest execution priority, while those in Step 2 and Step 3 are of medium and lowest execution priority.
4. 4.ehe meta plan should reasonably arrange the division of action between Alice and Bob in order to achieve the goal as soon as possible.
Here is an example for you:
[Goal: [Transport 3 pens, lighter, and iPods the bed.]
Meta plan: [Step I: Alice explores the current room. Bob explores current room. Step 2: If any target objects are found, Alice and Bob go grasp objects, put them into containers, and transport them to the bed.
the remaining rooms and explores Bob goes to one the remaining rooms and explores it]}
Step 3: Alice goes
Goal: SGOALS
Given the above goal, think step by step, and generate meta plan:
Figure 9: Prompts for LLM in generating meta plan.
21

---

Published as a conference paper at ICLR 2025
Alice. My teammate Bob want transport as many target objects as possible to the bed with the help of containers within 3000 steps. can hold two things at a time, and they can be objects containers. I can grasp containers and put objects into them to hold more objects time.
Assume that you are excellent leader for coordinating the task. Given shared goal, plan, dialogue history, latest progress, and my previous actions., please help generate a message sent Bob, order share my progress and inquire the opinion of Bob about the meta plan. Note that container can contain three objects, and will be lost once transported to the bed. I can only put objects into the container I hold after grasping it. All objects denoted <name> (id), such <table> (712). Actions take several steps to finish. may costly to go to another room or transport to the bed, use these actions sparingly.
The generated message should strictly meet following requirements:
The message has to be concise, reliable, and helpful for ussisting Bob and me to make an efficient and consistent action plan, and transport as many objects to the bed as possible. Don't generate repetitive messages.
Here is an example of generated massage for you:
Example:
(Message: Hi Bob, exploring the <living room? (2000) and found <apple> (1242543) there. made action plan to guide complete the task efficiently. Do you have any suggestions for the action plan according to your latest progress?]
Following provided information for you:
Goal: 'SGOAL'S
Previous meta plan: ISPREVIOUS, PLAN
Dialogue history: ISDIALOGUE: HISTORY'S
Progress: ISPRORRESSSS
Teammate progress: /SOP:, PROGRESSS
Figure 10: Prompts for LLM in the communication module of the mentor agent, e.g., Alice.
22

---

Published as a conference paper at ICLR 2025
Bob. My teammate Alice and want transport many target objects as possible the bed with the help of containers within 3000 steps. can hold two things at a time, they can be objects or containers. can grasp containers and put objects into them to hold more objects time.
Assume that you are an excellent leader for coordinating the task. Given our shared goal, action plan, dialogue history, progress, and my previous actions, please help analyze the feasibility of the action plan proposed by Alice and generate message to send Alice. Note that container can contain three objects, and will lost once transported to the bed. can only put objects into the container I hold after grasping it. All objects ure denoted us <name> (id), such <tuble> (712). Actions take several steps to finish. It may be costly go amother room or transport to bed, use these actions spuringly.
The generated meta plan should strictly meet following requirements:
The message has to be concise, reliable, and helpful for assisting Bob and me to make an efficient and consistent action plan, and transport as many objects to the bed as possible. Don't generate repetitive messages.
2.The message must strictly be in the following format: [Main message: detailed opinion and suggestions of Bob about the action plan. Satisfaction level: Yes No, decide whether you satisfied with the current action plan.) You should consider the progress of both Alice and Bob in determining the satisfaction level providing suggestions
action plan.
3.1f you are not satisfied with the current action plan, please point out the reason and your suggestion on how modify the action plan in the message. You can suggest the next action for Alice the message achieve the goal as soon possible.
Here is an example of generated massage for you:
Example:
Message: Main message: Hi Alice, your proposed action plan looks great. However, considering that found <wood basket> (1870213) and <plasticl busket=(184372) in the (2000) I think should grasp swood basket> (1870213), suggest you go <office>(2000) to grasp <plastici, (184322). Please update the meta plan. Satisfaction level: No!
Following are provided information you:
Goal: 'SGOALSS
Previous meta plan: (SPREVIOUS) PLANKS
Dialogue history: (SDIALLOUER_ HISTORYSS
Progress: SSROGRRESSS
Teammate progress: /SOPP. PPPP_PRRSSSSSSS
Figure 11: Prompts for LLM in the communication module of the teammate agent, e.g., Bob.
23

---

Published as a conference paper at ICLR 2025
am Alice. My teammate Bob und want to transport us many target objects as possible to the bed with the help of containers within 3000 steps. hold two things time, and they can objects or containers. can grasp containers and put objects into them hold more objects at a time.
Assume that you are an expert plan outline designer. Given our shared goal, previous meta plan, dialogue history, latest progress, please help me refine the meta plan into a more comprehensive and efficient plan for Bob and me, in order to achieve the goal collaboratively soon possible. Note that container transported the bed. can only put objects into the container hold after grasping it, All objects are denoted <name> (id), such as <table> (712). Actions take several steps to finish. It may be costly go another room transport to the bed, use these actions sparingly.
contain three objects, and will lost once
The generated meta plan should strictly meet following requirements:
1.The plan should be brief, reliable, authentic, and consistent with the latest progress of Alice Bob. Don't make random and meaningless plans.
2.There allowed actions you
to': specified room. 2) 'explore': explore room for underlying target objects. 3) go grasp's go to grasp a specified target object. 4) 'put': Place an object into specified container. 5) 'transport:: Transport holding objects or containers the bed and drop them the bed.
construct the meta plan.
3.T plan be structured strietly in three-step format: (Action Plan: Step Alice xxx, xxx; Step 2: Alice xxx, Bob xxx; Step 3: Alice xxx, Bob xxx). Here, 'xox' represents one or multiple allowed actions. The actions in Step 1 are of the highest priority, while those in Step and Step are of medium and lowest priority, respectively.
4. 4.ehe meta plan should reasonably arrange the division of action between Alice and Bob in order to achieve the goal as soon as possible.
Following provided information for you:
Goal: 'SGOALSS
Previous meta plan: (SPREVIOUS_PPA PLANSS
Dialogue history: 'SDIALOGUE: HISTORY'S
Progress: SPROGRESSS
Teammate progress: 'SOPP. PROGRESS'S
Figure 12: Prompts for LLM in cooperative planning module to generate progress-adaptive meta plan.
24

---

Published as a conference paper at ICLR 2025
am 'SAGENT, NAMESS. My leammate 'SPPL_ NAME' want transport as many target objects as possible to the bed with the help of containers within 3000 steps. can hold two things at time, and they can be objects containers. I can grasp containers and put objects into them hold more objects time.
Assume that progress, and previous actions, please help choose the best available action achieve the goal as soon as possible. Note transported to the bed. I can only put objects into the container hold after grasping it. All objects are denoted <name> (id), such <table> (712). Actions take several steps to finish. may costly go another sparingly.
an expert decision maker. Given shared goal, action plan, my
container can contain three objects, and will lost once
transport to the bed, these actions
Goal: 'SGAALS
Meta plan: SMETAL
Dialogue history: (SDIALOGUE: HISTORY'S
Progress: ISPROGRESSS
Previous action: 'SPREVIOUS! ACTIONSS
Action list: 'SACTION: LISTS
Figure 13: Prompts for L.M the plan praising module.
25


Reviews:
[{"review_id": "gSZpgcsb2a", "reviewer": "Reviewer_8MR1", "summary": "The paper introduced Cooperative Plan Optimization into the CoELA framework to improve embodied multi-agent cooperation. A new meta-plan mechanism ensures agents can form and follow a long-term strategic and coherent plan for efficient coordination. Multi-turn discussions following pre-defined templates among agents are triggered whenever new progress is made to adapt the meta plan to progress. The experiments on two embodied multi-agent cooperation benchmarks demonstrate the proposed method achieves better performance than sota.", "questions": "Please see the weaknesses above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Introducing a meta-plan mechanism to ensure agents can form and follow a long-term strategic and coherent plan can improve the coordination's efficiency.\n\n- The paper is clearly written and well-organized.\n\n- The ablation study and qualitative analysis provides valuable insights.", "weaknesses": "- Since the paper reuses 1) a perception module, 2) a memory module, 3) a communication module, and 7) an execution module from the CoELA paper, it may be better to put the discussion of these modules into the preliminary sections rather than in the method section to make this paper's contribution more clear.\n\n- The contribution is limited since the only improvement is achieved by introducing a pre-defined communication protocol by prompting into an existing CoELA framework.\n\n- The paper uses the same formalization and experiments set up as prior work CoELA, which is not a bad thing. For the main results in Table 1, the column with RHP*, RHP, and LLAMA-2 CoELA is identical as reported in the CoELA paper but with no appropriate notation. In the meantime, only the CoELA baseline's results are changed to use GPT-3.5 backbones instead of the GPT-4 used in the original paper, which also causes a major performance drop (avg. 71 -> 52; 85->72) compared to the reported results in the original paper. Results with gpt-4 are reported as original in Table 2 and Appendix Table 5, where the performance gain is down to 4 points rather than 13 points. Why the results are organized as such needs further clarification.\n\n- One of the interesting settings of the CoELA paper is that they formalized communication as a step that also comes with a cost so the agents need to balance the cost and gain of communication. They make the findings that it's not effective yet for LLM-based agents to communicate spontaneously, which may be part of the motivation for this paper. In this paper, however, it's not specifically discussed whether this costly communication is still employed, and if so, some analysis around the communication cost and corresponding efficiency achieved may provide valuable insights for the community.\n\n- Is the progress-adaptive planning module always triggered whenever there is new progress? How many rounds of meta-plan updates are there on average and how many steps would be used to conduct this meta-plan discussion? Is the communication module also needed after the progress-adaptive planning module makes the update to share it with the others? What happens if two agents make new progress at the same time and all want to be the meta-plan designer? What will happen if agents can't reach a consensus? Will they just have no communication afterward or try to communicate again after how many steps or stuck at arguing every few steps?\n\n- Minor typos:\n  - L70: typo, 'sever' -> 'several'", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ucuud59IhY", "reviewer": "Reviewer_zu5a", "summary": "The paper introduces Cooperative Plan Optimization (CaPo) to improve cooperation among LLM-based embodied agents. Existing methods lack strategic planning, leading to inefficiency and failures in complex cooperative tasks. CaPo enhances cooperation through two phases: meta-plan generation and progress-adaptive execution. In the first phase, agents collaboratively analyze the task and create a strategic plan with detailed steps. In the second phase, agents execute and adjust the plan dynamically based on real-time progress, supported by ongoing multi-turn discussions. This progress-based adaptation prevents redundant actions and boosts overall task efficiency. Empirical experiments on the ThreeDworld Multi-Agent Transport and Communicative Watch-And-Help tasks demonstrate CaPo’s superiority in task completion and efficiency.", "questions": "1. Can the authors elaborate further on their contributions compared to CoELA? The current presentation does not sufficiently highlight the novelty of this work.\n2. Would the authors consider conducting additional experiments in different environments to better demonstrate the generalizability and effectiveness of their method?\n3. Is it possible to reduce the communication cost by replacing the broadcasting communication mechanism with a different network design?\n4. In Figure 8, the authors claim that consensus is typically reached within 3 rounds. However, over 20% of trials still fail to reach consensus after 3 rounds. Is setting the discussion budget to 3 rounds a reasonable choice given this observation?", "limitations": "", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The paper is well-written and easy to follow. The figures and illustrative examples help demonstrate the effectiveness of the proposed progress-adaptive meta-plan.\n2. The related work section is sufficiently comprehensive, providing a clear context for the study.\n3. Complex ablation studies on each module of CaPo, the effect of agent number, and convergence analysis.", "weaknesses": "1. This work appears to be a straightforward extension of CoELA, with minimal innovation beyond basic prompt engineering for the meta-plan module. The problem setting and agent framework largely mirror those of CoELA. For instance, as noted in Section 4.1, the CaPo agent comprises seven modules: 1) perception, 2) memory, 3) communication, 4) cooperative planning, 5) progress-adaptive planning, 6) plan parsing, and 7) execution. The CoELA agent already includes five of these, lacking only the cooperative planning and progress-adaptive planning modules, while these 2 modules can be easily implemented by prompting LLM.\n2. The authors limited their experiments to the TDW-MAT and C-WAH testbeds, which are already used in CoELA. To demonstrate the broader effectiveness of their method, it would be beneficial to include experiments in new and varied environments.\n3. The process of updating the meta-plan relies on broadcasting communication among agents, which incurs significant communication costs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NByxW1QWy8", "reviewer": "Reviewer_kpSn", "summary": "This paper introduces the Cooperative Plan Optimization (CaPo) framework, which aims to enhance the long-term planning cooperation efficiency among Large Language Model (LLM)-based embodied agents. CaPo includes two-phase: meta-plan generation and progress-adaptive meta-plan and execution. Experimental results on the TDW-MAT and C-WAH tasks demonstrate CaPo's significant improvement over state-of-the-art methods.", "questions": "In the ablation study, the paper mentions the impact of the number of agents, but the game appears to only contain two players, Alice and Bod. I'm confused about the number of agents. Please clarify how you tested different numbers of agents if the main experiments only used two.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper introduces a novel two-phase planning approach, Cooperative Plan Optimization (CaPo), which addresses the limitations of single-step planning in CoELA. This approach demonstrates its effectiveness through experiments on the TDW-MAT and C-WAH environments.", "weaknesses": "The main content of the article appears to be an extension of CoELA[1], enhancing its single-step planning approach by incorporating multi-turn discussions after the execution phase. Meanwhile, several works, for example, [2,3] have already demonstrated multiple rounds of discussions in various contexts, so the novelty and significance of this article's contribution seem somewhat lacking.\n\nAdditionally, I confess that I am not familiar with ThreeDworld Multi-Agent Transport (TDW-MAT) task, and the Communicative Watch-And-Help (C-WAH) task, so I am not quite clear on how to assess their complexity. However, for works [4,5,6] based on Minecraft, which typically evaluates dozens or even hundreds of tasks. I am cautiously skeptical about whether considering only 6 or 10 target tasks in food and stuff settings is sufficient to thoroughly validate the capabilities of the proposed framework. How the complexity of TDW-MAT and C-WAH tasks compares to Minecraft tasks? Whether you have plans to evaluate on a larger set of tasks in future work?\n\n[1] Zhang, H., Du, W., Shan, J., Zhou, Q., Du, Y., Tenenbaum, J.B., Shu, T. and Gan, C., 2023. Building cooperative embodied agents modularly with large language models. arXiv preprint arXiv:2307.02485.\n[2] Du, Y., Li, S., Torralba, A., Tenenbaum, J.B. and Mordatch, I., 2023. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325.\n[3] Wang, Z., Mao, S., Wu, W., Ge, T., Wei, F. and Ji, H., 2023. Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. arXiv preprint arXiv:2307.05300.\n[4] Wang, Z., Cai, S., Chen, G., Liu, A., Ma, X. and Liang, Y., 2023. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560.\n[5] Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L. and Anandkumar, A., 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291.\n[6] Cai, S., Zhang, B., Wang, Z., Ma, X., Liu, A. and Liang, Y., 2023. Groot: Learning to follow instructions by watching gameplay videos. arXiv preprint arXiv:2310.08235.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ubuilr1Iol", "reviewer": "Reviewer_Z2bg", "summary": "This paper introduces a method called CaPo, which uses LLM agents having multi-turn conversations with each other to create meta-plans (a collection of subtasks for solving the overarching problem that are assigned to each agent). Not only does it precompute these tasks, it dynamically adjusts them over time as agents perceive new information or interact with objects in ways that impact the preexisting plan. By testing on realistic simulators emulating embodied household tasks, the authors show the efficacy of the approach compared to conventional LLM approaches and classical symbolic planners.", "questions": "What is the standard error for the metrics shown in Table 1? \n\nWhat is the runtime of this method at each turn? Can it be run in real-time with humans and how does planner quality change as a function of time?\n\nNice to have - what are results from playing this method with humans compared to other baselines? What do humans prefer?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The authors did a nice job ablating the different modular components of their agent architecture. Oftentimes in the LLM agent literature plans are either made in a single-turn or an agent just refines what to do in its own mind, but introducing multi-turn group communication seems like an effective strategy if one has enough of a communication budget. The architecture itself seems highly reproducible, the benchmarks are challenging, and the impact of which LLM underlies communication is thoughtfully examined.", "weaknesses": "The experiments could benefit more from being evaluated on real humans collaborating with LLM agents on tasks such as Watch and Help. If the ultimate goal of this line of work is to build embodied agents that help humans, understanding how real humans complete the task differently with different agents is key. Extra evaluations could show the same efficiency metrics but also metrics indicating which agents human preferred playing with. If CaPo is very verbose when making plans, humans may not prefer interacting with it despite the efficiency gains compared to a method like CoELA. It would be interesting to dissect this. \n\nMoreover, an analysis of the runtime cost of CaPo would be nice to see. If a multi-round discussion is needed to reformulate a plan every time new information is discovered, how long does this take? How long can prompts end up being for the LLMs (how large does the memory grow as well) and does the duration of an episode/memory size impact planner quality?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)