Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments

Authors: siddharth  nayak
adelmo  morrison orozco
marina  ten have
vittal  thirumalai
jackson  zhang
darren  chen
aditya  kapoor
eric  robinson
karthik  gopalakrishnan
james  harrison
brian  ichter
anuj  mahajan
hamsa  balakrishnan
usaf-mit  ai accelerator
google  deepmind
  apple
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 04:54:54

Abstract: The ability of Language Models (LMs) to understand natural language makes them a powerful tool for parsing human instructions into task plans for autonomous robots. Unlike traditional planning methods that rely on domain-specific knowledge and handcrafted rules, LMs generalize from diverse data and adapt to various tasks with minimal tuning, acting as a compressed knowledge base. However, LMs in their standard form face challenges with long-horizon tasks, particularly in partially observable multi-agent settings. We propose an LM-based Long-Horizon Planner for Multi-Agent Robotics (LLaMAR), a cognitive architecture for planning that achieves state-of-the-art results in long-horizon tasks within partially observable environments. LLaMAR employs a plan-act-correct-verify framework, allowing self-correction from action execution feedback without relying on oracles or simulators. Additionally, we present MAP-THOR, a comprehensive test suite encompassing household tasks of varying complexity within the AI2-THOR environment. Experiments show that LLaMAR achieves a 30\% higher success rate than other state-of-the-art LM-based multi-agent planners in MAP-THOR and Search \& Rescue tasks. Code can be found at [https://github.com/nsidn98/LLaMAR](https://github.com/nsidn98/LLaMAR)

Main paper: 
 introduction: 
 Creating embodied agents that assist humans in real-life scenarios is a significant challenge, especially when multiple agents are involved and humans communicate their intentions using natural language. Recent works [1, 2, 3, 4, 5, 6, 7, 8, 9] have shown that Language Models (LMs) 1 can effectively use language instructions to develop plans for robots. However, most studies focus on single-agent long-horizon task planning. Naïve extensions of single-agent planning algorithms to multi-agent settings often fail due to environment non-stationarity, where the policies of other agents-modeled as a part of the environment-are continuously changing [10, 11] . Such failures lead to suboptimal performance as agents struggle to anticipate and adapt to the actions of others. We therefore formulate a centralized process in which decisions are made simultaneously for all agents based on their (partial) observations, similar to the centralized multi-agent system framework (CMAS) proposed in [12] . Leveraging the ability of pre-trained LMs to generalize across diverse tasks, we aim to use LMs for long-horizon embodied multi-agent task planning.The key insight of our work is that integrating a plan-act-correct-verify framework with LMs enables a robust and adaptive approach to multi-agent task planning in dynamic, partially observable environments that allows agents to: (1) plan subtasks required to complete the task, (2) select highlevel actions for each agent to complete the proposed subtasks, (3) identify and correct failures after high-level action execution, and (4) self-verify subtask completion based on high-level action LMs for Multi-Agent Planning: Xu et al. [50] use LLMs in multi-agent games, while CoNavGPT [19] creates global plans for two robots in an embodied environment. RoCo [20] and CoELA [18] assign separate LMs to each agent for decentralized action prediction, allowing natural language communication between agents. However, RoCo and CoNavGPT require detailed environment information for planning, and CoELA's action space is filtered by an oracle. Relying on privileged information from an oracle is impractical in real-world applications. By contrast, our work focuses on free-form action generation and handles tasks with more ambiguous descriptions. Prior work [12] compare centralized (CMAS) and decentralized (DMAS) planning frameworks, showing that centralized planners perform better, though their experiments are in simple, known environments with limited number of agents. Two-Step [17] decomposes goals for main and helper agents, using PDDL planners for high-level actions. SmartLLM [15] uses multiple LLM modules for subtask decomposition, multi-robot group formation and task allocation but assumes robots have complete knowledge of the environment, making plans prone to errors in unknown settings. Wang et al. [16] use LLMs with conformal prediction for safe multi-agent planning, but the action choices are limited to a small set of objects. Table 1 presents a comparison of the characteristics of different LM-based approaches to multi-agent planning with our work.LMs can interpret high-level instructions and break them down into feasible subtasks, making them ideal for long-horizon, multi-task scenarios. Our work leverages LMs to enable long-horizon planning across a variety of tasks and environments, building on these advances to address the limitations of traditional RL and HRL methods. By integrating LMs into our planning framework, we enhance the ability to generalize across diverse tasks and scenarios, making significant strides toward practical, real-world applications of RL in dynamic, multi-agent settings.Dynamic Local Failure Self Planning Information Correction Verification Two-Step [17] ✗ ✗ ✗ ✗ Smart LLM [15] ✗ ✗ ✗ ✗ Conformal Prediction LLM [16] ✓ ✗ ✗ ✗ CoELA [18] ✓ ✓ ✗ ✗ LLaMAR (this paper) ✓ ✓ ✓ ✓Table 1 : The proposed model, LLaMAR: 1) performs dynamic planning, avoiding the open-loop plan-and-execute paradigm; 2) operates without privileged simulator information (e.g., access to all objects in the environment); 3) re-plans when low-level actions fail, not assuming perfect execution; and 4) self-verifies subtask completion without relying on the simulator. 
background: 
 Problem Setting: We consider a setting where multiple robots perform a series of tasks (a) such as cleaning a room or putting groceries in the fridge, in a home-like environment, and (b) rescuing missing personnel and putting out forest fires in a search & rescue environment (SAR). These tasks typically require long-horizon planning, involving around 100 low-level actions to reach the goal. Our objective is to compute plans for a team of robots to execute high-level language instructions, I. We formalize these tasks as partially observable Markov decision processes (POMDP) [51, 52] , denoted as ⟨N, I, S, {O i }, {A i }, P, G, T ⟩. N is the number of agents and I is the high-level language instruction set. Here, s ∈ S represents the joint state of all agents, and o ∈ O denotes the observation set for all agents. Particularly, o i ∈ O i is the observation set of agent i, that captures incomplete environment state information. a ∈ A = A 1 × A 2 • • • A N represents the joint action space. The joint action space comprises of different categories of high-level actions A = A N AV ∪ A IN T ∪ A EXP , where A N AV is the joint navigation action set, A IN T is the joint interaction actions which allow the agents to interact with objects, and A EXP are the joint exploration actions which allow the agents to explore the environment. Examples of the high-level actions include PickUp(object) ∈ A IN T and NavigateTo(location) ∈ A N AV . Each high-level action is associated with a lowlevel primitive action (pre-trained RL, behavior cloned, or heuristic-based policy). These actions are executed synchronously by all agents at every high-level decision step. P(s ′ |s, a) is the joint transition probability function that defines the probability of arriving at s ′ ∈ S after taking joint action a ∈ A in s ∈ S. G = {g 1 , • • • , g k } defines the subtasks that the agents need to perform to accomplish the language instruction task. T is the length of the planning horizon. Environment: To simulate open-ended, long-horizon tasks that resemble everyday activities in a home-like environment, we use the AI2Thor simulator [37] , which supports a diverse set of interactions and photorealistic rendering. Since our approach does not require any parametric training, it can potentially translate to other similar embodied environments like VirtualHome [53] , Habitat [38, 54, 55] , and ThreeDWorld [56] , possibly extending beyond household domains. Similarly, to simulate the search & rescue scenario, we create a custom Search & Rescue environment (SAR).More information about the MAP-THOR and SAR environments can be found in Appendix B and D respectively. We instantiate the problem with N agents cooperating to accomplish a long-horizon rearrangement task [57] in an indoor environment. The agents do not know the objects present in the environment a priori and are encouraged to explore the environment to gather more information to complete the task. Unlike previous solutions in similar settings [18, 15, 16] , we do not rely on an oracle/simulator to verify subtask completion. In prior works, a predefined conditional satisfiability check is used as subtask completion feedback.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
approach: 
 We describe our approach in this section. Figure 1 illustrates LLaMAR's architecture comprising four modules: Planner, Actor, Corrector, and Verifier, each an LM with a distinct role. Prior work [58] shows that splitting roles across different LMs improves performance in sequential decision-making.Our initial experiments confirm that LMs tasked with reasoning about multiple inputs and providing long outputs perform poorly. We iterate through these four modules at every high-level decision step. The pseudocode for our approach is in Appendix E. We define some key notation below:• Memory M: A textual description of the joint memory of all agents, summarizing past observations, high-level actions, plausible reasons for action failures, and specific subtasks that each agent is attempting to solve. • Open Subtasks G O ⊂ G: Feasible subtasks proposed by the Planner LM to achieve the environment task that are yet to be accomplished by the agents. • Completed Subtasks G S ⊂ G: Subtasks completed by the agents.• Corrective Actions a c : Corrective actions for each agent based on failure information from the previous step.At the start of each episode, Memory M, Open Subtasks G O , Completed Subtasks G S , Actions a, Corrective Actions a c , and Failure Information F are initialized as empty sets.Consider an example of a kitchen with groceries, a fridge, and a counter. Two agents are tasked with "Fetch the groceries and place them in the fridge". This example will help illustrate the utility of each module. All LMs receive a language task instruction I, joint observations from all agents, and information about open and completed subtasks and memory unless stated otherwise. We next discuss the various components in our architecture in detail:Planner Module The Planner LM module suggests feasible subtasks to ensure the completion of the environment task. This method, similar to SmartLLM's [15] zero-shot planning, uses only observations in the agents' field of view. The Planner suggests subtasks related to objects seen in the current observation or memory of all the agents. For the example considered, it decomposes the task into subtasks like "transport the tomato to the fridge" and "transport the lettuce to the fridge", which are added to G O . Verifier Module After executing high-level actions, the Verifier LM assesses whether these actions have completed any subtasks in the open subtask set. Successful subtasks are moved to the completed subtask set. Without the Verifier LM, the method would need to rely on the simulator/oracle for success or failure information. The Verifier LM along with other information uses the successfully executed high-level actions proposed by the Actor LM to predict subtask completion. For example, after transporting the lettuce to the fridge, the Verifier updates the completed subtasks with "transport lettuce to the fridge".Admissible Action parsing with Semantic Translation When LMs generate action plans, natural language outputs often fail to translate to executable high-level actions. This happens when the output does not match the predefined format or refers to unrecognized contextually similar objects. We use a cosine similarity method from [9] , fine-tuning a pre-trained sentence-BERT [59] to transform the free-form text into admissible high-level actions. Hyperparameters and additional details of the sentence transformer fine-tuning are provided in Appendix J.In unexplored environments, agents need to search for task-relevant objects.If agents cannot find the required objects, the language model can choose an 'exploration' action a exp . We use a semantically-guided heuristic to determine the choice of region to be explored. E d = |G O | i=1 g O,i •I d ∥g O,i ∥∥I d ∥ .The direction with the highest score d * = arg max d E d is chosen. Summing the scores helps select the best direction to explore in expectation. The agent rotates towards d * and moves J = 2 steps, repeating this process K = 3 times in one explore action. This approach ensures that images relevant to identifying potential subtasks are prioritized. For example, if G O includes "locate a computer", it is more likely to find a computer on a table than on a sofa, resulting in a higher cosine similarity score between the subtask CLIP text embedding and table CLIP image embedding. Refer to Appendix B.2 for more details about the exploration heuristic.The specific order of the modules in LLaMAR is due to natural causal relationships in which environment feedback is received. We use the Planner as the first module because it allows LLaMAR to come up with an initial list of open subtasks that could be completed based on the current observation and past memory to satisfy the task. This list serves as a rough high-level plan. The actor then uses this information to suggest the necessary actions. The Corrector is used after the Actor module to identify reasons for failures in the execution of the actions suggested by the Actor . Note that the failure module is inert and only suggests corrective actions. Only the Actor module decides the final actions to be executed. This role distinction allows for clear reasoning on failures when they occur and lets the actor module focus on choosing actions. The Verifier is used after the action is executed to update the list of closed subtasks so that LLaMAR can be current with the progress toward the completion of the environment task. This allows the planner to update the list of open subtasks in the next step. In essence, the Planner and the Verifier ensure that the progress of the agents is tracked and the actor and the corrector ensure that the actions are executed successfully to advance towards completion of the task.Multi-Agent features in LLaMAR While our method can be easily adapted to a single-agent setting, our design choice for the architecture was motivated to include the following multi-agent features:• Coordination through communication: Agents share their state information with the centralized LLaMAR modules to predict actions, enabling them to coordinate and avoid conflicts. This information sharing allows for the agents to cooperate and achieve the collective goal. • Dynamic Role Assignment: Agents are dynamically assigned roles based on the current task requirements and their capabilities. This flexibility allows LLaMAR to adapt to changing environments and task demands. • Hierarchical Task Decomposition: To handle the complexity of multi-agent planning, LLaMAR decomposes the action space by creating specific subgoals/subtasks available for any agent to assign itself (done by the actor module) based on the observation and current context. This decomposition reduces the overall search space and improves planning efficiency. 
experiments: 
 MAP-THOR: To evaluate the performance of LLaMAR and benchmark other baseline methods, we create a benchmark dataset of tasks which we call MAP-THOR (Multi-Agent Planning tasks in AI2-THOR). While Smart-LLM [15] introduces a dataset of 36 tasks within AI2-Thor [37] classified by complexity, their tasks are limited to single floor plans. This limitation hinders testing the robustness of planners across different room layouts. Additionally, some tasks in their dataset cannot be performed by multiple agents, regardless of task division, such as Pick up the pillow, Open the laptop to turn it on, and Turn off the lamp.By contrast, MAP-THOR includes tasks solvable by both single and multiple agents. We classify the tasks into four categories based on the ambiguity of the language instructions. To test the planner robustness, we provide five different floor plans for each task. We also include automatic checker modules to verify subtask completion and evaluate plan quality. Our dataset comprises 45 tasks, each defined for five distinct floor plans, ensuring comprehensive testing and evaluation.We conduct experiments with tasks of varying difficulty levels, where an increase in difficulty of the tasks corresponds to an increased ambiguity in the language instructions. The complete task list of each category can be found in the Appendix C.• Explicit item type, quantity, and target location: Agents are explicitly instructed to transport specific items to specific target locations. For example, put bread, lettuce, and a tomato in the fridge clearly defines the objects (tomato, lettuce, bread) and the target (fridge). • Explicit item type and target location but implicit item quantity: The object type is explicitly described, but its quantity is not disclosed. For example, Put all the apples in the fridge. Agents must explore the environment to locate all specified items and also predict when to stop. • Explicit target location but implicit item types and quantity: The target location is explicitly defined but the item types and their quantities are concealed. For example, Put all groceries in the fridge. • Implicit target location, item type, and quantity: Item types and their quantities along with the target location are implicitly defined. For example, Clear the floor by placing the items at their appropriate positions. The agent is expected to place items like pens, books, and laptops on the study table, and litter in the trash can.To showcase the effectiveness of LLaMAR with respect to explicit coordination in multi-agent settings, we evaluate LLaMAR in a partially observable search & rescue and fire relief environment in a grid world. Depending on the scene, there is a mix of missing people to be found, and wildfires to be stopped before they spread geographically. More details about the environment can be found in Appendix D.• Fire Extinguishing: Fires consist of expansive flammable regions with a fixed set of sources that propagate over time. The rate of fire spread is proportional to its intensity; higher intensities result in faster spread. Fires are categorized as either Class A or Class B, which are extinguished using water or sand, respectively. These extinguishing resources are sourced from reservoirs distributed across the environment. • Human Rescue: Each individual is initially located at an unknown position within the environment. The objective is to locate, rescue, and transport the individuals to a designated drop-off location, which is known beforehand. Transporting a person requires the coordinated effort of two agents simultaneously, who must carry them to the specified drop-off point.We evaluate the algorithms using the following metrics to compare their performances on the tasks:• Success Rate (SR): The fraction of episodes in which all subtasks are completed. Success equals 1 if all subtasks are successfully executed in an episode, otherwise it is 0. • Transport Rate (TR): The fraction of subtasks completed within an episode, provides a finer granularity of task completion. • Coverage (C): The fraction of successful interactions with target objects. It is useful to verify if the LMs can infer the objects to interact with, in scenarios where the tasks have objects that are specified implicitly. • Balance (B): The ratio between the minimum and maximum number of successful high-level actions executed by any agent that contributed towards task completion. We only check for a subset of high-level actions that must be executed to accomplish critical subtasks that lead to the successful completion of the language instruction task. If each agent i out of n agents completes s i successful tasks, the balance is defined as: B := min {s1,••• ,sn} max{s1,••• ,sn}+ϵ . This measures how evenly the work is distributed among agents. A balance of zero indicates at least one agent performed no successful high-level actions, while a balance of one indicates all agents performed the same number of successful high-level actions. Here ϵ = 1e -4 is a small number to avoid division by zero.• Average steps (L): The number of high-level actions taken by the team to complete the task, capped at L = 30 in our experiments. If the task is not completed within L steps, the episode is deemed a failure. Note that the metric L is presented in the table providing the complete results, located in Appendix F.For all the metrics, we report the means along with the 95% confidence interval across all the tasks (refer Appendix F for complete results). Since SR is a binomial metric, we report the Clopper-Pearson Interval as the confidence interval.For a fair comparison with our method, we make modifications to the baselines to make them work in partially observable settings with limited reliance on the simulator. More details about implementations can be found in Appendix H.• Act: We query the LLM with the task and the observations to suggest a high-level action.• Chain-of-Thought [61] : We modify the Act prompt with a chain-of-thought style addendum to let the LM reason about the possible implications while selecting a high-level action. • ReAct [62] : We use a ReAct-style prompting to let the LMs reason after suggesting high-level actions and possibly suggest ways to correct any failures. • SmartLLM [15] : We modify the official codebase to only include information from the local observations of the agents instead of assuming full observability. • CoELA [18] : We modify the list of available high-level actions to include all possible valid combinations of actions with interactable objects in the agent's local observation. As the scene becomes more cluttered, this list and the prompt becomes combinatorially longer. In the original implementation, the list of available actions is filtered based on the feasibility of the actions as suggested by a conditional checker.It should be noted that Act, Chain-of-Thought, ReAct, and SmartLLM are all CMAS frameworks where CoELA follows the DMAS framework. 
results and discussion: 
 Choice of the underlying LM: To understand the impact of the underlying LM's quality on decisionmaking, we initially experimented with different LMs on MAP-THOR. Specifically, we utilize both the language-only and vision-language models of GPT-4 [63] , IDEFICS-2 [64, 65] , LLaVA [66, 67] ,and CoGVLM [68] . Among these, GPT-4, when used solely with text inputs, exhibits the poorest performance. This is attributed to the agents' inability to reason about visual observations, which is particularly detrimental for the Corrector module. Substituting GPT-4V with other vision-language models results in a decline in performance (refer Table 2 ) and hence we use GPT-4V as the underlying VLM while comparing to the baselines.Baseline Comparisons: Table 2 compares our method, LLaMAR, with other baselines in a 2-agent scenario using GPT-4V as the underlying VLM. Act and ReAct show similar performance, with Act struggling due to its lack of strategic planning or correction, and ReAct performing slightly better by dynamically adjusting actions based on reasoning on immediate feedback. CoT's performance declines with longer planning horizons due to its inability to maintain coherence over extended planning sequences, consistent with findings in [69] , showing its effectiveness only with highly specific prompts. SmartLLM, operating in a plan-and-execute paradigm, generates impractical plans with issues like infinite loops and failure to handle low-level action failures, leading to lower success rates and poor transport metrics. It also tends to hallucinate objects. CoELA, using a decentralized multi-agent system (DMAS), performs poorly due to large input prompts and struggles to select the correct action from numerous choices. Its decentralized decision-making is less efficient than the centralized multi-agent system (CMAS) used by LLaMAR. Previous research [12] confirms CMAS frameworks are more effective than DMAS frameworks. Overall, our method, LLaMAR, benefits from its modular cognitive architecture, which integrates planning, acting, correcting, and verifying through distinct LLM roles, resulting in superior performance across various evaluation metrics.By avoiding reliance on privileged information and incorporating a robust exploration strategy that allows it to scout for objects that are not initially visible, LLaMAR ensures higher success rates and balanced task execution among agents.To demonstrate the effectiveness of the various modules in our cognitive architecture, we performed ablation studies by evaluating performance metrics with each module removed individually. The results are summarized in Table 3 . Using only the Actor module corresponds to the "Act" baseline, which demonstrates its fundamental capabilities in isolation but shows limited effectiveness without planning and correction due to relatively lower success and transport rates. Adding the Planner and Verifier modules improves performance, benefiting from better task planning and validation, increasing the overall SR and TR, and ensuring more effective task completion and even work distribution, as indicated by the increase in balance (B). However, in scenarios where the suggested action fails, the actor suggests the same action in the next decision step since it is not able to reason why the action failed until the end of the planning horizon. Incorporating the Corrector module, with access to privileged information from an environment oracle, significantly boosts performance, enhancing the SR, TR, C, and further improving B, consistent with the findings in [13] . This highlights the Corrector module's importance in adjusting actions based on controller feedback, resulting in higher task success and more efficient task completion, albeit with reliance on oracle knowledge. Finally, the complete LLaMAR system, without privileged information, achieves SR, TR, C, and B values close to those of the oracle setup. This demonstrates the system's robustness and effectiveness in a realistic setting. The Corrector module plays a crucial role in enabling agents to learn from past failures and avoid repeating actions, preventing task failures due to timeout. Despite lacking oracle knowledge, LLaMAR performs nearly as well as the oracle-enhanced setup. These results highlight the importance of each module in our cognitive architecture. Removing any module diminishes effectiveness.Increasing the number of agents Increasing the number of agents in the environment shows distinct trends in our method's performance metrics for both MAP-THOR and SAR environments (refer Table 4 ). In MAP-THOR, with two agents, we establish a solid baseline for success rate (SR) and transport rate (TR), which is similarly reflected in the SAR environment. Adding a third agent improves both SR and TR in both environments, indicating enhanced task completion and transportation efficiency. Coverage (C) also increases, suggesting better exploration and interaction with objects across both environments. There is a slight decrease in SR and TR when the number of agents increases from 3 to 5 in the MAP-THOR environment. The decrease in these metrics can be attributed to the rooms in MAP-THOR becoming crowded with 4 and 5 agents hence blocking the agents from navigating without colliding with other agents. But this phenomenon is not seen in the SAR environment which is comparatively more spacious and navigable. However, balance (B), which measures the even distribution of tasks among agents, decreases with more agents. This drop highlights the challenge of ensuring equal contributions from all agents in a larger multi-agent system. While SR remains high, the balance metric drops significantly from 2 to 5 agents, indicating some agents do more work than others. In summary, adding more agents improves task performance and efficiency but introduces challenges in maintaining balanced contributions. Addressing this imbalance is crucial for refining multi-agent planning algorithms.Correcting Failures: In numerous instances, the actions proposed by the Actor module, such as pick up <object>, are unsuccessful due to the agent's insufficient proximity to the target object.In such situations, the Corrector module uses visual feedback to learn from these failures and recommends appropriate corrective actions, such as navigate to <object> to facilitate closer proximity. Figure 2 shows examples where the Corrector module interprets low-level action failures and suggests remedies, highlighting its importance. 
limitations and future work: 
 Higher number queries to the LM: Since each high-level decision step requires querying 4 different LM-based modules, the cost and the compute times are higher than other baselines, especially compared to the plan-and-execute baselines like SmartLLM. An interesting future direction to improve this would be to fine-tune smaller LMs with trajectories collected in the simulator (eg: ALFRED [42] ) as done in [18] . Another potential direction worth exploring is using different sizes of LMs for each module based on their specific utility. Limited spatial reasoning: Although we use both textual descriptions and visual features to guide the language model's actions, it still lacks the ability to reason about the spatial features of the environment. Spatial reasoning is crucial in scenarios such as navigating around obstacles to reach an object, or determining the shortest path to collect multiple items scattered across different locations.One way to address this limitation is to inject information about the 3D world into the LM, as done in [70] , which is an interesting direction for future work.Performance limited by the underlying VLM: Although LMs make correct reasoning most of the time, they still occasionally make mistakes, including misunderstanding the environment rules specified in the prompt. For example, the agent assumes that the cleaning task requires putting soap, drying, and putting it in the sink when all it needs is the action "CleanObject", and can't figure out the appropriate level of abstraction. The performance of the algorithm is limited by the instruction following and reasoning capability of the underlying LM [71, 72] ; this calls for developing LMs that are fine-tuned to instruction-image pairs relevant to the environment (as done in [18] ). 
conclusion: 
 We address long-horizon planning in dynamic, partially observable multi-agent environments with LLaMAR, an LM-based planner using four specialized modules: Planner, Actor, Corrector, and Verifier. This framework iteratively refines action planning, adapts to failures, and verifies subtask completion using real-time observations and action feedback, without privileged information. We also introduce a heuristic-based exploration strategy to guide agents to semantically relevant regions. Additionally, we present MAP-THOR, a benchmark dataset for multi-agent tasks in the AI2Thor simulator. Empirical results show LLaMAR outperforms existing LM-based approaches, achieving a 30% higher success rate on MAP-THOR.Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: Yes, we denote all these details in the appendix.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: In our opinion, our work does not have any significant societal impact other than being able to progress towards enabling embodied multi-agent robots for household planning.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: No high risks in the paper.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We have cited all the authors of the works we base our work on especially the AI2Thor simulator. We have also provided links to the weights we used for the open-source VLMs in the appendix with the commit # number on HuggingFace.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: We provide documentation for the initialization of the multi-agent setting in the AI2Thor simulator in the supplementary material. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: We differentiate between the terms subtasks and high-level actions in this section. In essence, multiple high-level actions are needed to be carried out in a sequence to complete a subtask. Multiple subtasks need to be satisfied to complete the high-level language instruction.• Subtasks: A task is split up into multiple subtasks. For example, if a task is "Fetch all the groceries and put them in the fridge", then the initial subtasks could include: "Locate the groceries", "transport the groceries", "Locate the fridge". These subtasks could get updated with new observations. For example, while locating the groceries, the agents come across a tomato and a lettuce. Then the subtasks "transport the tomato to the fridge" and "transport the lettuce to the fridge" gets updated in the subtasks list. This basically splits up the high-level instruction I into multiple mid-level subtasks • High-level actions: These are the set of actions required to complete the subtasks. For example, to complete the "transport the lettuce in the fridge", we would require: the following set of actions:-Navigate to lettuce -Pickup lettuce -Navigate to the fridge -Open fridge -Put lettuce in the fridge -Close fridge Note that different agents have to complete different high-level actions that progress the subtasks efficiently whilst avoiding conflicts. • Conflicts can arise in the following ways:-Same actions: Agents performing the same action at the same time. Example: "Open the fridge". -Blocking: Agent 1 is blocking Agent 2 and not allowing it to complete its high-level action.Example: Agent 1 is attempting to execute "PlaceObject(Tomato)" in front of the fridge to place the tomato in its hand in the fridge and Agent 2 is attempting to execute "OpenFreezer()" needs to interact with the fridge. Would require some form of conflict resolution in the state cell domain. Agent 1 should move away to allow fridge access to Agent 2. In LLaMAR, the Corrector module helps in figuring out these conflicts and suggest different corrective high-level actions.The environment is based on the AI2Thor simulator with a multi-agent setup. All the experiments were performed in the single-room floor plans. When more than 3 agents are added to some of the floor plans (especially the kitchen floor plans), the environment gets crowded and does not allow for a lot of free space to navigate to different objects (the number of reachable paths reduces). The observations for each robot include an image of size resolution 1000 × 1000 × 3. The textual observation for each agent in the prompt is the list of objects visible in this image and the agents' current location and rotation. The field of view is 90 degrees. The agents can interact with the objects only if it is within its visibility range of 1.5m.The actions space A consists of navigation actions A N AV , interaction actions A IN T , exploration action A EXP .Navigation actions A N AV consists of the following actions:• Move(<direction>): Moves the robot by 0.25m towards the specified direction where <direction> can be one of (Ahead, Back, Right, Left) • Rotate(<direction>): Rotates the robot by 90 degrees towards the specified direction where, <direction> can be one of (Right, Left) • LookUp(<angle>) rotates the pitch of the robot camera upwards by the specified angle.• LookDown<angle> rotates the pitch of the robot camera downwards by the specified angle.• NavigateTo(<object_id>) makes the robot navigate to the specified object. The path is found using the A * -shortest path algorithm. Note that the robot is only able to find a path to the specified object in the environment only if it has encountered that object previously during the episode. Otherwise, the NavigateTo(.) action will be unsuccessful and the agent will have to explore. These actions can be combined in numerous ways, as the <object_id>, <directions>, and <angles> can vary significantly, resulting in a combinatorially large action space. To address this complexity, we do not impose constraints on the language model (LM) modules to select actions in a predefined format from this set. Instead, we allow for free-form language outputs, offering greater expressivity. The SentenceBERT module then maps these free-form natural language outputs to executable actions within the environment. o n,d = env.step(Rotate(Right, n))6:I d = CLIP img (o n,d )7:E d = I d •g O ∥I d ∥∥g O ∥ 8:end for 9:d * = arg max d E 10:while j < J do 11:o i = env.step(Move(d * , n))12:j ← j + 1 13:end while 14:k ← k + 1 15: end while -Move everything on the table to the desk (should move the laptop, pencil, pen, plate, credit card, book, and newspaper) -Slice the lettuce, trash the mug and switch off the light -Put all electronics on the couch -Make a dish by microwaving eggs and tomato -Put all readable objects on the sofa -Wash all fruits • Implicit target and object types: Here both the object type and the target are implicitly defined. E.g. Clear the floor by placing the items at their appropriate positions. Here the model is expected to keep items like pens, book, laptop on the study The Search & Rescue environment consists of multiple agents in an unknown environment that has multiple wildfires and missing personnel in the environment. The agents are tasked to extinguish all the fires before they spread and rescue the missing humans. Here, each fire is composed of a large flammable region with a fixed set of sources that spread through time. The higher the intensity, the faster the fire will spread. The fires can be of class A or B, extinguished through the use of water and sand respectively. Both these resources can be collected through resource reservoirs spread geographically. Each person is initially stranded in an unknown location. The goal is to rescue and transport each person to a drop-off location (known apriori). The person must have two agents simultaneously carrying them to be transported to the drop-off location. We provide a justification to why this task is relevant to long-term planning and multi-agent collaboration below.• Scalability -Unlike the AI2THOR environment, the search & rescue environment is more spacious and hence prevents congestion amongst agents. With the addition of multiple parallel emergency disaster scenarios at once, the environment scales comfortably to more agents. Additionally, the complexity slowly increases as the model has to coordinate between more actors and competing emergencies. • Tasks requires explicit cooperation --To move a group of humans, at least 2 agents are required. Thus, explicit multi-agent collaboration is required. -Due to the time pressure, in order to successfully stop the fire, agents must all effectively collaborate in collecting and strategically using resources in high-intensity regions. • Exploration & Task Assignment -There is an explicit tradeoff between allocating agents towards exploring to find lost people and fighting the current fires.• Dependency Chain: To resolve a fire it requires that the source is identified, the type of fire is identified, and the appropriate resources to stop it are acquired and then used. • Uncertainty: Inherent uncertainty as to where the lost people can be found, and at what point in the timeline they will. • Irreversibility and forced consequences -With a fire that spreads, present actions have irreversible future consequences. A balance needs to be struck between fighting the fire's source (to stop it from continuing) versus a periphery (to prevent geographic spread).We evaluate LLaMAR in five different scenarios in the search & rescue environment. Here each scene is evaluated on 5 different random seeds.• Global Observations O G consists of all the objects that are globally visible by the agent:• Fires: If visible, fire name, type, and average intensity.• Fire Regions: If fire visible, fire region name, type, and average intensity.• Reservoir: If visible, reservoir name, and reservoir resource type.• Deposit: If visible, deposit name, inventory of all the resources (water, sand) and persons in deposit.• Person: If visible, person name, carried or not status, dropped-off or not status.• Agent inventory: List of all the resources (water, sand) and person (being carried) in the agent's inventory.Local Observations O L consists of all the objects that are visible in grid cells adjacent to the agent:• For all the directions <direction> in (Up, Down, Left, Right, and Center), output of the following.• <direction>: At direction <direction>, either 'Empty' if there is no object, 'Flammable' along with corresponding intensity & fire name if object is a part of a fire, or 'Obstacle' for any other object.Names O N consists of a list of the name of all visible and interactable objects.The action space A consists of navigation actions A N AV , interaction actions A IN T , exploration action A EXP .Navigation actions A N AV consists of the following actions:• Move(<direction>): Moves the agent to the neighboring grid cell in the specified direction where <direction> can be one of (Up, Down, Left, Right, and Center) • NavigateTo(<targetID>): Moves the agent next to the location of the object <targetID> if <targetID> is visible.Interaction actions A IN T consists of the following actions:• Carry(<person>): Makes agent carry a person <person> if <person> is visible and interactable. The person is successfully 'group' carried, if at least the required number of agents successfully does the 'Carry(<person>)' action. If carry action is successful, all other resources in agent's inventory are dropped. • DropOff(<person>, <deposit>): Drops off person <person> at location <deposit>. This action is only successful if the person has been 'group' carried, all the agents carrying the person have the deposit be visible and interactable, and after all the agents do this action. • StoreSupply(<deposit>): Stores all the resources from the agent's current inventory in the deposit <deposit>. • UseSupply(<fire>, <supply-type>): Uses all the supplies of type <supply-type> on the fire <fire> at the location the agent is at. • GetSupply(<deposit>, <supply-type>): Fills the remaining of the agent's inventory space with the available supply of type <supply-type> in <deposit>. • GetSupply(<reservoir>): Collects 1 unit of supply from reservoir <reservoir> and stores it in the agent's inventory.Exploration action A EXP consists of the following actions:• Explore(): Takes the agent in a direction of exploration as described by the heuristic exploration function described in algorithm 2. G O ← Planner(I, o, G O , G C , M) 4: a, M ← Actor(I, o, a c , G O , G C , M) 5: o = (o 1 , • • • , o N ), d = (d 1 , • • • , d N ) = env.step(a)6: a c ← Corrector(I, o, a, d, G O , G C , M) 7: G C ← Verifier(I, o, a, d, G O , G C , M) 8: if G O = ∅One of the main motivations of our paper was to achieve better success rates in planning for a variety of tasks compared to previous approaches. While our method outperforms other baselines, we acknowledge that the success rate is still under the expectation for real-world deployment. We believe that our approach LLaMAR and the MAP-THOR benchmark would serve as a starting point for future research to improve the success rates in multi-agent embodied robotics tasks. Here we describe a few major types of failures in our experiments:• Mis-generalization: The agent can sometime fail to properly infer the level of abstraction to perform a task at, even if it should be clear from the action space. For example, for the washing tasks, the LM assumes that it must put the objects in the sink and add some, rather than just using the "CleanObject" action. We observe this error in the direction of performing actions more low-level than necessary, and not the other way around (more high-level than is feasible). • Mutual Interference (limited spatial reasoning): The agents sometimes block each other and thus fail to carry out actions such as placing an object on a receptacle, opening an object, etc. In particular, we see this behavior in the "put objects on the sofa" and the "put objects in a box" tasks, where the LM does not prevent the agents from blocking each other. • Improper Object Visibility (un-observability bias): The LM often fails to prioritize exploration for tasks that require objects not yet seen, simply due to it no being able to see it. This bias causes it to improperly assume that further exploration is not necessary, so it fails to find relevant objects. • SentenceBERT mismatch: The free-form natural language output from the Actor is incorrectly mapped to the executable action. Based on our experimental results, 96.7% of the time the free-form natural language was correctly mapped to the feasible actions. The 2 main reasons for failures in the few cases it failed were:-Incorrect mapping of objects: When the agents have not explored the environment enough and the Actor module suggests the agents interact with objects that are not yet available in the agents' combined memory. Example: The free-form output "navigate to the table" was mapped to the action "NavigateTo(ArmChair)". -Wrong object numeration: When there are multiple objects of the same type in the memory, the sentenceBERT maps "Cabinet_3" to "Cabinet_1" which sometimes leads to incorrect actions being executed for the completion of the plan. • Limited horizon: While each task is solvable by a single-agent human-controlled agent within our chosen episode length of L = 30, this is under the assumption of no failures during control and maximum visibility. For the autonomous agents the visibility distance is restricted to 1.5 meters, so while a human could identify objects from afar (from the rendered images) the agents have to explore the environment under a stricter partial observability constraint.We hypothesize that a higher episode length would lead to higher metrics, but the cutoff was chosen for computational budget considerations. As can be observed in figure 6 , the coverage and transportation rate metrics in particular seem to plateau around our chosen episode length L, so L = 30 is appropriate. We have observed the following categories of failures for the LM on the SAR environment.• Incorrect Causal Ordering: The LM sometimes fails to execute the steps in the proper causal ordering. For instance, we have observed the agent trying to drop off a lost person before picking them up, or properly getting supplies to stop the fire but not navigating to the fire location before using them. • Sub-Optimal Action Sequence: Due to the time-constraints of the fire spread, it is not sufficient for the LM to do the sequence of tasks in order. They must also be done in a non-redundant, timely fashion. Otherwise, we have observed that the LM takes the right action sequence, but does not do it efficiently enough to stop the fire spread. For instance, if the agent prioritizes exploration to find the person, then the fire might spread until beyond a critical point where it cannot be extinguished within the horizon. • Catastrophic Failure Misunderstanding: Even after the LM has completed the task, it can misunderstand a later failure as a cause for continuing the task. For instance, we have observed that if the LM has successfully dropped off a person, but then fails to pick them up again (which is expected since they're already dropped), it will think the task is incomplete.While there are a lot of impressive LLM-based multi-agent planners as mentioned in Table 1 , they vary in the assumptions about access to information about the environment. We were not able to find the official codebase for the Safe Multi-Agent Planning with Conformal Prediction [16] and TwoStep [17] . We describe the prompts used for our model as well as every baseline. Note that we show the prompt for the 2-agent case, but it is easily modified to generalize to the n-agent case. The italics and bolding added for emphasis.We describe the prompts used for each of the modules used in LLaMAR:You are an excellent planner who is tasked with helping 2 embodied robots named Alice and Bob carry out a task. Both robots have a partially observable view of the environment. Hence they have to explore around in the environment to do the task.You will get a description of the task robots are supposed to do. You will get an image of the environment from Alice's perspective and Bob's perspective as the observation input. To help you with detecting objects in the image, you will also get a list objects each agent is able to see in the environment. Here the objects are named as "<object_name>_<object_id>". So, along with the image inputs you will get the following information: The "Plan" should be in a list format where the subtask are listed sequentially. For example:### INPUT FORMAT ##["locate the apple", "transport the apple to the fridge", "transport the book to the table"] ["locate the cup", "go to cup", "clean cup"] When possible do not perform additional steps when one is sufficient (e.g. CleanObject is sufficient to clean an object, no other actions need to be done) Your output should be in the form of a python dictionary as shown below.Example output: {"reason": "Since the subtask list is empty, the robots need to transport the apple to the fridge and transport the book to the table.", "plan": ["transport the apple to the fridge", "transport the book to the table"]}Ensure that the subtasks are not generic statements like "do the task". They should be specific to the task at hand. Do not assign subtasks to any particular robot. You are an excellent task verifier who is tasked with helping 2 embodied robots named Alice and Bob carry out a task. Both robots have a partially observable view of the environment. Hence they have to explore around in the environment to do the task.You will get a description of the task robots are supposed to do. You will get an image of the environment from Alice's perspective and Bob's perspective as the observation input. To help you with detecting objects in the image, you will also get a list objects each agent is able to see in the environment. Here the objects are named as "<object_name>_<object_id>". So, along with the image inputs you will get the following information: The "Completed Subtasks" should be in a list format where the completed subtasks are listed. For example: ["locate the apple", "transport the apple to the fridge", "transport the book to the table"]### INPUT FORMAT ##Your output should be in the form of a python dictionary as shown below.Example output: { "reason": "Alice placed the apple in the fridge in the previous step and was successful and Bob picked up the the book from the table. Hence Alice has completed the subtask of transporting the apple to the fridge, Bob has picked up the book, but Bob has still not completed the subtask of transporting the book to the table", "completed subtasks": ["picked up book from the table", "transport the apple to the fridge"] } * NOTE: DO NOT OUTPUT ANYTHING EXTRA OTHER THAN WHAT HAS BEEN SPECIFIED When you output the completed subtasks, make sure to not forget to include the previous ones in addition to the new ones. Let's work this out in a step by step way to be sure we have the right answer.You are an excellent planner and robot controller who is tasked with helping 2 embodied robots named Alice, and Bob carry out a task. All 2 robots have a partially observable view of the environment. Hence they have to explore around in the environment to do the task.They can perform the following actions:["navigate to object <object_id>", "rotate in <rotation> direction", "pick up object <object_id>", "put object on <receptacle_id>", "open object <object_id>", "close object <object_id>", "slice object <object_id>", "toggle object <object_id> on", "toggle object <object_id> off", "clean object <object_id>", "look up by angle <angle>", "look down by angle <angle>", "move in <translation> direction", "stay idle", "Done"]Here "Done" is used when all the robots have completed the main task. Only use it when you think all the subtasks are complete."stay idle" is used when you want the robot to stay idle for a one-time step. This could be used to wait for the other robot to complete its subtask. Use it only when you think it is necessary.Here <rotation> can be one of ["Right", "Left"].Here <angle> is the angle in degrees and can only be one of [30, 60, 90, 120, 150, 180] .Here <translation> can be one of ["Ahead", "Back", "Left", "Right"].So, along with the image inputs you will get the following information: Common failure reasons to lookout for include: one agent blocking another so must move out of the way, agent can't see an object or its destination and must explore (such as move, rotate, or look in a different direction) to find it, agent doing extraneous actions (such as drying objects when cleaning), etc. If the previous action was successful, output "None".### INPUT FORMAT ##* Memory: Whatever important information about the scene you think you should remember for the future as a memory. Remember that this memory will be used in future steps to carry out the task. So, you should not include information that is not relevant to the task. You can also include information that is already present in its memory if you think it might be useful in the future. * Action: The actions the robots are supposed to take just in the next step such that they make progress towards completing the task. Make sure that these suggested actions make these robots more efficient in completing the task as compared to only one agent solving the task. Your output should just be in the form of a python dictionary as shown below.{ "failure reason": "Bob failed to put the mug in the cabinet earlier because Alice was blocking it when she was putting the knife. To fix this, Alice should close the cabinet and move away , Charlie should move away to a different open area than Alice to avoid congestion, and Bob should wait until the next timestep until Alice can move aside.", "memory": "Alice finished putting the knife in the cabinet when Alice was at co-ordinates (1, .5) and was facing north. Bob wanted to put the mug in the cabinet when Bob was at co-ordinates (1, 0.25) and was facing north.", "reason": "Alice can close the cabinet door and then later back out in order help Bob with completing the task. Bob can be idle until the next timestep when Alice moves aside, by then Bob can navigate to the cabinet.", "subtask": "Alice is currently closing the cabinet door, Bob is currently waiting to get to navigate to the cabinet", "Alice's action" : "close the Cabinet_1", "Bob's action" : "stay idle" } Example 2: { "failure reason": "Bob failed to clean the cup earlier because Bob had not navigated to it, Bob assumed the cup to be in the sink which was erroneous. To fix this, Bob should navigate to the cup and in the next step clean cup.", "memory": "Alice finished navigating to the dish when Alice was at co-ordinates (-.5, .5) and was facing east. Bob was not able to clean the cup in the cabinet when Bob was at co-ordinates (1, .25) and was facing north.", "reason": "Alice can now clean the dish since Alice has navigated to it. Bob should navigate to the cup in order to be close enough to clean the cup.", "subtask": "Alice is currently trying to clean the dish, Bob is currently trying to navigate to the cup", "Alice's action" : "clean the dish object", "Bob's action" : "navigate to the cup" } Note that the output should just be a dictionary similar to the example outputs. We describe the prompt used for the Act baseline:Prompt for the Act BaselineYou are an excellent planner and robot controller who is tasked with helping 2 embodied robots named Alice and Bob carry out a task. Both robots have a partially observable view of the environment. Hence they have to explore around in the environment to do the task.They can perform the following actions:["navigate to object <object_id>", "rotate in <rotation> direction", "pick up object <object_id>", "put object on <receptacle_id>", "open object <object_id>", "close object <object_id>", "slice object <object_id>", "toggle object <object_id> on", "toggle object <object_id> off", "clean object <object_id>", "look up by angle <angle>", "look down by angle <angle>", "move in <translation> direction", "stay idle", "Done"]Here "Done" is used when all the robots have completed the main task. Only use it when you think all the subtasks are complete. "stay idle" is used when you want the robot to stay idle for a one-time step. This could be used to wait for the other robot to complete its subtask. Use it only when you think it is necessary.Here <rotation> can be one of ["Right", "Left"].Here <angle> is the angle in degrees and can only be one of [30, 60, 90, 120, 150, 180] .Here <translation> can be one of ["Ahead", "Back", "Left", "Right"].You need to suggest the action that each robot should take at the current time step. We describe the prompt used for the Chain-of-Thought baseline:Prompt for Chain of Thought BaselineYou are an excellent planner who is tasked with helping 2 embodied robots named Alice and Bob carry out a task. Both robots have a partially observable view of the environment. Hence they have to explore around in the environment to do the task.They can perform the following actions: ["navigate to object <object_id>", "rotate in <rotation> direction", "pick up object <object_id>", "put object on <receptacle_id>", "open object <object_id>", "close object <object_id>", "slice object <object_id>", "toggle object <object_id> on", "toggle object <object_id> off", "clean object <object_id>", "look up by angle <angle>", "look down by angle <angle>", "move in <translation> direction", "stay idle", "Done"] Here "Done" is used when all the robots have completed the main task. Only use it when you think all the subtasks are complete. "stay idle" is used when you want the robot to stay idle for a one-time step. This could be used to wait for the other to complete its subtask. Use it only when you think it is necessary. Here <rotation> can be one of ["Right", "Left"].Here <angle> is the angle in degrees and can only be one of [30, 60, 90, 120, 150, 180] .Here <translation> can be one of ["Ahead", "Back", "Left", "Right"].You need to suggest the action that each robot should take at the current time step. Your output should ONLY be in the form of a python dictionary as shown below: {"reason": "Reasoning for action plan....", "Alice": "action to be taken by Alice", "Bob": "action to be taken by Bob"} Put all of your reasoning inside of the "reason" key of the dictionary. Do NOT put any text, spaces, or enter keys (i.e. "/n") outside of it.For example: If you think Alice should pick up an apple and Bob should navigate to the fridge, you will have to give the output as: {"reason": "since the subtask list is empty, the robots need to transport the apple to the fridge", "Alice": "pick up apple", "Bob": "navigate to fridge"} Let's think step by step, but make sure to put all of your reasoning inside of the "reason" key of the dictionary! * NOTE: DO NOT OUTPUT ANYTHING EXTRA OTHER THAN WHAT HAS BEEN SPECIFIEDNeurIPS Paper Checklist Justification: we mention the limitations of our paper in a separate section in the paper. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Justification: We have read the code of ethics on the website.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).We use the simulator just to provide a boolean value about the success of high-level action execution. 


Decision: This paper presents LM-based Long-Horizon Planner for Multi-Agent Robotics (LLaMAR) an approach that combines LLMs in a  plan-act-correct-verify framework to solve long-horizon multi-agent tasks. It also introduces the MAP-THOR test suite for evaluating results. The proposed method performs well in the experiments. 

LLMs are becoming more widely used but current methods are still not very good at complex, partially observable, long-horizon tasks. Developing methods that can solve more challenging tasks is an important area and the proposed method is a promising step in this direction.  

There are some questions about the motivation and details of the particular architecture, the high computational complexity, and whether the current approach is really multi-agent but the authors addressed them in the rebuttal/discussion. The authors should include these points (as well as other feedback from the reviewers) in the final paper.

Reviews: review: paper_summary: paper_summary: The paper addresses the problem of long-horizon household tasks in a multi-agent setup using LLMs and for this, the authors propose LLaMAR. This framework integrates planning, acting, correcting, and verifying parts. LLaMAR plans a high-level action sequence, assigns them to multiple agents, and checks if actions fail to be executed, followed by a heuristic exploration strategy. It also converts the planned actions to the admissible ones in a retrieval manner based on text similarity. The approach is validated in MAP-THOR, an extended version of AI2-THOR that includes household tasks in the multi-agent setting.
strength_weakness: strength_weakness: - The proposed framework takes a plan-act-correct-verify pipeline but this is not well-justified. Why do we need the four components in that order? As each plan, act, correct, and verify part has been actively investigated in the literature, the core contribution may come from how we combine them.
- L49 says that the proposed approach does not rely on oracle feedback, but L176 says it uses the simulator to check whether an action is executed successfully, which is contradictive. Using such GT action failure checking seems quite unrealistic.
- One argued contribution is MAP-THOR, but the current description (L207-L219) lacks the details of it. How many (training/evaluation) episodes are used? How different is MAP-THOR compared to previous benchmarks for various aspects? How to collect the dataset? Summarizing them in some tables with qualitative analysis might be helpful.
- What are the values in the parentheses in the tables? Describing what they are in the table captions might improve readability. Are they min/max values for the corresponding metrics?
  - If yes, then the variation seems quite high. For example, in Table 3, the min of "LLaMAR (w/ exploration)" is lower than the min of "LLaMAR (w/o exploration)," which raises a concern about the outperformance argued.
questions: questions: See weaknesses above.
limitations: limitations: The authors do not address the potential negative societal impact.
paper_summary: paper_summary: This paper explores the use of Language Models (LMs) to generate task plans for autonomous robots, emphasizing adaptability across diverse tasks without the need for extensive customization. The authors introduce LLaMAR, an LM-based agent framework that employs a “plan-act-correct-verify” strategy to manage long-horizon tasks in partially observable environments. The effectiveness of LLaMAR is assessed using MAP-THOR, a new benchmark featuring multi-agent scenarios and various household tasks at different complexity levels within the AI2-THOR environment. LLaMAR demonstrates a higher success rate than competing models in these tests.
strength_weakness: strength_weakness: 1. The method’s reliance on multiple LM-based modules for planning and execution might introduce significant computational overhead and complexity.
2. The detailed, step-by-step reasoning process, while beneficial for accuracy and responsiveness, could incur high costs, especially when utilizing commercial LM APIs.
questions: questions: 1. The use of fine-tuned sentence-BERT for action parsing is innovative, but the paper should discuss potential performance issues and the implications of incorrect mappings more quantitatively, for example, by comparing with more general function calling in LLMs. 
2. As indicated in Algorithm 2, the corrector works after the robot performs the actions, which may cause safety problems in real-world experiments. For example, the place action is dangerous when the fridge is not open. Is it possible to pre-check the action before the actual execution? Moreover, solely relying on VLM as a verifier may be unreliable; more traditional perception tools (e.g., segmentation and detection) may help improve the system's reliability.
limitations: limitations: YES
paper_summary: paper_summary: This paper focuses on the long-horizon multi-agent planning tasks. They propose a new agent structure  composed of four language models (Planner, Actor, Corrector, Verifier), named Language-Model-based Long-Horizon Planner for Multi-Agent Robotics (LLaMAR). LLaMAR operates without prior knowledge or privileged information from the environment, making it suitable for practical robotic applications. Additionally, the authors develop a multi-agent planning benchmark upon the AI2-THOR simulator, encompassing 45 household tasks. Empirical results demonstrate that their method outperforms existing agent methods.
strength_weakness: strength_weakness: While the agent structure design is commendable, its contribution to the multi-agent setting appears limited. Firstly, the proposed benchmark focuses on evaluating the effectiveness and robustness of multi-agent planning, yet these tasks are not necessarily crucial for multi-agent cooperation. As shown in Table 5, increasing the number of agents does not lead to improvement and, in fact, results in a decline in almost all metrics when the number of agents increases from 3 to 5. Secondly, task allocation, which is critical for a centralized multi-agent system, is not thoroughly discussed in this paper.

Moreover, considering the overall performance is relatively poor, with at least one-third of the trials failing, more in-depth analyses of the failure cases would be expected.
questions: questions: 1. Since the agent structure can be easily adapted to a single-agent setting, a comparison between single-agent and multi-agent systems would be valuable to demonstrate the effectiveness of using a multi-agent system for these tasks.

2. I would expect more in-depth analyses of the failure cases, such as examining the behavior of different methods across various categorized tasks and identifying the causes of the significant gap between the baselines and the proposed method. Additionally, it would be helpful to understand the conditions under which the proposed method tends to fail.

3. Some visualized examples would be helpful for understanding how the agents cooperate to complete long-horizon tasks.
limitations: limitations: The authors have addressed the limitations of their work.
paper_summary: paper_summary: This paper presents a multi-agent planning and execution engine called LLaMaR that is based on several multi-modal foundation models such as GPT4V, i.e., large models  (LMs) that can reason about text and visual inputs. LLaMaR has 4 LMs: planner, actor, corrector, and verifier. The planner decomposes the task into subgoals. The actor chooses which subgoal to perform and which low-level actions to perform to achieve it. The corrector modifies the actions chosen by the actor in case the chosen action is known to have failed. The verifier verifies which subgoals have been achieved. No additional knowledge is given to LLaMaR. LLaMaR controls all agents in a centralized manner. 
The paper also introduce briefly a benchmark for evaluating LLaMaR which is based on the AI2Thor simulator, and shows that LLaMaR outperforms similar LM-based planners. The main novelty in LLaMaR is the corrector module, and the fact that it is designed for partially observable situations. The latter is addressed by allowing the agents to invoke an exploration strategy if the object they seek is unavailable.
strength_weakness: strength_weakness: 1.	While the title promises some multi-agent aspect, I didn’t see anything in LLaMaR that is specific for multi-agent planning. What about interactions between actions? communication issues? Or decomposition of the action-space for efficiency purposes? 
2.	The analysis of Table 5 is, as I see it, incorrect. See questions below. 
3.	LLaMaR is described in a hand-wavy and shallow manner. 
4.	The exploration strategy seems ad-hoc and domain specific.
questions: questions: 1.	The separation of the Actor and the Corrector seems artificial. What does the Actor know that the Corrector does not? Since the Corrector needs to be smart enough to know the subgoal, know past failures, and suggest a course of action, it seems it has all the input that the Actor has and more. 
2.	What elements of LLaMaR are really multi-agent? The agents are sharing information in a seamless way, and there is nothing LLaMaR does to avoid the combinatorial challenge of planning for multiple agents (huge number of actions, interactions between the agents). 
3.	You say that you didn’t use the Smart-LLM benchmark since some of the tasks did not support multiple agents. Wouldn’t it be good to at least use the tasks there that do support multiple agents?
4.	With respect to Table 5 and the impact of adding more agents, you wrote that the success rate “remains high” and the number of steps “generally decreases with more agents”. Yet when moving from 3 to 4 agents and from 4 to 5 agents, the success rate drops and the number of steps increases. Is the conclusions drawn from Table 5 indeed incorrect?
limitations: limitations: Not relevant
paper_summary: paper_summary: The paper introduces the Long-Horizon Planner for Multi-Agent Robotics (LLaMAR), a cognitive architecture leveraging Language Models (LMs) for planning tasks in partially observable environments. Unlike traditional methods, LLaMAR uses a plan-act-correct-verify framework, enabling real-time self-correction and task verification without relying on oracles or simulators. The system's effectiveness is demonstrated through the MAP-THOR benchmark, which includes various household tasks in the AI2-THOR environment. LLaMAR achieves a 30% higher success rate than other state-of-the-art LM-based multi-agent planners.
strength_weakness: strength_weakness: 1. As with most LLM-based TAMP work, how to scale to the real world remains a real concern.
2. The use of multiple LM-based modules for each decision step increases computational and time costs compared to simpler baselines.
3. Some other LLM-based planning framework also has a correction strategy [1]. This framework utilizes A* with symbol operators to verify the plan. Also, this strategy can ensure that the generated plan is the shortest, which is a limitation in this paper.
4. In the appendix, the number of actions used by these tasks is at most about 10. In complex real-world environments, LMs may need to select the required actions from hundreds of different actions according to different tasks. This complex framework may lead to problems such as difficulty in finding the optimal solution and high computational complexity.
5. Too many restrictions might limit the capabilities of LMs.

[1]. Li, Z., Yu, K., Cheng, S., & Xu, D. (2024). LEAGUE++: Empowering continual robot learning through guided skill acquisition with large language models. In ICLR 2024 Workshop on Large Language Model (LLM) Agents. Retrieved from https://openreview.net/forum?id=xXo4JL8FvV
questions: questions: 1. Do you have any strategies in mind for optimizing or reducing this computational overhead?
limitations: limitations: 1. How to scale to the real world remains a real concern.
2. Computational complexity is a big concern in more complex environments.
3. Too many restrictions might limit the capabilities of LMs.

Comments: title: Part 2
Thanks for your response!
Part 1
Thanks! some follow up concerns
Response by Reviewer nYvP
comment: Dear Reviewer Eixs,

Thanks a lot for acknowledging the rebuttal. We appreciate your time and efforts in reviewing our paper. Your feedback has been valuable and we will incorporate your suggestions in the final paper.
Thanks a lot for acknowledging the rebuttal! We try to elaborate on the multi-agent aspect of our paper below:

The ease of scaling our pipeline from the single to the multi-agent setting shows that our approach is generalizable enough to encompass these two settings without having to distinguish the two. There were still many non-trivial roadblocks to naively scaling to multiple agents that we had to address.

The core of our architecture relies on the planner & verifier after each iteration to output and update, respectively, a globally accessible set of open and closed high-level subtasks to accomplish a task. Reasoning from memory, the environment state, and the aforementioned list, the actor & corrector modules output the appropriate actions for each agent. Thus, the actor & corrector modules have flexibility in the how of action execution, while the planner & verifier have over the what.

Here we quickly discuss how our architecture is able to handle challenges that naturally arise from a multi-agent setting.
- Task allocation & environment non-stationarity
    - Our architecture is able to distinguish between the deconstruction of a task into subgoals required for long-term planning, and the details of action execution such as low-level causal dependencies (e.g. grab object before storing), assigning specific responsibilities to agents, etc.
    This allows for detours, self-correction, and changes in responsibilities between agents on-the-fly, while not compromising long-term planning and focused task completion.

These considerations are of pressing importance in the multi-agent setting especially where there are collisions/interference and heavy non-stationarity in the environment state.

- Computational Complexity
    - To resolve the problem of increasing computational complexity in long-term planning, we opt for on-the-fly action responses (done by the actor module). Namely, we avoid generating the entire sequence, or even a subset, of each agent's actions through time. This avoids the challenges of ensuring non-interference & proper collaboration far into the future. These apriori planning approaches that might work in the single-agent setting would lead to untameable complexity as more agents are added.

Our pipeline, we found, was a flexible solution that gave a structure for the LM to reason about multiple agents without explicitly doing so.

We very much agree that the addition of asynchrony or decentralization (not sharing observations) in planning poses a greater challenge and is a fruitful direction for future research.
- Search and Rescue (SAR) tasks: The rationale behind using the SAR environment was to showcase the effectiveness of our approach in scenarios where cooperation is strictly necessary (e.g. 2 agents are needed to carry a human), the success is sensitive to the time-optimality of the actions (they need to split the agents efficiently between the fires to avoid uncontrolled spread), and exploration is necessary to find the missing human. It is important to note that we propose the SAR environment as an additional benchmark to evaluate the flexibility of LLAMAR under these different constraints rather than an extraneous component of the MAP-THOR dataset. Our current environment serves as an abstraction of the core search and rescue challenges, but we can imagine a more extensive & realistic simulation so we do not propose to use it as a definitive benchmark for future experiments.

3. Thanks a lot for pointing out the Ok-robot paper! We were initially thinking of including a histogram of different failure cases in the revised paper. But as you pointed out, the Sankey diagram (as in Fig 4 of the Ok-robot paper) is more informative to guide future improvements. We will include this in our revised paper.
Thank you for the comprehensive response. Your rebuttal has addressed my concerns, so I will maintain my original positive rating.
Dear reviewer QApg,

Thank you for acknowledging the rebuttal. We are hoping to perform some real-world experiments, especially in the search-and-rescue setting. We appreciate your time and efforts in reviewing our paper. Your feedback has been valuable and we will incorporate your suggestions in the final paper.
Thanks for the rebuttal! I appreciate the explanations for the action space and add some limitations. Although this framework has not been proven to be used in real world, it has potential to solve this challenging problem.
Thanks a lot for your reply and your follow-up questions! 

1. We chose $L=30$ as our episode length for computational and budget reasons (to limit the GPT-4 API costs) to fit all the experiments we wanted to do and hence we did not extend the horizon. However, we ensured that all of the tasks were solvable by a single-agent human-controlled agent. The major difference between a human-controlled agent and an autonomous agent is that the visibility distance is set to 1.5m for the autonomous agents whereas it is not restricted for a human-controlled agent. A human can identify objects from quite far away (from the rendered images) and hence are able to efficiently navigate in the environment to interact with the objects whereas autonomous agents have to explore the environment. Thanks a lot for bringing up this point. We apologize for not including these details in the appendix.
2. We agree that a table clarifying the different scenario types in MAP-THOR would help the readers understand the rationale behind having them. Below is a description of the type of tasks in MAP-THOR along with the differences.
    1. Explicit item type, quantity, and target location (Type 1): Since the name and the quantity of the items to be interacted with and their exact target locations are explicitly defined, these tasks allow the agents to come up with an initial plan even without needing the agents to explore the environment. 
        
        Example: the task “put bread, lettuce, and fridge in the fridge” can be solved by creating an initial plan:
        
        - transport bread to fridge
        - transport lettuce to fridge
        - transport bread to fridge
        
        These types of tasks are used in the experiments in the CoELA and SmartLLM paper.  
        
    2. Explicit item type and target location but implicit item quantity (Type 2): Similar to Type 1, since the name of the items to be interacted with and their target locations are known, the agents can come up with an initial plan. But here, the number of such objects is not known apriori. Hence the agents will have to explore the environment to identify all such instances of these objects.
        
        Example: the task “put all vases on the table” can be partially solved by creating an initial plan:
        
        - transport vase to the table
        - explore the environment to search for more vases
    3. Explicit target location but implicit item types and quantity (Type 3): In these type of tasks, the agents cannot create an initial plan to solve the task as easily as Type 1 and Type 2. Here the names of the objects that it is supposed to interact with are not known apriori. The agents need to explore the environment such that the planner module can add subtasks that need to be satisfied.
        
        Example:  for the task “put all groceries in the fridge”, the agents have to first identify the relevant objects that fit the category of “groceries” and then add subtasks to interact with those objects appropriately. This showcases the effectiveness of the planner module, verifier module, and exploration module to update the open and closed subtasks list by efficiently exploring the environment.
        
    4. Implicit target location, item type, and quantity (Type 4): This is similar to Type 3 tasks where the objects it needs to interact with are not known apriori. The key difference is that the target locations are also not known in this case whereas it is explicitly defined in Type 4 tasks. This checks whether the underlying LM is able to identify the appropriate target locations for the objects that it needs to interact with. 
        
        Example: “Clear the table by placing the items in their appropriate positions”. In this task, the LM is supposed to first identify the objects on the table and their appropriate target locations.
        
        - transport bread to the fridge,
        - transport apple to the fridge
        - transport tomato to the fridge
        - transport knife to the cabinet/countertop/drawer
        - transport bowl to the cabinet
        - transport book to the bookshelf
        
        Note that the ambiguity of the tasks increases in each type of task.
Thank you for acknowledging the rebuttal and increasing the score. We appreciate your time and efforts in reviewing our paper. We will incorporate your suggestions into the final paper.
Thanks for the rebuttal!
I appreciate the justification for the components' execution order and the justification for the Corrector. 
Re. the multi-agent aspect I admit I'm not fully convinced. It seems this paper could have been written for a single-agent problem without any changes to the theory. Having all agents share observations means this is very close to a single-agent situation. In general, I feel the multi-agent aspect of this paper is not fully developed.
Thank you for your detailed response. The rebuttal addresses my concerns and thus I am happy to increase my score.
Thank you for your thorough response and the effort you've put into addressing my concerns. Based on the feedback and new results, I have a few follow-up questions:

1. Given that a 30-step episode may be insufficient for a single agent to complete the task, could you explain why this episode length was chosen? Additionally, considering the limited space in the current environments, which might hinder effective multi-agent cooperation, what would happen if the episode length were extended?
2. The inclusion of additional search-and-rescue experiments seems to have complicated the organization of the MAP-THOR benchmark. Could you clarify the number of scenario types included in MAP-THOR and highlight the main differences between them? A table would be particularly useful to clearly present the structure of MAP-THOR and the rationale behind it.
3. I appreciate your detailed feedback on the failure cases. To better guide future improvements, more quantitative analysis would be helpful. For instance, similar to Fig 4 in [1], such analysis could clarify the impact of the proposed modular system on overall performance and identify the most critical modules.

[1] Liu, Peiqi, et al. "Ok-robot: What really matters in integrating open-knowledge models for robotics." in RSS 2024.