Habitat 3.0: A Co-Habitat for Humans, Avatars, and Robots
Authors: Xavier Puig
Eric Undersander
Andrew Szot
Mikael Dallaire Cote
Tsung-Yen Yang
Ruslan Partsey
Ruta Desai
Alexander Clegg
Michal Hlavac
So Yeon Min
Vladimír Vondruš
Theophile Gervet
Vincent-Pierre Berges
John M Turner
Oleksandr Maksymets
Zsolt Kira
Mrinal Kalakrishnan
Jitendra Malik
Devendra Singh Chaplot
Unnat Jain
Dhruv Batra
Akshara Rai
Roozbeh Mottaghi
Venue: ICLR 2024 poster (2024)
Keywords: Embodied AI
Simulation
Primary area: datasets and benchmarks
TL;DR: We present Habitat 3.0: a simulation platform for studying collaborative human-robot tasks in home environments.

Abstract: We present Habitat 3.0: a simulation platform for studying collaborative human-robot tasks in home environments. Habitat 3.0 offers contributions across three dimensions: (1) Accurate humanoid simulation: addressing challenges in modeling complex deformable bodies and diversity in appearance and motion, all while ensuring high simulation speed. (2) Human-in-the-loop infrastructure: enabling real human interaction with simulated robots via mouse/keyboard or a VR interface, facilitating evaluation of robot policies with human input. (3) Collaborative tasks: studying two collaborative tasks, Social Navigation and Social Rearrangement. Social Navigation investigates a robot's ability to locate and follow humanoid avatars in unseen environments, whereas Social Rearrangement addresses collaboration between a humanoid and robot while rearranging a scene. These contributions allow us to study end-to-end learned and heuristic baselines for human-robot collaboration in-depth, as well as evaluate them with humans in the loop. Our experiments demonstrate that learned robot policies lead to efficient task completion when collaborating with unseen humanoid agents and human partners that might exhibit behaviors that the robot has not seen before. Additionally, we observe emergent behaviors during collaborative task execution, such as the robot yielding space when obstructing a humanoid agent, thereby allowing the effective completion of the task by the humanoid agent. Furthermore, our experiments using the human-in-the-loop tool demonstrate that our automated evaluation with humanoids can provide an indication of the relative ordering of different policies when evaluated with real human collaborators. Habitat 3.0 unlocks interesting new features in simulators for Embodied AI, and we hope it paves the way for a new frontier of embodied human-AI interaction capabilities. For more details and visualizations, visit: https://aihabitat.org/habitat3.

Published as a conference paper at ICLR 2024
HABITAT 3.0: A CO-HABITAT FOR HUMANS, AVATARS AND ROBOTS
Xavi Puig". Eric Undersander", Andrew Szot", Mikael Dallaire Cote", Tsung-Yen Yen Yang", Ruslan Partsey', Ruta Desai", Alexander William Clegg", Michal Hlavac, So Yeon Min, Vladimír Vondruš, Theophile Gervet, Vincent-Pieryy Berges, John M. Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rail, Roozbeh Mottaghi
ht http://athabitat..or//hhat
ABSTRACT
We present Habitat 3.0: simulation platform for studying collaborative human- robot tasks in home environments. Habitat 3.0 offers contributions across three dimensions: (1) Accurate humanoid' simulation: addressing challenges in mod- eling complex deformable bodies and diversity in appearance and motion, all while ensuring high simulation speed. (2) Human-in-thee
infrastructure: enabling real human interaction with simulated robots via mouse/keyboard or a VR interface, facilitating evaluation of robot policies with human input. (3) Collaborative tasks: studying two collaborative tasks, Social Navigation and Social Rearrangement. So- cial Navigation investigates robot's ability to locate and follow humanoid avatars in unseen environments, whereas Social Rearrangement addresses collaboration be- tween humanoid and robot while rearranging scene. These contributions allow us to study end-to-end learned and heuristic baselines for human-robot collabora- tion in-depth, well evaluate them with humans in the loop. Our experiments demonstrate that learned robot policies lead to efficient task completion when collaborating with unseen humanoid agents and human partners that might exhibit behaviors that the robot has not seen before. Additionally, we observe emergent be- haviors during collaborative task execution, such as the robot yielding space when obstructing a humanoid agent, thereby allowing the effective completion of the task by the humanoid agent. Furthermore, our experiments using the human-in-the-looe tool demonstrate that our automated evaluation with humanoids can provide an indication of the relative ordering of different policies when evaluated with real human collaborators. Habitat 3.0 unlocks interesting new features in simulators for Embodied AI, and we hope it paves the way for a new frontier of embodied human-AI interaction capabilities. The following video provides an overview of the framework: ht https: http://tiittrnrrteeeeee com msywwanz.
INTRODUCTION
Today's embodied AI agents are largely hermits existing within and navigating through virtual worlds as solitary occupants (Batra et al., 2020; Anderson et al., 2017; Zhu et al., 2017; Chen et al., 2020; Krantz et al., 2020; Ku al., 2020; Wani et al., 2020; Xia et al., 2020; Deitke et al., 2020; Gervet et al., 2023)². Even in virtual worlds that feature interactivity (Deitke et al., 2022; Szot et al., 2021; Gan et al., 2021; Mu et al., 2021), where agents manipulate and move objects, the underlying assumption is that changes to the environment occur solely due to the actions of this one agent. These scenarios have formed a solid foundation for embodied Al and have led to the development of new architectures (Chaplot et al., 2020; Brohan et al., 2023; Shah et al., 2023; Kotar et al., 2023), algorithms (Wijmans et al., 2019; Wortsman et al., 2019; Li et al., 2019; Huang et al., 2022), and
team, project leads. Work done FAIR, Meta. 'Throughout this paper, we use avatar or humanoid to refer to virtual people in simulation and human to
refer to real people in the world.
With few exceptions such Li al. (2021a).

---

Published as a conference paper at ICLR 2024
Figure Habitat 3.0: Embodied Al framework designed to facilitate simulation of human avatars and robotic agents within wide array of indoor environments.
results (Anderson et al., 2020; Kumar et al., 2021; Truong et al., 2022; Chebotar et al., 2018). However, they stand in stark contrast to the motivation behind the development of assistive robots, where the goal is to share the environment with human collaborators capable of modifying and dynamically altering it. We believe it is now time to more comprehensively study and develop social embodied agents that assist and cooperate with humans.
compelling Aimulation-to-realll
Consider the social robot shown in Fig.
Boston Dynamics Spot robot collaborating with humanoid in a household setting, rearranging objects while cleaning up the house. To succeed in such tasks, the agents need to efficiently divide the task based on their abilities and preferences while avoiding interference and respecting each other's 's space. For instance, the robot yields space to the humanoid by moving backward when passing through door (Fig. 1, right).
Training and testing such social agents on hardware with real humans poses inherent challenges, including safety concerns, scalability limitations, substantial cost implications, and the complexity of establishing standardized benchmarking procedures. A simulation platform can overcome these challenges; however, the development of collaborative human-robee simulation platform also comes with its own complexities. First, major challenge is the modeling of deformable human bodies synthesize realistic motion and appearance of the body parts. Second, to ensure the generalizability of human interaction models, it is crucial to incorporate diverse human behaviors and motions within the simulator. Finally, current state-of-the-art learning algorithms Embodied AI often require significant number of iterations converge, leading to long training times. Hence, optimization techniques in human motion generation and rendering become crucial for achieving learning convergence within manageable wall-clock times.
In this paper, we introduce Habitat 3.0 - simulator that supports both humanoid avatars and robots for the study of collaborative human-robot tasks in home-like environments.
Human simulation - Habitat 3.0 enables the following features for human simulation: (1) articulated skeletons with bones connected with rotational joints, facilitating fast collision testing, (2) surface "skin' mesh for high-fidelity rendering of the body, enhancing visual realism, (3) parameterized body models (by Pavlakos al. (2019)) generate realistic body shapes and poses, (4) library of avatars made from 12 base models with multiple gender representations, body shapes, and appearances, (5) motion and behavior generation policy, enabling the programmatic control of avatars for navigation, object interaction, and various other motions. Despite the emphasis on realism and diversity, the speed of our simulation remains comparable to scenarios involving non non-humanoid agents 199 FPS with humanoid and a robot compared to 1345 FPS with two robots). This is achieved through optimization techniques that involve caching human motion data collected from diverse human shapes. This data is subsequently adapted to new environments using projection techniques, resulting in simulation that is both fast and accurate, while respecting the geometry of the environment.
ciiman-in-the-lowe tool - Beyond humanoid simulation, pivotal feature of our system is human- in-the-loop tool, an interactive interface, facilitating the evaluation of AI agents with real human collaborators. Through this tool, humans collaborate with an autonomous robot using mouse and keyboard inputs or a virtual reality (VR) interface. This interactive setup enables us to evaluate the performance of learned AI agents within scenarios that closely resemble real-world interactions.
Social tasks - Aiming at reproducible and standardized benchmarking, we present two collaborative human-robot interaction tasks and suite of baselines for each. We 'lift' the well-studied tasks of
2

---

Published as a conference paper at ICLR 2024
navigation and rearrangement from the single-agent setting to dlmaa--sssistvee setting. The first task, Social Navigation, involves the robot finding and following a humanoid while maintaining safe distance. The second task, Social Rearrangement, involves a robot and humanoid avatar working collaboratively to rearrange a set of objects from their initial locations to desired locations, using series of pick-and-place actions (emulating the cleaning of house). In social navigation, the robot's objective is independent of the humanoid's 's goal, while in social rearrangement the robot and the human must coordinate their actions to achieve a common goal as efficiently as possible. We evaluate these tasks within an automated setting, where both humanoid avatars and robots operate based on their learned/scripted policies. Furthermore, we evaluate social rearrangement in the human- in-the-loop framework using subset of baselines, allowing real humans to control the humanoid avatar while the robot controlled using a learned policy.
We conduct an in-depth study of learned and heuristic baselines on both tasks, with a focus on generalization to new scenes, layouts and collaboration partners. In social navigation, we find that end-to-end RL learns collaborative behaviors such as yielding space to the humanoid, ensuring their unobstructed movement. In social rearrangement, learned policies efficiently split the task between the robot and the humanoid, even for unseen collaborators, improving efficiency over the humanoid operating alone. These findings extend to our human-in in-the-loop study, where social rearrangement baselines enhance humans' efficiency over performing the task alone.
In summary, we provide a platform for simulating humanoids and robots, thereby providing an environment for in-depth studies of social human-robot tasks. We hope that this platform will encourage greater engagement from the Embodied Al community in this important research direction. Our framework is open-sourced, for more details see Appendix A.
RELATED WORK
Embodied AI Simulators. Several simulators have been developed recently to support embodied
tasks (Savva et al., 2019; Xia et al., 2018; Wani et al., 2020; Ramakrishnan et al., 2021; Deitke et al., 2020; Chen et al., 2020; James et al., 2020; Yu et al., 2019; Mu et al., 2021; Zhu et al., 2020; Kolve et al., 2017; Ehsani et al., 2021; Weihs et al., 2020; Szot et al., 2021; Shen et al., 2021; Gan et al., 2021; Xiang et al., 2020; Deitke et al., 2022). In contrast to these works, we simulate human movements and interactions in addition to robot simulation. iGibson 2.0 (Li et al., 2021b) simulates humanoids in the context of social navigation task. However, humanoids are treated as rigid bodies that move on pre-defined tracks or rails. Building on iGibson 2.0, Srivastava et al. (2021) collect human demonstrations using a VR interface. VRKitchen (Gao et al., 2019) also provides such VR interface to collect demonstrations and benchmark to evaluate agents. These simulators focus on human teleoperation rather than humanoid simulation. Habitat 3.0 supports both humanoid simulation and human teleoperation. Overcooked-Ar (Carroll et al., 2019) is a cooking simulator that is designed for coordination with humans, but their environments are simplified grid-like 2D environments. VirtualHome (Puig et al., 2018 8; 2021) is a simulator which supports humanoid avatars interacting in 3D environments, but does not support interactions with robots. In contrast, simulate and model both robot and humanoid agents together and allow interactions between them. SEAN (Tsoi et al., 2022) supports both humanoid and robot simulation in 3D environments. However, it is limited to 3 scenes and primarily focused on social navigation. In contrast, we support object interaction in a wide range of indoor environments. Moreover, our simulator an order of magnitude faster than existing humanoid simulators (detailed comparison in Table 4 in the Appendix).
Human-in-the-Lon Loop Training& & Evaluation. Human-in-the-loop (HITL) training (Fan et al., 2022: Zhang et al., 2021; MacGlashan et al., 2017) and evaluation (Hu et al., 2020; Puig et al., 2021; Carroll et al., 2019; Puig al., 2023) have been studied in many different contexts such as NLP, computer vision, and robotics. Such studies in the Embodied AI domain are challenging, since they require an environment where both users and agents can interact. Prior works have focused on simplified grid or symbolic state environments (Carroll et al., 2019; Puig et al., 2021). A few works (Ramrakhya et al., 2022; Padmakumar et al., 2021; Das et al., 2018) collect user data in 3D environments at scale, but focus data collection and offline evaluation rather than interactive evaluation. We present HITL framework capable of interactive evaluation and data-collection.
Embodied Multi-agent Environments & Tasks. Most multi-agent reinforcement learning (MARL) works operate in a diwdeeniiional state space (c.g.. giid-like mazes, tabular tasks, or Markov games) (Jaderberg et al., 2019; Samvelyan et al., 2019; oai; Resnick et al., 2018: 8; Suarez et al., 2019;
3

---

Published as a conference paper at ICLR 2024
(a)
(b)
(d)
Figure 2: Humanoid Avatars. Visualization of the skeleton rig and skinned mesh (a). A subset of the sampled avatars featuring distinct genders, body shapes, and appearances (b). Simulation of realistic walking and reaching behaviors, which can adapt to different environment layouts (c,d).
Baker et al., 2020; Giles & Jim, 2002; Lazaridou et al., 2016; Foerster et al., 2016; Sukhbaatar et al., 2016; Mordatch & Abbeel, 2018). Different from these frameworks, Habitat 3.0 environments capture the realism necessary for household tasks. Closer to our work is prior work in visual MARL in 3D embodied settings: (Jain et al., 2019; 2020) introduce multiple agents in embodied simulation for moving furniture; Szot al. (2023) study collaborative rearrangement home environments, where two robots perform tasks such as tidying up, and setting dinner table. Thomason et al. (2020); Roman et al. (2020); Patel et al. (2021); Jain et al. (2021) 1) study heterogeneous agents with different capabilities towards solving shared task. In other visual MARL work such as (Chen et al., 2019; Juliani et al., 2018; Weihs et al., 2021 202;;;a; Kurach et al., 2020) agents compete (e.g., hide-and-seee in teams or individually. However, none of these prior works simulate humanoids. Wang et al. (2022) simulate humanoids and learn to generate human-robot collaborative behavior. In comparison, we scale each axis of simulation scene and environment variations, and realism, fidelity and diversity of humanoid models.
simulated
3 HABITAT 3.0 FRAMEWORK
We first explain the intricacies of humanoid simulation and how we address key challenges such as efficiency, realism, and diversity. In particular, we focus on a caching strategy, the key element for achieving high speeds in appearance and motion generation. Then, we present our human-in-the-loop tool that enables evaluating robot policies with real humans in simulation.
3.1 HUMAN SIMULATION
To enable robots that learn effectively interact with humans, we need build fast, diverse, and realistic human models that can be deployed and used to train agents in simulation. We describe here how we design these humanoid agents, considering two components: their appearance and motion/behavior.
Following the traditional graphics approaches (Kavan & Zára, 2005: Kavan et al., 2007), we represent humanoids an articulated skeleton, with bones connected via rotational joints (Fig. 2(a)). Additionally, a surface mesh is attached to the skeleton, and its vertices are updated based on the pose and shape of the skeleton using linear blend skinning (LBS)) (Fig. 2(a)). The skelcton is used to represent poses and check for collisions with the environment, whereas the skinned mesh provides visual fidelity without affecting physics. The choice of articulated skeleton for physics and a skinned mesh for appearance enhances the simulation speed without compromising fidelity. To generate realistic skeletons and meshes, rely SMPL-X (Pavlakos et al., 2019), data-driven parametric human body model that provides compact representation of 3D human shape and pose. In particular, SMPL-X represents a human's pose and shape into two sets of parameters J.3 for each gender. J E R209 encodes a subject's pose via the hands, body and face joint rotations, and | I. captures variations body shape through the principle components of mesh vertex displacements. This allows us to create realistic body shapes by randomly sampling B. To encode body deformations different poses, SMPL-X uses pese-dependent blend shapes, changing the relationship between the humanoid joints and skin mesh based on the pose. To enable fast humanoid simulation, we cache male, female and 4 neutral body shapes and store their corresponding skeleton and skinned meshes. This allows us to compute the model rig and blend shapes offline, only requiring LBS during simulation. Since the blend shapes are constant, we lose the ability to create pose-dependent skin deformations, resulting in loss of some visual fidelity. However, this trade-off allows for the efficient loading and reposing of humanoid models. Fig. 2(b) and the supplemental video show some of the generated humanoids.
Humanoid Appearance:
4

---

Published as a conference paper at ICLR 2024
Humanoid Motion and Behavior: Our humanoid avatars are capable of realistic, long-range motion and behavior that can adapt to different tasks and environments. To achieve this, we design a hierarchical behavior model, in which a learned policy planner executes sequence of low-level skills to move the humanoid, generating long-range behaviors. Since our objective is to enable humanoid models perform rearrangement and navigation tasks, we specifically consider two essential low-level skills: navigating around the environment and picking and placing objects'.
For navigating, we first use path planner to generate waypoints that reach the goal location without colliding with the environment. The humanoid then transitions between these waypoints by first rigidly rotating its base to face the next waypoint, and then moving forward. For animating the forward motion, we use walking motion clip from the AMASS dataset (Mahmood et al., 2019), trim it to contain a single walking cycle (after both the left and right foot have moved) and play cyclically until reaching the next waypoint (Fig. 2 (c)). For generating a picking motion, we use VPoser (Pavlakos et al., 2019) to pre-compute set of humanoid poses for each body model, with one of the hands reaching set of 3D positions around the human, while constraining the feet to be in a fixed location. We then store these poses offline, indexed by the 3D positions of the hand relative to the humanoid root. At evaluation time, get humanoid poses for reaching arbitrary 3D points by interpolating over the closest stored poses, without requiring to query VPoser. This allows fast simulation of pick/place motions. A continuous picking and placing motion is obtained by drawing a 3D line from the current position of the hand to the target 3D position, and retrieving poses for intermediate points on the line. Once the hand reaches the object position, we kinematically attach or detach the object to or from the hand A resulting motion can be seen in Fig. 2(d). While this approach generates smooth motions for the range of positions computed on VPoser, it fails for positions out of this range, which happens when the hand is either too close or too far from the body.
Benchmarking:
Our simulation design enables efficient humanoid simulation with minimal perfor- mance impact compared to simulating a robot. In our simulator, robot operates at 245 frames per second (FPS) in single environment, while the humanoid achieves 188 FPS. The difference in FPS is primarily due to the larger number of joints in the humanoid model, as compared to the robot. Removing skinming or motion capture data does not notably affect performance, implying that our simulation scales well to realistic visuals and animations. When using two agents, robot-robot achieves a frame rate of 13, while robot- humonoh achieves 136.s. Increasing the number of environments leads to substantial speed improvements, with robat-humanoid reaching 1191 FPS across 16 environments on a single GPU. Further details are in Appendix F.1.
3.2 HUMAN-IN-
INFRASTRUCTURE
We aim to study the generalization capabilities of our robot policies when interacting with collabora- tors that exhibit diverse behaviors, including real human collaborators. To facilitate numan-robor interaction, we have developed a Human-in iomt-ii-thhe -n-naan---the-LLog Loop (HITL) evaluation platform. This tool allows human operators to control the humanoids within the simulated environment, using a mouse/keyboar or VR interface, enabling online human-robot interaction evaluations and data-collection. Below, we highlight some of key features:
Our HITL tool leverages the infrastructure of AI Habitat simulator (Savva et al., 2019; Szot et al., 2021), enabling seamless integration and utilization of existing datasets and simulation capabilities. Further, we implement end-user logic in Python and provide convenient wrappers for the low-level simulation logic for ease of development.
Portability to Other OS/Hardware Platforms. We adopt client-server architecture for our HITL tool that enables portability to other OS/hardware platforms. The server handles all logic (e.g., simulation, agent inference, and avatar keyboard controls), while the client handles only platform-specife rendering and input-device handling. This allows running the compute-heavy server component on a powerful machine while porting the client to other platforms, e.g., resource- constrained web browsers and VR devices. Hence, users can use this tool for a range of tasks: small-scale local evaluations using mouse and keyboard, large-scale data-collection on a browser, or high-fidelity human-robot interaction in VR.
Replayability. To support data collection and reproducibility, our platform provides functionalities for recording and replaying of HITL episodes. The playback functionality supports different levels of
Whiile our focus is on restricted set of motions, the SMPI-XX format allows to add wide range of motions, such MoCap data, or the ones generated hy motion generation models (e.g., Tevet et al. (2023)).
5

---

Published as a conference paper at ICLR 2024
abstraction, ranging from high-level navigate/pick/place actions to precise trajectories of humanoid poses and rigid objects. Additionally, the platform provides the capability to re-render the entire episode from the viewpoints of different egocentric exocentric cameras.
4 INTERACTIVE HUMAN-ROBOL TASKS
We study two tasks in Habitat 3.0: social navigation and social rearrangement. In social navigation, robot must find and follow a humanoid, while maintaining safe distance. In social rearrangement, a humanoid and a robot collaborate to move objects from their initial to target locations in a scene. Social navigation requires the robot to achieve its objective without interfering with the humanoid, while in social rearrangement, the two agents must collaborate to achieve a shared goal.
4.1 SOCIAL NAVIGATION: FIND & FOLLOW PEOPLE
Task description. An assistive robot should be able execute commands such as "Bring me my mug". or "Follow Alex and help him in collecting the dishes", which require finding and following humans at a safe distance. With this in mind, we design a social navigation task (Francis et al., 2023), where a humanoid walks in a scene, and the robot must locate and follow the humanoid while maintaining a safety distance ranging from 1m to 2m. Our task is different from prior work on social navigation such Li et al. (202 la): Biswas et al. (2021) as they focus primarily on avoiding humans. also differs from Luo et al. (2018), which focuses on following and tracking humans, but does not consider exploration to locate the humans, or constraints of the robot embodiment.
Fig. 3 (left) illustrates the social navigation task. The robot is placed in an unseen environment, and tasked with locating and following the humanoid, using depth camera on its arm (arm depth), binary humanoid detector that detects if the humanoid is in the frame (humanoid detector), and the humanoid's 's relative distance and heading all times. This task is different from other navigation tasks such as image-goal (Zhu et al., 2017), point-goal (Anderson al., 2018), and object-goal (Batra et al., 2020) navigation since it requires reasoning about dynamic goal and dynamic obstacles. The robot must adapt its path based on the position of the humanoid and anticipate the humanoid's future motion avoid colliding with it. The episode terminates if the robot collides with the humanoid or reaches maximum episode length. The humanoid follows the shortest path to sequence of randomly sampled points in the environment.
the robot (humanoid GPS), available to the robot at
Metrics. We define 3 metrics for this task: (1) Finding Success (S): Did the robot find the humanoid within the maximum episode steps (reach within 1-2m while facing the humanoid)? (2) Finding Success Weighted by Path Steps (SPS): How efficient was the robot at finding the humanoid, relative to an oracle with ground-truie knowledge of the full humanoid trajectory and a map of the environment? Let / be the minimum steps that such oracle would take to find the humanoid, and p be the agent's path steps. SPS S max((.p) (3) Following Rate (F): Ratio of steps that the robot maintains a distance of 1-2m from the humanoid while facing towards it, relative to the maximum possible following steps. For an episode of maximum duration E, we assume that an oracle with ground-truft knowledge of humanoid path and the environment map can always follow the humanoid, once found. Thus, the following steps for such an oracle are E I, where is the minimum number of steps needed to find the humanoid. Let w denote the number of steps that the agent follows the humanoid, then following rate F max(E-(,w)) [,,). (4) Collision Rate (CR): Finally, we report the ratio of episodes that end in the agent colliding with the humanoid. More details about the metrics are in Appendix A.
we
Baselines. We compare two approaches in the social navigation task:
We create heuristic baseline with access the environment map that uses shortest path planner to generate a path to the humanoid's is current location. The heuristic expert follows the following logic: When the agent is farther than 1.5m from the humanoid, it uses the 'find' behavior, i.e., uses path planner to approach the humanoid. If the humanoid is within 1.5m, it uses a backup motion to avoid colliding with the humanoid.
Heuristic Expert:
End-to-end RL: A 'sensors-to-action' recurrent neural network policy, trained using DDPPO (Wi- jmans et al., 2019). Inputs to the policy are egocentric arm depth, a humanoid detector, and a humanoid GPS, and outputs are velocity commands in the robot's local frame. This policy does not have access to a map of the environment. Architecture and training details are in Appendix A. We also provide ablations where we study the impact of the different sensors input the policy.

---

Published as a conference paper at ICLR 2024
Humanoid sartt pos.
St
SPST
Ft
CR₁
Humanoid path
Heuristic Expert
1.00
0.97
0.51
0.52
Desector
Rebort pos
End-to-end RL
go.o
g.63
0.44gom
0.51.o000
Find Phose
humanoid GPS
0.76
0.341
0.291a.m
0.48_a.03
humanoid detector
0 0.98
0.68
0.3720.0
0.64.0.05
Humanuid
Found
depth
0.94
0.54
0.19
0.71=0.05
arm depth arm RGB
0.90go.co
0.61ga.0
0.38
0.56go.04
Follow Phaaoe
Social Navigation. Overview of the Social Navigation task and sensors used (left). Bascline Results (right). The bottom rows show different variations of removing sensors.
Figure 3:
Scenes and Robot. We incorporate the Habitat Synthetic Scenes Dataset (HSSD) (Khanna et al., 2023) in Habitat 3.0, and use the Boston Dynamics (BD) Spot robot. We use 37 train, 12 validation and 10 test scenes. For details the robot and scenes, refer to Appendix D and E.
Results. Fig. shows the performance of different baselines. The heuristic expert achieves 100% finding success (S) vs 97% for the RL policy. This is expected the heuristic has privileged access to map of the environment, and can compute the shortest path to the humanoid while the RL policy has to explore the environment. As result, the heuristic expert finding SPS is also higher than the RL policy. However, even without access to a map, the RL policy learns to follow the humanoid by anticipating their motion, backing up avoid collisions, and making way in narrow spaces, resulting in similar success (S) and collision rate (CR) as the heuristic expert (0.52 for heuristic vs 0.5 0.51+0.03 for RL policy), and competitive SPS and following rate F (0.51 for heuristic expert vs 0.44 for the RL policy). For qualitative results, refer to the supplementary video and Appendix C.I.
Ablations. We analyze the impact of individual sensors on the end-to-end RL policy's 's performance (bottom rows in Fig. 3). In general. the results can be grouped into two cases: before and after finding the humanoid. Before finding the humanoid, the humanoid GPS is the most important sensor since enables agents to locate the humanoid. As result, the baseline without the humanoid GPS has the lowest finding success and SPS. However, after finding the humanoid, arm perception (either RGB or depth) becomes important since the agent uses this information to avoid collision while following the humanoid. As a result, the baseline without arm cameras tends to have higher collision rate and lower following rate than the baselines with either depth or RGB sensor.
4.2 SOCIAL REARRANGEMENT
Task Description. In this task (Fig. 4, left), the robot's goal is to efficiently assist humanoid collaborator in rearranging two objects from known initial positions to designated goals. Object positions and goals are specified as 3D coordinates in the robot's start coordinate frame. We modify Social Rearrangement from Szot et al. (2023), generalizing it to heterogeneous agents (a humanoid and robot) and dealing with more diverse, unseen environments and humanoid collaborators. The objects are spawned on open receptacles throughout an unseen house, with multiple rooms, and assigned a goal on another open receptacle within the house. The robot has access to its own egocentric depth cameras, proprioceptive state information (arm joint angles and base egomotion), and the humanoid's relative distance and heading. However, does not have access to the humanoid's actions, intents, or complete states. Episodes end when all objects are placed in their desired locations or the maximum allowed steps are reached. During evaluation, the trained robot collaborates with new human humanoid partners in diverse, unseen homes, described later in this section.
Metrics. We evaluate all approaches on two metrics: (1) Success Rate (SR) at completing the task in unseen home, with an unseen configuration of objects. An episode is considered successful (SR 1) if both objects are placed at their target locations, within the maximum number of allowed episode steps. (2) Relative Efficiency (RE) task completion, relative to the humanoid doing the task alone. For an episode with maximum allowed steps E, if the humanoid alone takes I huuman huuman
steps, and the humanoid-robot team takes joint steps to finish the task, RE- maxtT F.j. If the humanoid-robot team finishes the task in half the steps as humanoid alone, RE will be 200%.
Policy architecture. We adopt two-layer policy architecture for all baselines, where a learned high-level policy selects a low-level skill from a pre-defined library of skills to execute based on observations. We consider 2 variants of low-level skills for the robot oracle and learned lowelevel skills. Oracle skills use privileged information, i.e., a map of the environment for navigation and perfect, 'instantaneous' pick/place skills. Learned low-level skills are realistic and do not assume privileged information, resulting in more low-level failures. Refer to Appendix A for more details. All humanoid low-level skills are as described in Section 3.1.
pre-trained and frozen, but
7

---

Published as a conference paper at ICLR 2024
Baselines. We study 3 different population- based approaches (Jaderberg al., 2019) at the social rearrangement task, where a single high-level robot policy is trained to coordinate with collaborators within 'population'. During training, for each episode, from the population, and the robot is trained to collaborate with it. The robot policy is trained using end-to-end RI., using the same policy architecture, training parameters and scenes across all approaches. The approaches differ only in the training population.
collaborator is randomly selected
Learn-Single In this baseline, we jointly learn a humanoid and a robot policy; the population consists of single humanoid policy, resulting in low training collaborator diversity.
Plan-Pop,; We consider four different population sizes, with collaborators driven by privileged task planners. This results in four baselines (indexed by p), consisting of population sizes of to 4. p = 1 is population of size 1, with a humanoid collaborator consistently rearranging the same object across all episodes. (2) p = 2 is a population of size 2, where each collaborator is responsible for one of the two objects. (3) p = 3 is population of size 3, where collaborators rearrange either one or both objects. (4) 4 is a size 4 population where collaborators rearrange either one, both or none of the objects. Note that in all baselines, there is only one humanoid in the scene with the robot at a time, randomly sampled from the population. We train a different robot policy per population, and present results on all four baselines.
Learn-Pop: In this baseline, instead of relying on a privileged planner form the population, we learn a population of collaborators, following the approach of population 2019). We randomly initialize 8 humanoid policies, considering that different initializations may lead to diverse final behaviors. During training, each episode pairs a randomly chosen humanoid policy from this population with the robot, and both are jointly trained at the collaborative task. This baseline examines the efficacy of random initialization at learning populations.
(Jaderberg et al.,
Evaluation population. We use the same agent embodiment and train/evaluation dataset as Social Navigation. However, social rearrangement evaluation also considers different collaborators, with different high-level behaviors. We create a population of 10 collaborators for zero-shot coordination (ZSC-pop-eval) by combining a subset of training collaborators from all baselines. Specifically, we collect 3 learned humanoid policies from the Learn-Singg -Sngle and Learn-Pop Pop baselines, and 4 planner- based collaborators (one from each population p in Plan-Popp). As a result, each baseline sees about 1/3 of the ZSC-eval collaborators during learning and needs to generalize to the remaining 2/3 of the population. We also evaluate each approach against its training population (rrain-pop-eval) but in unseen scenes and environment configurations. This lets us study the difference between collaborating with known collaborators (train-pop- pop-eval) and unseen collaborators (ZSC-pop-eval).
Results. Fig. 4 shows the results of train-pop and ZSC-pop evaluation of the different baselines. Learn-Single and Plan-Pop; result in the highest success rate when evaluated against their training population. This is expected since both are trained with only one partner, and hence have the simplest training setting. However, their SR significantly drops during ZSC-pop-eval (98.50% -> 50.9% and 91.2% + 50.4%). This is because these baselines are trained to coordinate with just a single partner, and hence have poor adaptability to partners that are different from their training partner. Amongst approaches trained with a population >> 1, Learn-por pop has the highest train-pop SR but poor ZSC-eval performance (92.2% + 48.5%), because of low emergent diversity in the learned population. In contrast, Plan-pop2,s, 4 have smaller drop in performance between train-pop and ZSC-pop, signifying their ability to adapt to unseen partners. In particular, Plan-pops.4 perform similarly, with the highest ZSC-pop-eval SR of 71.7%. We observe similar trends in RE, where Learn-Single has the highest RE with train-pop. and a drop with ZSC-pop (159.2 2 + 106.0). Plan- pop3,4 have lower drop (105.49 + 101.99 for Plan-pop.) but on average result in similar RE as Learn-Singla and Plan-Pop; in ZSC-pop-eval This is because on average over the 10 ZSC partners, Learn-Single significantly improves efficiency for some, while making others inefficient, as seen by its large variance. On the other hand, Plan-pops.4 are able to adapt and make most partners slightly more efficient, as shown by their lower variance, but on average perform similar to Learn-Single. For more details and additional analysis, refer to Appendix C.
Ablations. We conduct ablation experiments over inputs to the robot policy, trained with a Plan-Pops population (Fig. 4 (right), bottom rows). We replace 'oracle' skills with non-privileged learned low-level skills without re-training the high-level policy. We find a notable drop in performance, stemming from level execution failures (77.79% 41.96% in train-pop-eval SR), indicating that the performance can be improved by training the high-level policy with learned skills to better adapt such failures. We also retrain our policies with RGB instead of depth, and do not observe
8

---

Published as a conference paper at ICLR 2024
Humanoid start pos.
Method
ESC-pop-eval
SRT
RE
SRT
RE7
Humanoid path
Learn Single
98. =0
159. 159.2.1.0
50.94
106.02:34.a0
Robot start pos.
Plan-Pop:
91 22222.6
152.4+5.4
50.44-20.02
109.75=33.00 08.08
Plan- Plan-Pops
66. 80 2:.4
110.06=a.88
70.23
102.13
Robot path
Plan-Pops
17.79
118.95.m.0s
71.79.7.55
101.99 15.11
Plan Plan-Pup; Pup,
72.42
105.4921.7
71.32
103.53_
Object start pos.
Learn Pop
92. g2.20
135.32g4.48
48.52_36.61
90.80=a1.0
onacle learned skills
2210
79.6221.76
21.44_18.16
76.45.00.23
Target pos.
depth + RGB
70.70
110.0423.05
70.89=a.1s
100. 2624.77
Humamoid-GPS
76.15
108.90 108.96_2.0
08.70
98.58.10.55
Figure 4: Overview of social rearrangement (left). Baseline results, averaged over seeds (right).
Method
CR L
TS
RC
RET
Mcan
95% CI
Mcan
95% CI
Mean
95% CI
Solo
0.0
1253.17
[1020.90 1533.26]
100.0
Learn Single
0.12
[0.19
936.60
[762.50 11446.2]
0.36
[0.33 0.39]
133,80
Plan-Pops
0.13
[0.20 0.08]
1015.05
[826.42 1242.60)
0.44
[0.41 0.46
123.46
Table 1: Human-in-the-Loop Coordination Results. We report estimated mean and 95% confidence
intervals (CT) across 30 participants.
drop in performance. Removing the humanoid- results in slight drop in SR and RE, indicating that this sensor is useful for collaboration, especially in the ZSC setting, though not essential.
4.3 HUMAN-IN-THE-LOOG EVALUATION
We test trained robotic agents' ability to coordinate with real humans via our human-in tuman-in-the-loop thop (HITL) tool across 30 participants. After brief keyboard/mouse control training, we ask the participants to perform the social rearrangement task in the test scenes from our dataset. Particularly, the study operates under 3 conditions: performing the task alone (solo), paired with robot operating with Learn-Single, or paired with a Plan-Pops agent. Each participant performs the task for 10 episodes per condition in one of the test scenes. We measure the collision rate (CR), task completion steps (TS), ratio of task completed by the robot (RC), and Relative Efficiency (RE) across all episodes. RE is the same in Sec 4.2. CR is the ratio of episodes where the robot collides with the humanoid. RC is the proportion of objects that were rearranged by the robot per episode.
Both Plan-Pop3 and Learn-Single improve RE to 123% and 134% respectively (Tab. 1). This shows that the robot makes the human more efficient than the human operating alone, even for completely unseen real human partners. Our analysis shows that pairwise difference of the estimated mean for TS between solo and Learn-Single, and solo and Plan-Pop, was significant, but the difference between estimated mean TS of Learn-Single and Plan-Ppp, was not (details in Appendix G). Plan-Pop; leads to higher task offloading as measured by RC despite having lower RE than Learm-single
In general, we observe that humans are more reactive to robot behavior than ZSC agents, which leads to success rate of I across all episodes and high RE. For example, humans quickly adapt their plan based on their inferred goal of the robot, or move out of the way to let the robot pass. However, the relative order of average RE and RC between automated and HITL evaluations holds, wherein Learn-Single makes the partner more efficient than Plan-Pops, but Plan-Popa has higher RC than Learn-Single. This reveals few interesting insights: (I) The automated evaluation pipeline can give an indication of the relative ordering of different approaches when evaluated with real human partners. (2) Our ZSC agents do not accurately capture the dynamics of human-robot interaction, and there is room for improvement. (3) Even approaches such as
which do not use diverse training population, can enhance human efficiency compared to performing a task alone.
5 CONCLUSION
We introduce Habitat 3.0, an Embodied AI simulator designed to efficiently simulate humanoids and robots within rich and diverse indoor scenes. Habitat 3.0 supports a diverse range of appearances and motions for humanoid avatars, while ensuring realism and fast simulation speeds. In addition to humanoid simulation, we provide an infrastructure for numan-in-the-loop (HITL) control of humanoid avatars, via a mouse/keyboard or VR interface. This interface allows us to collect real human-robot interaction data in simulation and evaluate robot policies with real humans. These capabilities allow us to study two collaborative tasks social navigation and social rearrangement in both automated and human-in- in-the-loop evaluation settings. We observe emergent collaborative behaviors in our learned policies, such as safely following humanoids, making them more efficient by splitting tasks. Our HITL analysis reveals avenues for enhancing social embodied agents, and we hope that Habitat 3.0 will accelerate future research in this domain.

---

Published as a conference paper at ICLR 2024
6 ACKNOWLEDGEMENTS
We thank the reviewers for their helpful suggestions. We would also like to thank Manolis Savva and Angel Chang for their support in developing and curating the HSSD scenes used in this work. We are also grateful to Meshcapade GmbH for their support in incorporating and releasing the SMPL-X body models and sample motions represent humanoids Habitat.
REFERENCES
Openai five. http:////pppaaacccccmeeeeerraaap
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian D. Reid, Stephen Gould, and Anton van den Hengel. Vision-and-language navigation: Interpreting visually- grounded navigation instructions in real environments. In CVPR, 2017.
Peter Anderson, Angel Chang. Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, et al. On evaluation of embodied navigation agents, arXiv, 2018. 6, 16, 19
Peter Anderson, Ayush Shrivastava, Joanne Truong, Arjun Majumdar, Devi Parikh, Dhruv Batra, and Stefan Lee. Sim-to-real transfer for vision-and-language navigation. In CoRL, 2020.
Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. Emergent tool use from multi-agent autocurricula. In ICLR, 2020. 4
Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 2015. 30
Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Oleksandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, and Erik Wijmans. Objectnav revisited: On evaluation of embodied agents navigating to objects. arXiv, 2020. 1, 6, 16
Abhijat Biswas, Allan Wang, Gustavo Silvera, Aaron Steinfeld, and Henny Admoni. Socnavbench:
A grounded simulation testing framework for evaluating social navigation. THRI, 2021.
James V Bradley. Complete counterbalancing immediate sequential effects a latin square design. Journal of the American Statistical Association, 1958. 29
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alexander Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil Joshi, Ryan C. Julian, Dmitry Kalashnikov, Yuheng Kuang. Isabel Leal, Kuang Huei Lee, Sergey Levine, Yao Lu, Utsav Malla, Deeksha Manjunath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell Quiambao, Kanishka Rao, Michael S. Ryoo, Grecia Salazar, Pannag R. Sanketi, Kevin Sayed, Jaspiar Singh, Sumedh Anand Sontakke, Austin Stone, Clayton Tan, Huong Tran, Vincent Vanhoucke, Steve Vega, Quan Ho Vuong, F. Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt-l: Robotics transformer for real-world control at scale, In RSS, 2023.
Berk Calli, Arjun Singh, James Bruce, Aaron Walsman, Kurt Konolige, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. Yale-cmu-berkeler dataset for robotic manipulation research. IJRR, 2017. 26
Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, and Anca Dragan. On the utility of learning about humans for human-ai coordination. In NeurlPS, 2019.
Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, and Ruslan Salakhutdinov. Object goal navigation using goal-oriented semantic exploration. In NeurIPS, 2020. 1
Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan D. Ratliff, and Dieter Fox. Closing the sim-to-real loop: Adapting simulation randomization with real world experience. In ICRA, 2018. 2
10

---

Published as a conference paper at ICLR 2024
Boyuan Chen, Shuran Song, Hod Lipson, and Carl Vondrick. Visual hide and seek. arXiv, 2019. 4
Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Kr- ishna Ithapu, Philip Robinson, and Kristen Grauman. SoundSpaces: Audio-visur navigation in 3d environments. In ECCV, 2020. 3,3
Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Embodied question answering. In CVPR, 2018. 3
Matt Deitke, Winson Han, Alvaro Herrasti, Aniruddha Kembhavi, Eric Kolve, Roozbeh Mottaghi, Jordi Salvador, Dustin Schwenk, Eli VanderBilt, Matthew Wallingford, Luca Weihs, Mark Yatskar, and Ali Farhadi. Robothor: An open simulation-to-real embodied ai platform. In CVPR, 2020. 1,3
Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Jordi Salvador, Kiana Ehsani, Winson Han, Eric Kolve, Ali Farhadi, Aniruddha Kembhavi, and Roozbeh Mottaghi. ProcTHOR: Large-Scale Embodied AI Using Procedural Generation. In NeurlPS, 2022. 1,3
Kiana Ehsani, Winson Han, Alvaro Herrasti, Eli VanderBilt, Luca Weihs, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. ManipulaTHOR: A Framework for Visual Object Manipulation. In CVPR, 2021. 3
Xiang Fan, Yiwei Lyu, Paul Pu Liang, Ruslan Salakhutdinov, and Louis-Philippe Morency. Nano: Nested suman-in-the-lope reward learning for few-shot language model control. arXiv, 2022. 3
J. N. Foerster, Y. M. Assael, N. de Freitas, and S. Whiteson. Learning to Communicate with Deep Multi-Agent Reinforcement Learning. NeurIPS, 2016.
Anthony Francis, Claudia Pérez-d' Arpino, Chengshu Li, Fei Xia, Alexandre Alahi, Rachid Alami, Aniket Bera, Abhijat Biswas, Joydeep Biswas, Rohan Chandra, et al. Principles and guidelines for evaluating social robot navigation algorithms. arXiv,, 2023. 6
Chuang Gan, Jeremy Schwartz, Seth Alter, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megumi Sano, Kuno Kim, Elias Wang, Damian Mrowca, Michael Lingelbach, Aidan Curtis, Kevin T. Feigelis, Daniel Bear, Dan Gutfreund, David Cox, James J. DiCarlo, Josh H. McDermott, Joshua B. Tenenbaum, and Daniel L. K. Yamins. Threedworld: A platform for interactive multi-modal physical simulation. In NeurIPS Datasets and Benchmarks Track, 2021. 3,3
Xiaofeng Gao, Ran Gong, Tiammin Shu, Xu Xie, Shu Wang, and Song-Chun interactive 3d virtual environment for task-oriented learning. arXiv, 2019.
Chun Zhu. Vrkitchen: an
Theophile Gervet, Soumith Chintala, Dhruv Batra, Jitendra Malik, and Devendra Singh Chaplot. Navigating to objects in the real world. Science Robotics, 2023.
C. L. Giles and K. C. Jim. Learning communication for multi-agen systems. In Proc. Innovative Concepts for Agent- Based Systems, 2002. 4
Jiayuan Gu, Devendra Singh Chaplot, Hao Su, and Jitendra Malik. Multi-skill mobile manipulation for object rearrangement. In ICLR, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, 2016. 17, 20
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 1997. 16
Hengyuan Hu, Adam Lerer, Alex Peysakhovich, and Jakob Foerster. "other-play" for zero-shot coordination. In ICML, 2020. 3
Wenlong Huang, Xia, Ted Xiao, Harris Chan, Jacky Liang, Peter R. Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Noah Brown, Tomas Jackson, Linda Luu, Sergey Levine, Karol Hausman, and Brian Ichter. Inner monologue: Embodied reasoning through planning with language models. In CoRL, 2022.

---

Published as a conference paper at ICLR 2024
Max Jaderberg, Wojeiech M. Czarnecki, lain Dunning, Luke Marris, Guy Lever, Antonio García Cas- tañeda, Charlic Beattie, Neil C. Rabinowitz, Ari S. Morcos, Avraham Ruderman, Nicolas Sonnerat, Tim Green, Louise Deason, Joel Z. Leibo, David Silver, Demis Hassabis, Koray Kavukcuoglu, and Thore Graepel. Human-level performance in 3d multiplayer games with population- reinforcement learning. Science, 2019. 3,
Unnat Jain, Luca Weihs, Eric Kolve, Mohammad Rastegari, Svetlana Lazebnik, Ali Farhadi, Alexan- der G. Schwing, and Aniruddha Kembhavi. Two body problem: Collaborative visual task comple- tion. In CVPR, 2019. 4
Unnat Jain, Luca Weihs, Eric Kolve, Ali Farbadi, Svetlana Lazebnik, Aniruddha Kembhavi, and Alexander Schwing. cordial sync: Going beyond marginal policies for multi-agent embodied tasks. In ECCV, 2020. 4
Unnat Jain, lou-Jen Liu, Svetlana Lazebnik, Aniruddha Kembhavi, Luca Weihs, and Alexander G Schwing. Gridtopix: Training embodied agents with minimal supervision. In ICCV, 2021. 4
Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 2020. 3
Arthur Juliani, Vincent-Pierre Berges, Ervin Teng, Andrew Cohen, Jonathan Harper, Chris Elion, Chris Goy, Yuan Gao, Hunter Henry, Marwan Mattar, Unity: A general platform for intelligent agents. arXiv, 201 8. 4
Maurits Kaptein. Using generalized linear (mixed) models in hei. Modern Statistical Methods for HCI, 2016. 29
Ladislav Kavan and Jifíí Zára. Spherical blend skinning: a real-time deformation of articulated models. In Symposium on Interactive 3D Graphics and Games, 2005. 4
Ladislav Kavan, Steven Collins, Jifí Zára, and Carol O' O'Sullivan. Skinning with dual quaternions. In Symposium on Interactive 3D Graphics and Games, 2007. 4
Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X. Chang, and Manolis Savva. Habitat synthetic scenes dataset (hssd-200); An analysis of 3d scene scale and realism tradeoffs for objectgoal navigation. arXiv,, 2023. 7, 26, 28
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXivv, 2014. 17
Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi. Ai2-thor: An interactive 3d environment for visual arXiv, 2017.
Klemen Kotar, Aaron Walsman, and Roozbeh Mottaghi. Entl: Embodied navigation trajectory learner. In ICCV, 2023.
Jacob Krantz, Erik Wijmans, Arjun Majundar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision and language navigation in continuous environments. In ECCV, 2020.
Alexander Ku, Peter Anderson, Roma Patel, Eugene Ic, and Jason Baldridge. caaomm-cross-ooooo navigation with dense spatiotemporal grounding. In EMNLP,
Multilingual
2020. 1
Ashish Kumar, Zipeng Fu, Deepak Pathak, and Jitendra Malik, Rma: Rapid motor adaptation for legged robots. In RSS, 2021.
Karol Kurach, Anton Raichuk, Piotr Stańczyk. Michal Zajkac, Olivier Bachem, Lasse Espeholt, Carlos Riquelme, Damien Vincent, Marcin Michalski, Olivier Bousquet, et al. Google research football: novel reinforcement learning environment. In AAAI, 2020. 4
Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. Multi Multi-agent cooperation and the emergence of (natural) language. arXiv, 2016. 4
12

---

Published as a conference paper at ICLR 2024
Chengshu Li, Fei Xia, Roberto Martin Martin, and Silvio Savarese. Hrl4in: Hierarchical reinforcement learning for interactive navigation with mobile manipulators. CoRL, 2019.
Chengshu Li, Jaewoo Jang, Fei Xia, Roberto Martín-Martin Claudia D D'Arpino, Alexander Toshev, Anthony Francis, Edward Lee, and Silvio Savarese. iGibson Challenge 2021. bttps://svll stanford. edu/ i igibson www.chaallaaaae html, 2021a. 1,
Chengshu Li, Fei Xia, Roberto Mart in-Mart'in in, Michael Lingelbach, Sanjana Srivastava, Bokui Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, Andrey Kurenkov, Karen Liu, Hyowon Gweon, Jiajun Wu, Li Fei-Fei, and Silvio Savarese. igibson 2.0: Object-centric simulation for robot learning of everyday household tasks. In CoRI., 2021b.
Wenhan Luo, Peng Sun, Fangwei Zhong, Wei Liu, Tong Zhang, and Yizhou Wang. End-to-end active object tracking via reinforcement learning. In International conference on machine learning, pp. 3286-3295. PMLR, 2018. 6
James MacGlashan, Mark K. Ho, Robert Loftin, Bei Peng. Guan Wang. David L., Roberts, Matthew E. Taylor, and Michael L. Littman. Interactive learning from policy-dependent human feedback. In ICML, 2017.
Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black. Amass: Archive of motion capture as surface shapes. In ICCV, 2019.
Igor Mordatch and Pieter Abbeel. Emergence of Grounded Compositional Language in Multi-Agent Populations. In AAAI, 2018. 4
Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang. Xuanlin Li, Stone Tao, Zhiao Huang, Zhiwei Jia, and Hao Su. ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations. In NeurIPS Datasets and Benchmarks Track, 2021. 1,3
Aishwarya Padmakumar, Jesse Thomason, Ayush Shrivastava, Patrick Lange, Anjali Narayan- Chen, Spandana Gella, Robinson Piramithu, Gokhan Tur, and Dilek Z. Hakkani-Tür, Teach: Task-driven embodied agents that chat. AAAI, 2021.
Shivansh Patel, Saim Wani, Unnat Jain, Alexander G. Schwing, Svetlana Lazebnik, Manolis Savva, and Angel Chang. Interpretation of emergent communication in heterogeneous collaborative embodied agents. In ICCV, 2021. 4
Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3D hands, face, and body from single image. In CVPR, 2019. 2, 4, 5,
Xavier Puig, Kevin Kyunghwan Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating household activities via programs. CVPR, 2018.
Xavier Puig, Tianmin Shu, Shuang Li, Zilin Wang, Yuan-Hong Liao, Joshua B. Tenenbaum, Sanja Fidler, and Antonio Torralba. Watch-and-help: challenge for social perception and human-AI collaboration. In ICLR, 2021.
Xavier Puig, Tianmin Shu, Joshua B. Tenenbaum, and Antonio Torralba. Nopa: Neurally-guided online probabilistic assistance for building socially intelligent home assistants. In ICRA, 2023. 3
Santhosh K. Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alexander Clegg, John Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel Xuan Chang, Manolis Savva, Yili Zhao, and Dhruv Batra. Habitat-matte mattetttt 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. In NewrIPS Datasets and Benchmarks Track, 2021.
Ram Ramrakhya, Eric Undersander, Dhruv Batra, and Abhishek Das. Habitat-web: Learning embodied object-search strategies from human demonstrations scale. In CVPR, 2022. 3
Cinjon Resnick, Wes Eldridge, David Ha, Denny Britz, Jakob Foerster, Julian Togelius, Kyunghyun Cho, and Joan Bruna. Pommerman: A multi-agent playground. arXiv, 2018. 3
13

---

Published as a conference paper at ICLR 2024
Homero Roman Roman, Yonatan Bisk, Jesse Thomason, Asli Celikyilmaz, and Jianfeng Gao. Rmm: A recursive mental model for dialog navigation. In EMNLP Findings, 2020.
Mikayel Samvelyan, Tabish Rashid, Christian Schroeder De Witt, Gregory Farquhar, Nantas Nardelli, Tim GJ Rudner, Chia-Man Hung, Philip HS Torr, Jakob Foerster, and Shimon Whiteson. The starcraft multi-agent challenge. arXiv, 2019.
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub. Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A platform for embodied ai research. /CCV, 2019. 3,5
Dhruv Shah, Ajay Kumar Sridhar, Nitish Dashora, Kyle Stachowicz, Kevin Black, Noriaki Hirose, and Sergey Levine. Vint: A foundation model for visual navigation. arXiv, 2023.
Bokui Shen, Fei Xia, Chengshu Li, Roberto Mart Marttin-Mart'in, in, Linxi (Jim) Fan, Guanzhi Wang, S. Buch, Claudia. Pérez D' Arpino, Sanjana Srivastava, Lyne P. Tchapmi, Micael Edmond Tchapmi, Kent Vainio, Li Fei-Fei, and Silvio Savarese. igibson, a simulation environment for interactive tasks in large realistic scenes. In IROS, 2021. 3
Spot. Spot robot. https:: wwwp http::://wwwwbottooooooeoeoc
wwm/ httn///oducts//pppee 7, 26
Sanjana Srivastava, Chengshu Li, Michael Lingelbach, Roberto Mart' Mart'in-Mart') in, Fei Xia, Kent Vainio, Zheng Lian, Cem Gokmen, Buch, C. Karen Liu, Silvio Savarese, Hyowon Gweon, Jiajun Wu, and Li Fei-Fei. Behavior: Benchmark for everyday household activities in virtual, interactive, and ecological environments. In CoRL, 2021.
Joseph Suarez, Yilun Du, Phillip Isola, and Igor Mordatch. Neural mmo: A massively multiagent game environment for training and evaluating intelligent agents. arXiv, 2019. 3
Sukhbaatar, A. Szlam, and R. Fergus. Learning multiagent communication with backpropagation. In NeurIPS, 2016. 4
Andrew Szot, Alexander Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Singh Chaplot, Oleksandr Maksymets, Aaron Gokaslan, Vladimir Vondrus, Sameer Dharur, Franziska Meier, Wojeiech Galuba, Angel Xuan Chang, Zsolt Kira, Vladlen Koltun, Jitendra Malik, Manolis Savva, and Dhruv Batra. Habitat 2.0: Training home assistants to rearrange their habitat. NeurlPS, 2021. 1, 3,5
Andrew Szot, Unnat Jain, Dhruv Batra, Zsolt Kira, Ruta Desai, and Akshara Rai. Adaptive coordina- tion for social embodied rearrangement, In ICML, 2023. 4, 7
Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffusion model. In ICLR, 2023.
Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. Vision-and-dialog naviga- tion. In CoRL, 2020. 4
Joanne Truong, Max Gustav Rudolph, Naoki Yokoyama, S. Chernova, Dhruv Batra, and Akshara Rai. Rethinking sim2real: Lower fidelity simulation leads to higher sim2real transfer in navigation. In CoRL, 2022. 2
Nathan Tsoi, Alec Xiang, Peter Yu, Samuel S. Sohn, Greg Schwartz, Subashri Ramesh, Mohamed Hussein, Anjali W. Gupta, Mubbasir Kapadia, and Marynel Vázquez. Sean 2.0: Formalizing and generating social situations for robot navigation. IEEE Robotics and Automation Letters, 2022.
Chen Wang, Claudia Pérez-D' Arpino, Danfei Xu, Li Fei-Fei, Karen Liu, and Silvio Savarese. Co-gail: Learning diverse strategics for human-robo collaboration. In CoRL, 2022.
Saim Wani, Shivansh Patel, Unnat Jain, Angel X. Chang, and Manolis Savva. Multi-ON; Benchmark ing Semantic Map Memory using Multi- Objectt Navigation. In NeurIPS, 2020. 1.
Luca Weihs, Jordi Salvador, Klemen Kotar, Unnat Jain, Kuo-Hao Zeng, Roozbeh Mottaghi, and Aniruddha Kembhavi. Allenact: A framework for embodied ai research. arXiv, 2020. 3
14

---

Published as a conference paper at ICLR 2024
Luca Weihs, Unnat Jain, Jordi Salvador, Svetlana Lazebnik, Aniruddha Kembhavi, and Alexander Schwing. Bridging the imitation gap by adaptive insubordination. In NeurIPS, 2021a. 4
Luca Weihs, Aniruddha Kembhavi, Kiana Ehsani, Sarah M Pratt, Winson Han, Alvaro Herrasti, Eric Kolve, Dustin Schwenk, Roozbeh Mottaghi, and Ali Farhadi. Learning generalizable visual representations via interactive gameplay. In ICLR, 2021b. 4
Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames. In ICLR, 2019. 1, 6, 16, 17
Mitchell Wortsman, Kiana Ehsani, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Learning to learn how to learn: Self-adaptive visual navigation using meta-learning. CVPR, 2019. 1
Fei Xia, Amir R Zamir, Zhiyang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. Gibson env: Real-world perception for embodied agents. In CVPR, 2018. 3
Fei Xia, William B Shen, Chengshu Li, Priya Kasimbeg, Micael Edmond Edmond Tchapmi, Alexander Toshev, Roberto Martín-Martin, and Silvio Savarese. Interactive gibson benchmark: A benchmark for interactive navigation in cluttered environments. RA-L, 2020.
Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. C.Canng, Leonidas Guibas, and Hao Su. SAPIEN: A SimulAted Part-based Interactive ENvironment. In CVPR, 2020.
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In CoRI, 2019.
Lvmin Zhang, Xinrui Wang, Qingnan Fan, Yi Ji, and Chunping Liu. Generating manga from illustrations via mimicking manga creation workflow. In CVPR, 2021. 3
Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi. Target-driven visual navigation in indoor scenes using deep reinforcement learning. In ICRA, 2017. 1, 6
Yuke Zhu, Josiah Wong. Ajay Mandlekar, Roberto Martín-Martin, Abhishek Joshi, Soroush Nasiriany, and Yifeng Zhu. robosuite: A modular simulation framework and benchmark for robot learning. In arXiv, 2020. 3
15

---

Published as a conference paper at ICLR 2024
APPENDIX
A IMPLEMENTATION DETAILS
We include the implementation details necessary for reproducing results for the tasks de- scribed in Section 4. Code for reproducing experimental results can be found here: https://githube http///ootteeaaacccaaaaaaaeeeaaa
habitat-baselime ines -aabitttt--mmll-aaaaaeee-ee -aaaaaa--------------------aaaa-
A.1 SOCIAL NAVIGATION
The robot uses neural network policies to find the humanoid, and the humanoid is scripted to navigate to random waypoints using a shortest path planner. We train all the end-to-end RL social navigation baselines using DD-PPO (Wijmans al., 2019), distributing training across 4 NVIDIA A100 GPUs. Each GPU runs 24 parallel environments, and collects 128 steps for each update. We use a long short-term memory networks (LSTM) (Hochreiter & Schmidhuber, 1997) policy with ResNet18 as the visual backbone and two recurrent layers, resulting nearly 8517k parameters. We use a learning rate of 10-4 and the maximum gradient norm of 0.2. It takes about 200 million environment steps (roughly 4 days of training) to saturate. All baselines are trained with 3 different random seeds, and results reported averaged across those seeds. Inspired by the reward design in training point-goal (Anderson et al., 2018), and object-goat policies (Batra et al., 2020), the social navigation reward is based on the distance the humanoid at time t, and defined follows:
((be, he) ((b 11-1),
if if, h4)
distance
2,
if < A(b,, ht) S
((b (bre, ft),
otherwise
where A(be, ht) is the geodesic distance between the robot location b, and the humanoid location hy at time L. The first condition encourages the robot to move away from the humanoid when is closer than Im, ensuring that the agent maintains a safe distance from the humanoid. The second condition gives constant reward for reaching within 1-2m, and the last condition rewards the robot to get closer to the humanoid.
To make sure the robot can face toward the humanoid, we further add an orientation reward when the robot is approzching the humanoid:
[ bb) forward
if A fea) < 3
T prientation
{0,
otherwise
where of forwwadd is the robot normalized forward vector in the world frame, and the vector (he - bb)) is also normalized.
During training, the episode terminates if there is a collision between the humanoid and the robot. The robot receives bonus reward of +10 if the robot successfully maintains a safety distance between Im and 2m to the humanoid and points to the humanoid for at least 400 simulation steps. The criteria for 'facing the humanoid' is computed by the dot product of the robot's forward vector and the vector pointing from the robot to the humanoid, with the threshold of > 0.5. We assume the robot has access to an arm depth camera (224 x 171 with the horizontal field of view (hFOV) 55), an arm RGB camera (480 x 640 with hFOV of 47), a binary human detector (1 (1-dim), and the relative pose of the humanoid in polar coordinate system (2-dim). In addition, a slack reward of -0.1 is given to encourage the agent to find the humanoid as soon possible. In all episodes, to make sure that the robot learns to find the humanoid, the robot location is initialized at least 3m away from the humanoid. The final social navigation reward is as follows:
social
vecess
distance
+ 3r
orientation
0.1.
During evaluation, the total episode length is 1500 steps and the episode terminates if there is collision between the humanoid and the robot. In addition, the robot location is initialized at least 5m away from the humanoid, and the initial locations of the robot and the humanoid, and the humanoid
16

---

Published as a conference paper at ICLR 2024
Aug. Dis. Humanoid
1500
1250
4
Rewand
1000
End and
750
End end
humanold GPS
humanold GPS
humanold detector
500
humanold detector
arm depth
arm depth
depth
depth
SdM
150M
200M
250M
0
200M
25bM
Steps
Steps
(a) Training avg. dis. to the humanoid
(b) Training reward
Figure 5: Social Navigation training curves. We plot the training average distance to the humanoid and reward for the social navigation baselines and ablations. We use 3 seeds for each model.
path are fixed across the baselines. This ensures that we have fair comparison across different baselines.
For the social navigation task, the output space of the policy is the linear and angular velocities with the range of -1 and +1 (2-dim), followed by scaling to - and +10, which is equivalent to 2.5m/s in the real world. We use the same maximum linear and angular velocities of 2.5m// (rad/s) for both humanoid and robot. Since the Spot robot has a long body shape and cannot be represented by single cylinder, for collision detection. This ensures that the robot arm camera does not penetrate walls or obstacles while allowing the robot to navigate in cluttered scene.
2-cylinder representation, placed in the center and the front of the robot
Fig. 5 shows the average distance between the humanoid and the robot and reward learning curve over the number of simulation steps for the end-to-end RL policy and its ablations. We see that the agent is able to improve the reward while minimizing the distance to the humanoid for finding and following the humanoid over training.
Oracle for the Minimum Steps. To compute the Finding Success Weighted by Path Steps and Following rate, we need to measure the optimal finding time This measure similar to the optimal path length in navigation tasks, but in this case the target to navigate to is dynamic. We thus define the optimal path length as the minimum time that it would take for an agent to reach the humanoid if it knew the humanoid's s trajectory in advance. Formally, let h; be the humanoid position at step i and r; the minimum number of steps to go from the robot starting position to h4, we define as:
arg min(r; < i),
(I)
measuring the earliest time where the robot will be able to find the humanoid. To compute this measure, split the humanoid trajectory into equally spaced waypoints, then path planner to measure the number of steps it would take to reach each waypoint from the robot starting position and take the earliest waypoint satisying Eq. 1. Given this measure, the optimal following time corresponds to that of a robot which can find the humanoid in / and follow it until the end of the episode, i.e. E' I, with E being the episode length.
A.2 SOCIAL REARRANGEMENT
We train all the rearrangement baselines using DD-PPO (Wijmans et al., 2019), distributing training across 4 NVIDIA 1000 GPUs. Each GPU runs 24 parallel environments, and collects 128 steps for each update. We train with Adam (Kingma & Ba, 2014) using learning rate of 2.5e-4, We use 2 PPO minibatches and epoch per update, an entropy loss of 14,4, and clip the gradient norm to 0.2. All policies are trained for 100M environment steps. The policy uses ResNet-18 (He et al., 2016) visual encoder to embed the 256 x 256 depth input image into a 512 dimension embedding. The visual embedding then concatenated with the state sensor values and passed through a 2 2-layer LSTM network with hidden dimension 512. The LSTM output is then set to an action and value prediction network. All methods the same reward function specified 10 for succeeding in the
17

---

Published as a conference paper at ICLR 2024
0.8
25
20
Success Rate
0.6
Reward
15
Learn-Single
Learn-Single
0.4
Plan-Pop1
10
Plan-Popi
Plan-Pop:
Plan-Pop;
0.2
Plan-Pops
Plan-Pops
Plan-Pop-
Plan-Pop.
Learn-Pop
Learn- Pop
0.0
20M
40M
60M
80M
20M
40M
60M
80M
Steps
Steps
(a) Baselines training success over time
(b) Baselines training reward over time
0.6
0.5
15.0
Success itaee
0.4
12.5
Reward
10.0
0.3
7.5
0.2
Plan-Pop:
Plan-Pops
Depth + RGB
Depth + RGB
0.1
Humanoid-GPs
2.5
Humanoid-GPS
actions
0
20M
40M
60M
80M
0
20M
40M
60M
80M
Steps
Steps
(c) Ablations training success over time
(d) Ablations reward over
Figure 6: Social Rearrangement training curves, We plot the training success and reward for the social rearrangement baselines (top) and ablations (bottom). We use seeds for each model.
overall task, +5 for completing any subgoal consisting of picking one of the target objects or placing an object at its goal, -0.005 penalty per simulator timestep encourage faster completion, and - penalty and episode termination if the agents collide. All baselines are trained with three different random seeds, and results are reported averaged across those seeds.
The final social rearrangement reward is as follows:
sncial-rearange 10 success + 5 gennhgoa
5 gecollision 0.005.
Fig. shows learning curves for baselines and ablations on the social rearrangement task. We present the overall task success well as the training reward for all approaches, averaged over seeds. We observe that some baselines like Learn-Single and Plan-pop: are able to learn the task much faster than other baselines like Plan-pop2.3,4 due simpler training setting, Among the ablations, removing the sensors used in original training make learning slower, with primitive actions having the most effect.
Learned and Oracle skills. For ease of learning, we adopt a two-layer policy architecture for all baselines. where learned high-level policy selects low-level skill to execute based on observations. The action space of the learned high-level policy consists of discrete selections from all possible combinations of skills and objects/receptacles allowed at each step. We work with a known, fixed library of low-level skills that can accomplish instructions like "navigate to the fridge" or "pick an apple." For the robot, we consider both learned skills and oracle skills that use privileged information from the environment. Additionally, we provide 4 'primitive' actions to the high-level policy that move the robot forward/backwward or turn it left/right by a fixed amount. During training, we use the oracle low-level skills train the high-level policies due to faster training speed, but present evaluation results with both oracle and learned skills. For the humanoid, low-level skills are always oracle, described in Section 3.1.
18

---

Published as a conference paper at ICLR 2024
The oracle navigation skill has access map of the environment, and plans shortest path to the agent's 's destination. Manipulation oracle skills like "pick or place an apple" instantaneously attach the apple to the gripper, or place it at the target location. If the high-level policy chooses execute an infeasible low-level action, such as attempting to pick an object that is out of reach (based on predefined pre-conditions), the action results in no-op with changes the environment. If the robot approaches the humanoid within a short distance (< 1.5m) while executing a skill, the current skill is aborted, and the high-level policy replans its next action. This results in reactive behaviors, like the high-level policy commanding the robot to move backwards to give way to the humanoid in narrow corridors, or choosing to pick the second object after realizing that the humanoid is picking the first.
When using learned skills, we use the same 2-layer policy architecture, except use learned navigation, and learned pick/place skills, which operate entirely using robot depth and onboard sensors. These skills do not use privileged information, and hence are more prone to failures in the diverse set of scenes considered in our tasks. Refer to Section B for more detail on training and design of low-level skills.
When evaluating social rearrangement with learned low-level skills, we keep the high-level policy frozen, after training it with oracle skills. Hence the high-level policy is not robust to low-level execution failures. As result observe considerable drop in the overall performance, when using learned skills (Table 3). This performance can potentially be improved by training the high-level policy with learned low-level skills in-the-loop, by fine-tuning in this setting. We leave this to future work.
B LOW-LEVEL SKILL TRAINING
We include the implementation details for training the low-level skills: navigation, pick, and place skills, which coordinated by the high level policy described in Section 4.2.
Navigation Skill. Given the location of the target object, the robot uses neural network policies* to find the object, similar to PointNav (Anderson et al., 2018). This navigation skill is different from the social navigation policy since the former does not require the robot to continuously follow the humanoid while avoiding collision. The robot has access to arm depth camera (224 x 171 with hFOV of 55), and the relative pose of the target object in polar coordinate system (2-dim). Similar to the policy used in the social navigation task, the output space of the navigation skill is the linear and angular velocities with the range of -1 and +1 (2-dim), followed by scaling to -10 and +10. We also use the same 2-cylinder collision shape representation as the one in the social navigation task to ensure that the robot arm camera does not penetrate walls or obstacles while allowing the robot to navigate in a cluttered scene.
During training, the object navigation reward that encourages moving forward the target object r distance at time defined as A(b A (b,, et), where A(b, et) is the shortest path distance between the robot location b, and the object et at time t. To encourage the robot to orient itself toward the target object for improving grasping, it receives an additional 'orientation' reward rrrientaaion when A(br-1, ct-1) A(b, ex) < 3, in which the orientation reward penalizes the dot product of the robot forward vector and the robot to target object vector weighted by the scale of 5 x 10-2, 2. A navigation success reward of 10 is given if (1) the distance between the agent and the target object is less than 1.5m, and (2) the dot product of the robot forward vector and the robot to target object vector > 0.5. To reduce the collision between the robot and the scene, penalty of -5 x 10-3 is given if there is a collision. Finally, slack reward of -1 x 10-2 is given to encourage the robot to find the target object as soon as possible. The final navigation reward is as follows:
20ccess + distance + 0.05r
orientation
0. 0.005 0.01.
T
During training, the episode terminates if the robot finds the object or reaches the maximum episode simulation step of 1500. In addition, the episode also terminates if the robot collides with a humanoid that walks randomly, similar to the setup in training social navigation policies. To make sure that the robot learns to navigate in complex scenes, the robot is placed at least 4m away from the target object
'In this section, we use 'policy' and 'skill' interchangeably to refer to controller parameterized by neural networks. In addition, we use "robot" and 'agent' interchangeably refer reinforcement learning agent.
19

---

Published as a conference paper at ICLR 2024
location. We train the navigation skill using DD-PPO, distributing training across 4 NVIDIA 1 GPUs, and with learning rate of x 10 and the maximum gradient norm of 0.2. Each GPU runs 24 parallel environments, and collects 128 steps for each update. We use long short-term memory networks (LSTM) policy with ResNet-18 (He et al., 2016) as the visual backbone and two recurrent layers, resulting nearly 851 parameters. It takes about 300 million simulation steps (roughly 6 days of training) to reach 90% navigation success rate using the above hardware setup.
Pick Skill. Given the location of the target object, the robot uses neural network policies pick up an object by controlling the arm and moving the base (i.e., mobile pick, as defined in Gu et al. (2023)). The robot has access to (1) an arm depth camera (224 x 171 with hFOV of 55), (2) the relative pose of the target object in a Cartesian coordinate system (3-dim), (3) the arm joint angles (7-dim), (4) binary holding detector if the robot is holding an object (1 (1-dim), and (5) the relative pose of arm end-effector to the target resting location in Cartesian coordinate system (3-dim). The output space of the pick skill (1) the linear and angular base velocities with the range of and +1 (2-dim), followed by scaling to 10 and +10, (2) the delta arm joint angles applied to the arm with the range of 1 and +1 (7-dim), followed by a scaling factor of x 10 10-2, and a binary command to snap/desnap the object to the end-effector (1-dim). The robot can only pick up the object if the distance between the end-effector and the object is less than 0.15m (we teleport the object to the
end-effector to simulate the grasping).
During training, before picking up the object, the pick reward that encourages the arm to move toward the object moov at time is defined as ((- Aca, et), where A(ce, ex) is the geodesic distance between the robot's arm end-effector location and the object et at time t. After picking up the object, the retract-arm reward that encourages the robot to retract the arm rt y.fetract at time t is defined as (c,,11 1t-1) Ace, qt), where (C,, (t) is the distance between the robot's arm end-effector location C₁ and the target end-effector resting location qi at time t. The robot receives the success reward of +-2 if the robot (1) picks up the right target object, and (2) A(c,, q:) is less than 0. 0... Finally, a slack reward of 5 x 10 given to encourage the robot to pick up the target object as soon as possible. The final pick reward is as follows:
pick
21 sIccess
rt
move
retract 0.003.
During training, the episode terminates if the robot (1) picks up the wrong object or drops the object, both with the penalty of -0.5, (2) reaches maximum simulation steps of 1250, or (3) successfully picks up the right target object and retracts the arm, with the success reward of +-2 To make sure that the robot learns to orient itself to pick up the object, the robot is placed at least 3m away from the target object location. We train the pick skill using DD-PPO, distributing training across NVIDIA GPUS, and with learning rate of 3 x 10-1 and the maximum gradient norm of 0.2. Each GPU runs 18 parallel environments, and collects 128 steps for each update. We use long short-term memory network (LSTM) policy with ResNet- 18 as the visual backbone and two recurrent layers, resulting nearly 8540k parameters. takes about 100 million simulation steps (roughly one day of training) to reach 90% pick success rate using the above hardware setup.
Place Skill. Given the location of the goal location, the robot uses neural network policies to place an object by controlling the arm and moving the base (i.e., mobile place Gu et al. (2023)). For consistency, the input space of the place skill has the exact same input space as the pick skill. The output space of the place skill also shares the same output space of the pick skill. The robot can only place the object if the distance between the end-effector and the target place location is less than 0. 1.15m (we teleport the object from the end-efffcctor to the target place location).
During training, before placing the object, the place reward that encourages the robot to move close to the target location r ymooe at time t is defined as A(c-11, 1t-1) A(c,, et), where A(c,,e)) is the distance between the robot's arm end-effector location C and the object e at time t. After placing the object, the retract-arm reward retmact is the same the one in pick skill learn to reset the arm. In addition, the robot receives an addition bonus reward bonus of +5 if the robot places the object in the right location. Finally, the robot receives the success reward of +10 if (1) the robot places the object in the right location, and (2) A(c,, 9t) is less than 0. 15m. A slack reward of -5 x 10 is given to encourage the robot to place the object as soon as possible. The final place reward is as follows:
place
100
bonus
retract 0.005.
r
re
20

---

Published as a conference paper at ICLR 2024
St
SPSt
Ff
CR4
BYR
TD4
FDJ
Hcuristic Expert
1.00
0.97
0.51
0.52
0.24
2.56
1.72
End-to-end RI.
0.97
0.65
0.44
0.51.0.00
0.1910.02
3.431..07
1.7010.04
humanoid GPS
0.76.10.02
0.34
0.20
0.48
0.13
5,1810.11
1.64
humanoid detector
0.98
0.68
0.30
0.64
0.16
3.44
+..09
urm depth
0.00
0.54=00.01
0. 0.19=0.01
0.71=0.08
0.15+0..01
4.94+0.08
1.21+
arm depth arm
0.96
0.61
0.38
0.55=0.04
0.17
3.74
1.8240.05
Table 2: Social Navigation baseline results. We report three additional metrics: (1) Backap- Yield
Rate (BYR), (2) The Total Distance between the robot and the humanoid (TD), and (3) The 'Following' Distance between the robot and the humanoid after the first encounter (FD).
Humanoid path
Find Phase
Follow Phase
Figure 7: Illustration of Social Navigation end-to-end RL policy's behavior. The robot finds the humanoid (left). The robot yields humanoid by doing 'three-point-tura" motion (center). The robot yields to the humanoid by backing up (right).
During training, the episode terminates the robot (1) places the object in the wrong location, (2) reaches maximum simulation steps of 1250, or (3) successfully places the right target object and retracts the arm, with the success reward of +10. To make sure that the robot learns orient itself to place the object, the robot is placed at least 3m away from the target object location. Similar to the one in pick skill, we train the navigation skill using DD-PPO, distributing training across NVIDIA GPUs, and with learning rate of 3 x 10 and the maximum gradient norm of 0.2. Each GPU runs 18 parallel environments, and collects 128 steps for each update, We use long short-term memory network (LSTM) policy with ResNet- 18 as the visual backbone and two recurrent layers, resulting nearly 8540k parameters. takes about 50 million simulation steps (roughly half of day of training) reach 90% place success rate using the above hardware setup.
C DETAILED COMPARISON RESULTS
Here we provide additional detailed results comparing the performance of different baselines along additional metries.
C.I SOCIAL NAVIGATION
In this section, provide detailed analysis of the Social Navigation task, studying the behavior of the different baselines at evaluation time.
C.I.I ADDITIONAL METRICS
In Table 2, we further report three additional metrics: (1) Backup- Yield Rate (BYR): How often did the robot do a backup or yield motion to avoid collision when the human is nearby? We define 'backup motion' as backward movement by the robot to avoid collision with the humanoid when the distance between them is less than 1.5 meters. Furthermore, we define 'yield motion' as a robot's motion aimed at avoiding collision with the humanoid when the distance between them is less than 1.5 meters, and the robot's velocity is less than 0. 0.1m/s; ss (2) The Total Distance between the robot and the humanoid (TD): What was the L2 distance (in meter) between the robot and the humanoid over the total number of episode steps?, and (3) The 'Following Distance between the robot and the humanoid after the first encounter (FD): What the 1.2 distance between the robot
21

---

Published as a conference paper at ICLR 2024
and the humanoid after the robot finds the humanoid? In summary, the backup- rate lets us quantitatively measure the frequency of backup and yield motions, and the two distance matrices provide observation of how close the robot and the humanoid are during the finding and following stages. Ideally, the FD should be between 1-2m, while policies with higher SPS have lower TD.
C.1.2 ADDITIONAL ANALYSIS
In this section, we provide additional analysis for the end-to-end RL policy and its ablations. Fig. 7 provides example of how the robot moves to find the humanoid and produces backup motion to yield to the humanoid by anticipating where the humanoid will walk next. For the ablation baseline without the humanoid GPS, we observe that it learns two types of finding humanoid strategies. The first strategy is that the robot randomly walks to increase the chances of finding the humanoid. The second strategy that the robot keeps rotating until there is humanoid in sight (i.e., scanning the environment). captured by the humanoid detector. As result, compared to the method with the humanoid GPS, the one without the humanoid GPS needs more steps to find the humanoid (lower SPS). It also tends to lose track of the humanoid when the humanoid walks into another room, leading to the case that the humanoid is not in sight, and the robot needs to find the humanoid again (lower following rate).
For the ablation baseline without the humanoid detector, we observe that it has worse following performance (7% drop), and higher collision rate (13% increase) than those of the one with the humanoid detector. This occurs because while the humanoid GPS offers a relative L2 distance to the humanoid, it does not account for scenarios where a wall or obstacle obstructs the robot's view. As result, the robot has no idea if there is a wall with a low L,2 distance to the humanoid, leading to a lower following rate and higher collision rate. Providing the humanoid detector allows the agent to 'see' if there is a humanoid there, and thus follow the humanoid.
For the ablation baseline without the arm depth, we find that it has the highest collision rate (leading to the lowest following rate). This is because the arm depth provides useful information to know and record where the empty space/obstacles are (incorporated with the LSTM memory) when the robot needs to avoid collision. As result, we find that the baselines with the arm depth or arm RGB tend have lower collision rate.
Finally, for the ablation baseline with the arm depth being replaced by the arm RGB, we find that it has slightly higher collision rate than the one with the arm depth. This is because the arm depth provides information about the distance to obstacles, leading to a better moving strategy to avoid collisions.
Overall, we find that the end-to-end RL policy and its ablations have comparable Backup- Yield Rate, suggesting that humanoid avoidance motion is learned through RL. training. At the same time, the RL policy and its ablations can maintain close distance to the humanoid (low Following Distance). But still, there is a performance gap between the RL policy and the Heuristic Expert in terms of SPS and the following rate. This leaves room for future improvement.
C.2 SOCIAL REARRANGEMENT
Here we present additional metrics and analysis for social rearrangement. We also describe the different ablations in more details.
C.2.1 ADDITIONAL METRICS
Collision Rate (CR). Together with the success rate and relative efficiency, we are interested in measuring whether the social rearrangement agents can complete tasks safely, without colliding with the humanoid agent. Thus, we measure the collision rate (CR) both the vrain-populatio and the zsc-population settings. Following Sec. 3.2, we define CR as the proportion of episodes containing collisions between the robot and the humanoid. We report the results in Table 3, along with the Success rate (SR) and Relative Efficiency (RE) metrics. We observe similar trends to the success rate measure, with Learn-Sigge,, CR with ZSC population (0.09 + 0.23 with train-pop for Plan-pop1). Plan-Pop, obtains the best collision rate in the training population, with a rate of 9%, despite having a more diverse population than other baselines. This is because one of the agents in the training population for Plan-pop4 stays
Poop. having low CR with the training population but high
22

---

Published as a conference paper at ICLR 2024
Method
Train pop-eval eval
7SC pop-eval eval
SRT
REf
CR.
RC1
SRT
RET
CR.
RC1
Learn-Single
98.50
159.
0.12
50.94_a0.5
106.02
0.25
0.45
Plan Pop:
91.2+2.6
152. 152.4455
0.09 os
0. 2.40
50.44ga0.02
0.23
0.43
Plan-Pops
06.89-1.47
110.06+a.83
0.10+0.10
0.50
70.23-7.02
102.13
0.15
0.52+0.05
Plan Plan
77.79=2.80 96
118.9526.04
0.12
0. 0.48
71.79
101 15.11
0.1740.19
2.53
Plan-Pop,
72.42=1.52
105.4921.7
0.09
0.5522..0 oo
71.32=29.77
103.53
0.1640.17
0.53
Learn-Pop
22.20
135.32
0,15
0 0.50
18.32
99.30
0,26
0. 0.46
learned skills
=1.09
79. 63
0.37
0.12
11.44
-5.25
0.17
0.1220.12
depth RGB
76. 76.70
100.04...05
0.1340.14
0 0.49
70.89=a.18
100. 1614.77
0.1640.18
0.54g0.04
Humanoid-GPS
76.45g18
108.96 2.2.9
0.18
0.
68.70....5
98. 18 18.2
0.224
0.53
Primitive actions
85.71g1.58
121.3613.9
0.32
76.80 0.
111.97
0.331031
0.58
Table 3: Social Rearrangement baseline results.
in place does no part of the task, thus reducing the total number of collisions. In the ZSC setting, Plan-Pop2 obtains the lowest collision rate, though the rates are comparable across Plan-Pop2,4-
Ratio of Completion (RC). We also report, for all our baselines, the ratio of the task completed by the robot, which measures the proportion of objects that were rearranged by the robot agent. A value of 1.0 indicates that the task is completely done by the robot, whereas 0.0 indicates that the task was done by the humanoid alone. Values closer to 0.5 indicate that the task was split among the robot and humanoid, resulting in increased efficiency. We show the results in Table 3. Almost all baselines achieve a RC close to 0.5 with training population. Learn- Single achieves RC close to 0.5 in the train population, showing that both agents learned split the task to perform it efficiently. Plan-Popı shows a slight decrease in RC, which is consistent with the drop in the SR, indicating that agents are still splitting the task evenly, while overall achieving lower success rate. Plan-pop4 has the highest train-pop RC, because one of its training population agents complete part of the task, requiring the robot to rearrange both objects. The results on ZSC population follow success rates, with Hlan-Pop2,3. achieving higher RC, since the agents are trained to rearrange either object. In comparison, Leann-Sigle, Plan-popı and Learn-Pop Pop have slightly reduced RC due to inability to generalize to partners that rearrange different objects than their training population.
C.2.2 ADDITIONAL ABLATION RESULTS AND ANALYSIS
In the main paper we presented ablation experiments where we: (1) replaced oracle skills with learned skills. (2) replaced depth arm camera with RGB, (3) Removed the humanoid GPS. Here, we present an additional ablation, where we remove some primitive actions like move backwards, forwards, turn left or right from the action space of the high-level policy. This measures the effect of these primitive navigation actions in the Social Rearrangement task, called- Primitive actions. The robot agent is trained in the same setting as Plan-Pops. but without the four low level navigation actions. We report the results in Table 3. As can see, the collision rate (CR) increases significantly, showing that the primitive navigation actions are essential to reduce collisions between both agents. At the same time, removing the navigation actions improves the success rate and RE both in the Train-Pop and ZSC-pop settings. This is because the agent does not spend time making way for the humanoid, which allows to complete the task in a lower number of steps. Despite being faster at completion, the high CR makes this baseline non-ideal as it reduces the "safety" of human or humanoid collaborators.
C.2.3 ZERO-SHOT POPULATION DETAILS
Here we describe the ZSC population used in our experiments in detail, and their effect in the behavior of the trained agents in the zero-shot coordination setting.
The 10 ZSC population collaborators used in ZSC eval are created as follows: 3 are trained check- points from the training of Learn-Singe 3 are trained checkpoints from the training run of Learn-Pop and 4 are planner-based humanoids, where picks up both objects, 2 pick up one of the two, and I stays still. For the learned checkpoints, we only use the learned policy for the humanoid, and discard the learned robot policy. As a result, most baselines have seen about 1/3 of the ZSC population during training, and need to generalize 2/3 of the population. In general, the learned ZSC evaluation agents tend to focus on rearranging one of the 2 objects in the environment, while the planner- based agents follow their scripted behavior.
23

---

Published as a conference paper at ICLR 2024
Learn-Single
32.59
62.52.4411
47.87.c43
64.33,xc
Training agents
Plan-Popy
41
Plan-Popz
73 65
62
70.41.s.1
39 2451
69:04.5.5
72.97
L456
67.26..3
_040
Plan-Pops
73.21ac
e8.07ex.c
59.b74c.
72:19
71 71.80a5.
28.8
78.22:am
68.90.244
73 22.7
Plan-Pop4
76.48...x
72 39
71.50
63. 6.
69.14.
73.03
06 66.63
68.59 a0.03
27.72ata
75.14
71.32...
Learn-Pop
67.37
3825
37.05 054104
49.79 49.79.0711
34.38
34. 08.42.0 23.59
47.30
64 64.48
48.41 48.41o0m
48.52:3051
Learn-Partnen Part--artters
Learn-Partner,
Leam- Leam-Partten Part-Pa
Learn-Parther Partter
Learn-Pate Partners
Leam-PParten Partnere
Plan- Pan-Partner;
Plan- Partnera
Plan-Partnerg
Plan-Partnete
Averaged
ZSC-Agents
(a) Success rate across different agents
Learn-iingle
47.45
127.85
125,
77.64usev
105. 5lazass
41.M.
129
100.78.r
108 92..9
95.81
62.m
Training agents
Plan-Popy
150.57a5
67.46.x0t
123.59.1.s
67.52
16024.154
109 (0044
309.75:34
Plan-Popz
130.18,230
79.37
98.36.137
90.03gu.
11326,125
112.2
Plan-Pops
9724
113.12.
110.93
81.00as00
27.56
91.46
113.5....m
17.200
98.49.138
101.9..s
Plan-Pop4
100 M
109 88
107.67
84.90z39
95.31
96 96.67.75:
109.59 19.59
107 17
113 11
105 34.2
103 54
Lear-Popp
224.02...
27.57
96 08uaaa
85.14
24.79
110.03.a
97.75,200 98.70
108 LLa
95 05.34
99.0.a11:
Learn-Partner
Learn-Partner:
Leam-Partnery
Learn-Partrier
Learn-Partners
Plan-Partner Learn-Partners Partners
Plan- Plan-Partnerz Partnera
Plan-Partherr
Plan-Partnera
Averaged
Learn-Parte
ZSC-Agents
(b) Efficiency rate ZSC-agents
Learn-Single
0.3aaus
0.48ua.
0.49ac.a
0.74
u.53
0. 0.74
0.29
0.36
0.38aa
0.9aa.t
025
Training agents
Plan-Pop1
0.26.c
0.30
96
0.19pm
0.31
026
0
Plan-Pop:
Plan-Pops
0.35
0.34cc m
0.39
0.5
0.19
Plan-Pop4
0.33.osa
6.34
0.35
0.47
0.40
0.38.cs4
0.20.a44
0,21,a
0.33a.u
0.39
0.16.117
Lear--Ppp
8 1.50..
6.71amm
3.7
0.59
0.74
0.60
0.32 m
0.39pam
0.30
0.2
Learn-Partnerf
Learn-Partner; Partner,
Lear-- Learn- Learn-Partner,
Lear--Partner, Partner,
Learn- Learn-Partners Partners
Learn. Learn-Partners Partners
Plan Plan-Partnery Plan-Partner; Partner
Plan-Partners
Plan-Partner,
Averaged
ZSC-Agents
(c) Collision rate ZSC-agents
Learn-Single
8:45ueez
6.47e1
0.47
0.45
0.40cess
5.67
0.47
0.64eamo
D.M
0.46.a
Training agents
Plan-Popj
0.38,oss
6.48ua.y
0.48
0.43
0.48_acc
0.37
0.47
0.44
0.43
0.42uac3
0.44
Plan-Popy
0.48
0.58
0/A0-ar2
0:574
0.51.
D.54wmm
Plan-Pops
0.58
0.51.a43
0.55
0.52.c.4
0.53.oca
Pian-Pope
0.55.ces
6.52
0.52
0.58.
0.51.a65
0.55.c45
0.50gaa7
8.53
0.49,1
0.57.ac2
0.53,
Leam-Pop
0.47.1m7
0.16
0.50
B/44wae4
0.46waar
0.15
0.44amm
D.45
0.16.a31
Learn-Partnery
Learn-Parte Learn-Partner; Partner;
Leam Partner, Learn-Parther,
Learn-Partners
Learn Partners
Learn Partner
Plan Plan-Partners Partner,
Plan- Plan-Partners Part-artter
Plan-Partners
Plan-Parthier,
Averaged
ZSC-Agents
(d) Ratio of Completion for 7SC-agents
Figure 8: Zero-shot coordination. We report the performance of the baseline agents in the zero-shot coordination setting. Each row corresponds to one of the baselines and the columns represent the different types of zero-shot coordination agents.
24

---

Published as a conference paper at ICLR 2024
Simulation
Real
Figure 9: Robot Embodiment. Spot robot in the simulation environment is designed to minimize the embodiment gaps to the robot in the physical world.
We measure the trained agents' performance when evaluated with different agents from the ZSC-pop- eval population. In Figs. 8a, 8b, 8c and 8d, we show the success rate (a), efficiency rate (b), collision rate (c) and ratio of completion (d) of the different baseline trained agents under this setting. Each row corresponds to one of the baselines, trained and averaged across three seeds, and each column corresponds to one of the 10 ZSC evaluation agents. The last column corresponds to the ZSC-pop-val results in Fig. 4.
The Learn-Single agent overfits to one type of partner, learning to split the task by focusing rearranging a single object and letting the other agent rearrange the remaining one. When evaluated with a ZSC partner, will either be highly successful and efficient, if the evaluation partner was trained to pick the opposite object, or exhibit low performance if the evaluation agent is focused on the same object. For this reason the success rate in the first row of Fig. 8a is close 33% and 66% for the Learn-Single agent, corresponding to one or two of the 3 training seeds matching the ZSC-partner.
Plan-Popı exhibits a similar behavior to the Learn-Single baseline, but in this case, the agent is trained with planner that always focuses on the same object, which makes Plan-Popı focus on the opposite objects across all the training seeds. As result, the agent has a success rate close to 100% or 0% for the different ZSC evaluation agents. As expected, also exhibits high success rate when partnering with the Plan; agent. Because of this, the agent also exhibits significant increase in relative efficiency when partnering with certain agents. As a results is the method with highest relative efficiency, when averaged across ZgC-agents, despite exhibiting high variance.
Plan-Pop2 is trained with a population of agents arranging one of the two objects at random and therefore it cannot specialize anymore by always focusing the same object. As result, the agent shows much lower variance across the members of the ZSC-population, resulting in higher success rate. At the same time, it needs to adapt to the behavior of the ZSC partnering agent, which results in lower peak efficiency than Plan-Popı or Learn-Single resulting on a lower relative efficiency on average. The average collision rate is also reduced relative the previous baselines, with lower collision rate with planning-bast based partners than the learning-based ones. Interestingly, this trend of lower CR with ZSC plan-agents holds across all baselines. We believe this because ZSC plan-agents stop after finishing their portion of the task, while 7SC learned-agents continue to move in the environment (since our reward function does not penalize this). a result, the chances of colliding with ZSC plan-partners is higher than with ZSC Iearned partners.
Plan-Pops.1 exhibit very similar performance across the different agents the ZSC population. Their success rate shows slight improvement with respect to Plan-Pop2. whereas the relative efficiency decreases slightly for Plan-Pops and increases slightly for Plan-Popa, though not significant. In general, adding extra agents to Plan-Pop; that remains still does not seem change the performance. To improve performance of learned coordination robot policies, we might need to incorporate other types of diversities, like humanoid speed, instead of just which object they rearrange.
25

---

Published as a conference paper at ICLR 2024
C.3 QUALITATIVE EXAMPLE
We also show in Fig. 10 an example episode of social rearrangement. We use Plan-Pops as the baseline policy with learned low-level skills. In this example, the task is split amongst both agents, with each rearranging one of the goal objects. Frames 1,3 show the robot and humanoid picking up objects. 4,5 show them placing each object its desired location. Frame 2 shows a scenario where the humanoid and robot cross paths, and the robot backs up to let the humanoid pass before continuing, avoiding a collision.
D ROBOT
We use the Boston Dynamics (BD) Spot robot the robot agent (Fig. 9) due to its robust hardware for real world deployment of trained policies in the future. Spot is quadruped robot with five pairs of depth and RGB cameras (front-left, front-right, left, right, back), and degree of freedom one pair of depth and RGB cameras mounted on the gripper. The arm depth camera has 224 x 171 resolution and hfov of 55. Spot robots are capable of grasping rigid objects, climbing up/down stairs, and outdoor/indoor navigation given users' control input using BD's ' Spot control APIs. Its versatility and robustness make it an ideal mobile platform for studying sensing, manipulation, and human-robot interaction. On the simulation side, the Spot robot simulation moves its base by commanding linear and angular velocities (2D, continuous) and can move backwards if turning or moving forwards results in collision. The action space of the robot is continuous forward and angular velocity in the local frame of the robot (2-dim) for navigation and delta joint angles (7-dim) for the arm.
with
E SCENES
We incorporate scene assets from the Habitat Synthetic Scenes Dataset (HSSD-200) (Khanna et al., 2023) in Habitat 3.0. HSSD is a dataset 211 high-quality 3D homes (scenes) containing over 18k individual models of real-world objects. An example scene is shown in Fig. 11. For experiments, we use a subset of 59 scenes and limit our objects to the YCB dataset (Calli et al., 2017) for simplicity. Specifically, use 37 scenes from training, sampling 1000 episodes per scene, 12 for validation, with 100 episodes per scene and 10 for test, with 15 episodes in each scene. Note that the Habitat3.0 framework will be released with the complete HSSD) dataset.
F SIMULATOR DETAILS
F.1 BENCHMARKING
We benchmark here Habitat 3.0 speed under varying scene sizes, different number of objects in the scene, and the type and number of agents. All tests are conducted on single Nvidia V100 GPU. We report results on a single environment (Fig. 12, left) and 16 parallel Habitat 3.0 environments (Fig 12, right). For each measure we sample random actions for 300 steps per agent and report average and standard error results across 10 runs. All the solid blue curves correspond to small scenes sizes and two objects in the environment. For each agent, we always render single depth image. Our small scene is 68.56m2 in size and has 1 bedroom and I bathroom, the medium scene is 136. 1 1m² in size with bedrooms and 2 bathrooms, and the large scene 846.15m² in size with 4 bedrooms, bothrooms, and a den and office space.
On a single environment, we obtain performances on the range of 140 to 250 FPS, depending on the setting, As we can see, varying the number of objects has no significant effect in the performance speed, while we notice significant differences when changing the scene size (245 FPS down to 144. Switching from single spot agent to two spot agents drops the performance from 19 to 150+1, having a similar effect to varying the scene size. The humanoid is also slower than the Spot robot 1245 vs 155=26), due to the higher number of joints in the skeletal model. However, the difference between robot and humanoid agents, becomes much less pronounced when switching to the two agent setting, obtaining comparable performances between the Robot- Robot and Hluann-Robor settings (150+1 vs. 136=1). Since momaamoid-rbbb simulation is the primary use-case of Habitat3.0, 00, this is positive signal, that shows that adding humanoid over robot does not decrease simulation
26

---

Published as a conference paper at ICLR 2024
Humanoid path
Spot path
Object start pos.
Target pos.
2
2
3
Figure 10: Social Rearrangement Example: We show an example behavior from the Plan-Pops
baseline with learned low-level skills. Here, the agents split the rearrangement task, with each picking one of the objects (frames 1, 3) and placing them in their target location (frames 4, 5). In frame 2, we observe an emergent collaborative behavior: the robot moves backward let the humanoid pass through the hallway, before continuing with its task, avoiding a collision.
27

---

Published as a conference paper at ICLR 2024
Figure 11; Example Scene. Habitat 3.0 based on HSSD (Khanna et al., 2023) scenes. An example scene which includes various types of objects and scene clutter has been shown.
Habitat 3.0 Benchmark Processes
Habitat 3.0 Benchmark 16 Processes
Small Scenes
Small Scenes
Robot Obj.
Medium Scenes
Robot Obj
Medium Scenes
Robot Obj.
Large Scenes
Robot Obj.
Large Scenes
Robot 10 Ob).
Robot 10 Oo).
Robot Obj.
Robot Obj
Robot Obj
Robot Obj
Robot 10 Ob).
Robot 10 0b)
Robot Obj
Robot Ooj
Robot 5 Obj
Robot Obj
Robot-10 10 Obj.
1 Obj
Robot
Robot
Humanoid
Humanoid
Humanoid w/o Skin
Humanoid w/o Skin
Humanold Pick
Humanold Pick
Robot-Robot
Robot-Robot
Roban-Humanaid
Robat-Humanaid
50
100
150
200
250
500
1000
1500
2000
FPS
FPS
(a) Benchmark with a single environment
(b) Benchmark with 16 environments
Figure 12: Benchmark results in Habitat 3.0. We study the effect of varying scene size, number of objects, type of agents and single or multi-agent.
speed. We don't 't notice a difference in performance between representing the humanoid skeleton or applying linear blend skinning, implying that the visual realism of skinned humanoids has very little effect on our simulation speed. We also don't ' notice significant differences in performance between the picking or navigation actions.
We observe similar trends when switching to 16 environments, obtaining promising scaling results, with performances in the range of 1100 2290 FPS. Note that these environments are still on single GPU, and Habitat 3.0 natively supports environment parallelization. As result, the most common simulation speed that users will experience when training policies is between 1100 to 2290 FPS depending the scenario.
F.2 COMPARISON WITH EXISTING SIMULATORS
Habitat 3.0 is designed to support efficient simulation of tasks with humans and robots indoor environments. Since Habitat 3.0 is built upon the Habitat platform, also supports features present Habitat 1.0 and Habitat 2.0, including physics simulation, support for multiple types robots, artieulated objects real scanned scenes. Our work opens opportunity to update Embodied AI tasks previously studied the Habitat platform with humanoid simulation or human in the loop interaction, allowing to train robots that better interact with humans.
In Tab. 4 we provide a comparison of Habitat3.0 .0 with some existing related simulators. Habitat3.0 provides support for both real robot models and humanoids, enabling train policies for both types of
28

---

Published as a conference paper at ICLR 2024
agents. Humanoids in Habitat3.0 are based on the SMPL-X model, opposed to Authored Designs (AD). This enables to scale up the number of possible body models, and provide motions coming from motion captured data motion generation models. The humanoids can also be controlled at different levels, including joints (Jt), inverse kinematics (IK) or high level commands, such as walk or pick up (HL). A key feature in our platform is the HITL interface, which allows to control the humanoid via a mouse-keyboard interface ( or VR (e). Furthermore, we include a large
number of authored multi-room scenes enabling training policies in wide diversity of realistic indoor environments, and provides significant increase in simulation speed, compared to other simulators with robot and humanoid support. Note that the speed is influenced by several factors, including rendering fidelity, physics simulation, resolution, hardware capabilities, and more (refer to Section E.1 for detailed benchmarking). As such, speed numbers between different simulators are not directly comparable and serve as rough reference points. We take the reported numbers in individual papers for single environment settings.
Robot
Humanoid
HITL Interface 2B
Type of Simulation
#Authored
Speed
Support
Support
Body Model AD AD
Control IK, HL Jt IK
Multi-Agent
Scenes
(steps/s)
Virualloome-Sciall VRKitchen
PP RBF PP RBF
50
10
16
15
SAPIEN
200-400
A12-THOR
RBF. PP
120
90-180
Habital 2.0
RBF
105
1400
TDW
AD
IK,
RBF, PS
15
5 168
SEAN 2.0
AD
HL
RBF
3 60'
iGibson
AD
R Jr, TK, HI.
RBF
50
100
Habitat
SMPL-X
RBF
200
140-250
Table 4: Comparison with related simulators. We compare Habitat3.0 with other related simulators for Embodied AI supporting simulation of real robots or humanoids. Speeds were taken directly from respective publications or obtained via direct personal correspondence with the authors when not publicly available (indicated by t).
Body Model: Authored Design (AD), SMPL-X (Pavlakos et al., 2019).
Control: Joints (Jt), Inverse Kinematics (IK). High Level Commands (HL), Rigid Transforms (R).
HITL Interface: Mouse and Keyboard 200 ). Virtual Reality (C)).
Type of Simulation: Rigid Body Physics (RBF), Pre-post-conditions (PP), Particle Simulation (PS)-
Speed: The reported numbers correspond to benchmarks conducted by different teams using different hardware configurations to simulate diverse capabilities. Thus, these should be considered only as qualitative comparisons representing what user should expect to experience single instance of the simulator (without parallelization..
G HUMAN-IN-THE-LOON (HITL) EVALUATION
We test the ability of trained robotic agents coordinate with real humans via our human-in-the-llee (HITL) infrastructure across 30 participants. Our user study consisted of 3 conditions: doing the task alone (solo), paired with robot operating with Learn-Single, or paired with Plan-Pop, agent. We have a brief training step, where the users get accustomed to the tool, followed by solving each condition in random order. However, as the users interact more with the tool, they get better the overall task. We account for this learning effect by leveraging latin-square counter-balancing (Bradley, 1958) for the ordering of these conditions.
Each participant performs the task for 10 episodes per condition in one of the test scenes. We measure the collision rate (CR), task completion steps (TS) and Relative Efficiency (RE) across all episodes. and report them in Table 1. CR is the ratio of episodes that contain collisions with the robot. SR, RE are the same as defined in Sec 4.2. RE still captures the relative efficiency of task completion when the task is done with a robot and requires the computation of task completion steps with the robot relative to the human doing the task alone. However, unlike the automated evaluation (Train-pop-eval) and (ZSC-pop-eval,, we cannot compute the RE per episode in HITL evaluation. Since humans exhibit learning effect when asked to do an episode multiple times, we cannot ask the human to do the same episode with and without the robot, preventing the computation of RE per episode. Instead, we fit a generalized linear mixed-effect model (GLMM) (Kaptein, 2016) with a Poisson distribution, for TS as the dependent variable and method/condition the independent variable while controlling
29

---

Published as a conference paper at ICLR 2024
Task completion steps (TS)
1.00
4000
Collision rate (CR)
0.75
0.50
2000
0.25
1000
0.00
leam-single plan-pop
solo
learn-single
plan-pop Method
solo
Method
Figure 13: TS and CR across all participants in the user study.
Estimated mean p-value difference in TS
Method
Plan-pop Learn- Single
Learn-Single
78.45
0.0533
Solo Learn-Single
316.58
<.0001
Solo Plan-pop
238.12
<.0001
Table 5: Post hoc pairwise comparison between the three conditions of the user study for TS.
for variations from participants and scenes using random intercepts (Bates et al., 2015). We then use the ratio between the estimated means of TS for Learn-Single and Plan-Pop conditions w.r. the solo condition compute their average RE respectively. For CR, we fit logistic regression model with a binomial distribution, again controlling for the random effects of participants and scenes. We use Ime4 package (Bates et al., 2015) R version of 4.3.1 to fit the GLMMs. Table shows the results over three conditions across the 30 users. Fig. 13 shows the TS over all successful episodes and CR over all episodes across the three conditions in HITL.
After fitting GLMM model with Poisson distribution for TS, also compute post hoc pairwise comparisons between the three conditions. We find that the difference in estimated mean of TS between Learn-Single and Plan-pop is not significant, while that between both these conditions and the solo condition is significant (Table 5). To the best of our knowledge, this is the first analysis of performance of learned collaboration agents when paired with real human partners in realistic, everyday rearrangement task. The indication that the robot agents indeed make humans more efficient very promising, with huge significance for the assistive-robotee community.
H LIMITATIONS
In this section, address the limitations of our work, focusing three primary contributions: Human simulation. While our framework can accommodate various actions, our experiments utilize only walking and reaching behaviors. We implement the reaching behavior by obtaining static poses representing distinct reaching positions and interpolating between them to generate motion. For the walking motion, our approach for navigating across waypoints is suitable for path planners, but shows visual artifacts when the humanoid is rotating in place. While this approach suits the behaviors we consider, may be suitable for more complex motions, such as opening cabinet or sitting down. This aspect will be explored in our future work. Additionally, we employ fixed linear blend skinning to represent the human, with statie weight assignments for various poses. Consequently, certain pose configurations may exhibit skinning artifacts, as mentioned in the main paper.
Human-in-the- thop tool. There is a gap between the observations a human acquires through the HITL tool and those accessible to our humanoid in automated evaluations. Although provide visual cues to bridge this gap (such as markers that indicate the direction from the human to the target objects), they still represent different observation space between the two domains. Tasks. Our current focus is on zero-shot coordination without communication, but introducing communication between the robot and human could potentially enhance task efficiency. Furthermore,
30

---

Published as a conference paper at ICLR 2024
models currently benefit from access ground truth information regarding the initial and final object locations in the rearrangement task. A future challenge lies in integrating search capabilities into the rearrangement task. Additionally, make the assumption that objects are initially placed on open receptacles. Handling rearrangements that involve searching for and interacting with articulated objects, like drawers, poses greater challenge.
31


Reviews:
[{"review_id": "wJ8ITufGCr", "reviewer": "Reviewer_FYWe", "summary": "- This work introduces Habitat 3.0, a simulation platform for studying human-robot collaboration in home environments.\n- The environment includes accurate humanoid simulation and a human-in-the-loop infrastructure for real-time interaction and provides a way to evaluate different robot policies with real human collaborators.\n- It also allows the exploration of collaborative tasks: Social Navigation and Social Rearrangement. \n- Finally, the authors demonstrate that learned robot policies can effectively collaborate with unseen humanoid agents and human partners. Shows emergent behaviors during task execution, such as yielding space to humanoid agents.", "questions": "1. For social arrangement, what is the motivation for using population-based approaches? More discussions would be helpful to understand the setting.\n2. Discuss relevant previous works. What is the relationship between the proposed social navigation task and visual tracking (e.g. [a])? Seems quite similar and more discussions are needed. Besides, [a] also contains humanoid simulation and end-to-end RL with Conv+LSTM.\n3. For object interaction, Sec. 3.1 explains that \"Once the hand reaches the object position, we kinematically attach or detach the object to or from the hand.\" Are all the objects simplified as a point particle? Are all the objects in the environment interactive? Is it possible to add more properties to them (e.g. geometry - shape, physics - weight, material, etc.)? It would be great to explore/discuss how to incorporate more general and complex object interactions, from pre-defined motion primitives (e.g. lie, sit) to freeform actions (e.g. grasp).\n4. Currently this work focuses restricted set of motions while more motions can be potentially added with the SMPL-X representation. In demo video 2:56, it also discusses complex motions (e.g. wiping, knocking). It would be beneficial to discuss how future works can incorporate more motions with interactive objects and form meaningful tasks. \n5. Another question is, what kind of tasks can Habitat 3.0 support in addition to navigation and arrangement? Again, I understand that these two tasks are already great for this work, but more discussions on the potential of Habitat 3.0 would make this work more general and influential.\n6. I wonder if the HITL tools would also be standardized and open-source.\n\n[a] End-to-end Active Object Tracking via Reinforcement Learning, ICML 2018.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This work complements existing works (e.g. Habitat, VirtualHome, etc.) in a human-centric way. It allows more flexible human models, human-object interactions, human-robot interactions, and human-in-the-loop evaluation, which are often ignored in previous works.\n2. This work is in general well-written, providing a good survey for the field of embodied AI environments.\n3. This work designs two social/collaborative tasks, navigation and rearrangement, and shows promising results.", "weaknesses": "The main limitation of this work lies in universality. While I believe this work is interesting and helpful to the field, I am wondering if it could be scaled to incorporate more elements, supporting more tasks, so that the progress wouldn't stop here at the two example tasks. While I understand that these aspects might be beyond the scope of a single work, it would be beneficial to demonstrate, or at least discuss, how future works can develop upon Habitat 3.0. For example,\n- physics simulation \n- fine-grained object interactions\n- sim2real deployment", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AP0VMSQyzK", "reviewer": "Reviewer_ndxG", "summary": "The paper presents Habitat 3.0, an Embodied AI simulator designed to facilitate research in human-robot interaction and collaboration within complex indoor environments. The proposed platform aims to address the need for efficient simulation tools to study AI agents' capabilities in realistic and diverse human-robot interaction scenarios. The main contributions of the paper are as follows:\n\nDiverse Humanoid Simulation: Habitat 3.0 offers a framework for creating and simulating diverse humanoid avatars. These avatars encompass various appearances and motions, enhancing the realism of agent interactions within the simulated environments. By employing techniques like skeletal models and linear blend skinning, the simulator achieves a balance between efficiency and visual fidelity.\n\nHuman-in-the-Loop (HITL) Evaluation Tool: The paper introduces a HITL interface that allows real human operators to control humanoid avatars within the simulated environment. This tool enables online human-robot interaction evaluations and data collection, providing a unique platform for studying how AI agents collaborate with humans.\n\nSocial Navigation and Social Rearrangement Tasks: The paper explores two collaborative tasks—social navigation and social rearrangement—to assess AI agents' ability to interact with human or humanoid partners. These tasks require the agents to find and follow humans at a safe distance or to collaborate with a humanoid in rearranging objects within the environment.\n\nEvaluation of AI Agents: The paper compares multiple AI agent baselines in both automated and HITL evaluation settings. These agents include heuristic experts and end-to-end reinforcement learning policies. The evaluations highlight the agents' adaptability in working with different partners to some extent and provide insights into their efficiency and success rates.\n\nRobust HITL Assessments: By conducting HITL evaluations involving real human participants, the paper evaluates AI agents' coordination abilities in scenarios involving diverse partners. These assessments help in understanding how baseline AI agents impact human efficiency and reveal insights into the dynamics of human-robot interactions.", "questions": "- what are possible solutions/integrations to alleviate the unrealistic humanoid locomotion problem, i.e., the agent first rigidly rotates to face the next target waypoint and then follows a straight path. The autonomous agents trained against such human movement models will not be directly transferable to real-world settings, nor the analysis would not be informative.\n\n- it is unclear how easy and flexible to import motion capture data. Can you elaborate on that?\n\n- it is also unclear how trivial it is to use the AMASS dataset along with VPoser to compute humanoid poses and then to import them into the simulator. Trying to use such external tools that the benchmark providers do not support/maintain themselves frequently becomes a huge hassle and ease-of-use of such external tools is critical, so can you also provide some clarification on their integration and/or usability? \n\n- About reliance on VPoser: Depending on the complexity of the task, simple interpolation between poses might not be sufficient, what would be possible solutions?\n\n- is it possible to incorporate physical collaboration scenarios, i.e., partners acting on the same object? would it require additional steps than what was explained on the paper?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Quality:\nThe paper demonstrates a fair level of quality in its methodology and execution. The development of Habitat 3.0 is well-detailed and addresses a clear need in the field of HRI research. The simulator provides an effective platform for investigating human-robot interaction. The evaluation of AI agents in both automated and HITL settings enhances the paper's quality.\n\nClarity:\nThe paper is generally well-written and clear, with detailed explanations of the simulator framework and the tasks studied. \n\nOriginality:\nHabitat 3.0 introduces valuable original contributions to the field. The combination of diverse humanoid simulation, HITL control, and the study of social navigation and rearrangement tasks provide some initial steps and ideas in this domain. Furthermore, the paper explores HITL evaluations involving human participants, which adds to the originality of the work. While the components themselves are not entirely novel, their integration and application within a single framework is original and significant.\n\nSignificance:\nHabitat 3.0, with its focus on embodied AI and human-robot interaction, addresses a critical aspect of AI development. The simulator has the potential to open new avenues for research, including collaborative AI, human-robot teamwork, and social embodied agents. The HITL evaluations are particularly significant, offering insights into how AI agents impact human performance and behavior. The paper's findings and methodology are likely to influence future research in these domains.\n\nOverall, its strengths can be listed as:\n- focus on human-robot interaction compared to previous platforms offering single-agent or multi-homogeneous-agent training.\n- efficient simulation implementation that enables faster progress in training/evaluating developed algorithms.\n- human-in-the-loop evaluation tool that can open up interesting use-cases and approaches to improve and analyze HRI methods.\n- a fair amount of evaluations.", "weaknesses": "The critical weaknesses are:\n- currently, the focus of simulations seem to be more on the visual realism, which is a valid concern. However, the movement of the agents lacks physical realism, which hinders the extend of how human-robot interaction can be evaluated accurately. \n\n- the focus of this work is not proposing novel learning algorithms but still the results indicate that none of the baselines achieve useful following rate (F) nor feasible collision rate (CR) in the social navigation task. Similarly, for the social rearrangement task, none of the methods seem to generalize and let the robots assist their partners effectively (checking the relative efficiency (RE) metric). Even for HITL evaluations, which would be simpler since humans adjust to robots on the fly, the results are not encouraging. This, then, makes it harder to evaluate and take some insights from these evaluations, which is a major component of the paper.\n\n- humanoid shape diversity has been considered, however, robotic agent diversity was not addressed. \n\n- similar to the previous point, the platform, as it is now, lacks task diversity as well.\n\n- it feels like (looking at the efficiency improvements (RE metric) when collaborated vs. solo cases) maybe the tasks do not offer enough opportunities for collaboration. \n\n- personally, I find the HITL evaluations more interesting, however, the paper does not cover detailed evaluation and analysis of these experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mNKQF5WEP4", "reviewer": "Reviewer_HoUA", "summary": "This paper provides a simulator supporting humanoid avatars and robots for the study of collaborative human-robot tasks in home environments. Diverse and realistic human models are constructed, addressing the challenges of accurate modeling of the human body as well as the human-like appearance and motion diversity of the models. Besides, The platform supports interaction between a real person and a simulated robot via a mouse and keyboard inputs or VR interface, enabling human-in-the-loop simulation. This paper also investigates two collaborative human-robot interaction tasks, social navigation, and social reorganization, and provides insights into learned and heuristic baselines for both tasks in the simulator.", "questions": "See Weaknesses", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- This work presents a Co-Habitat for Humans, Avatars, and Robots, offering a simulation environment for humanoids and robots within a wide range of indoor settings, which can promote the development of human-robot tasks in the Embodied AI field.\n- Habitat3 provides realistic human and robot simulation. In the process of realistic human modeling, this work addresses the challenges posed by efficiency realism and diversity in terms of appearance and movement.\n- This work develops a Human-in-the-Loop evaluation platform within the simulator, allowing the control of humanoid robots using a mouse, keyboard, or VR devices. It provides a method for interacting and evaluating with real humans. Furthermore, it supports data collection and reproducibility during the interaction, offering a convenient tool for further research.\n- This paper introduces two benchmark tasks for human-robot interaction, along with baselines for each task. This paper leverages end-to-end RL to study collaborative behaviors and examines the performance of various learning strategies. The Human-in-the-Loop evaluation in the social rearrangement task reveals potential avenues for improving social embodied agents.\n- This simulator can be used for end-to-end reinforcement learning for robot agents, significantly reducing the time required for reinforcement learning. It also provides a validation environment for a broader range of robot agents, thus reducing potential risks to the environment and humans.", "weaknesses": "- Current robots are equipped with only a depth camera, human detector, and GPS, providing a relatively limited amount of information. It is worth considering whether additional sensor types, such as LiDAR and sound sensors, can be integrated in the future to enhance obstacle avoidance and navigation capabilities.\n- It has been noted that in human simulation, fixed hand movements can lead to a decrease in visual accuracy. It may be considered to address the deformability of the skin during the simulation process and create hand motion animations during activities like grasping and walking.\n- This simulator has a wide range of potential applications and can be further explored to implement other embedded artificial intelligence tasks, such as visual-language navigation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)