Habitat 2.0: Training Home Assistants to Rearrange their Habitat
Authors: Andrew Szot
Alexander Clegg
Eric Undersander
Erik Wijmans
Yili Zhao
John M Turner
Noah D Maestre
Mustafa Mukadam
Devendra Singh Chaplot
Oleksandr Maksymets
Aaron Gokaslan
Vladimír Vondruš
Sameer Dharur
Franziska Meier
Wojciech Galuba
Angel X Chang
Zsolt Kira
Vladlen Koltun
Jitendra Malik
Manolis Savva
Dhruv Batra
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: Benchmark
Simulator
Reinforcement Learning
Embodied AI

Abstract: We introduce Habitat 2.0 (H2.0), a simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios. We make comprehensive contributions to all levels of the embodied AI stack – data, simulation, and benchmark tasks. Specifically, we present: (i) ReplicaCAD: an artist-authored, annotated, reconfigurable 3D dataset of apartments (matching real spaces) with articulated objects (e.g. cabinets and drawers that can open/close); (ii) H2.0: a high-performance physics-enabled 3D simulator with speeds exceeding 25,000 simulation steps per second (850x real-time) on an 8-GPU node, representing 100x speed-ups over prior work; and, (iii) Home Assistant Benchmark (HAB): a suite of common tasks for assistive robots (tidy the house, stock groceries, set the table) that test a range of mobile manipulation capabilities. These large-scale engineering contributions allow us to systematically compare deep reinforcement learning (RL) at scale and classical sense-plan-act (SPA) pipelines in long-horizon structured tasks, with an emphasis on generalization to new objects, receptacles, and layouts. We find that (1) flat RL policies struggle on HAB compared to hierarchical ones; (2) a hierarchy with independent skills suffers from ‘hand-off problems’, and (3) SPA pipelines are more brittle than RL policies.

Habitat 2.0:
Training Home Assistants to Rearrange their Habitat
Andrew Szot², "Alex Clegg', Eric Undersander' Erik Wijmans .2 Yili Zhao', John Turner', Noah Maestre', Mustafa Mukadam', Devendra Chaplot', Oleksandr Maksymets'. Aaron Gokaslan', Vladimir Vondrus, Sameer Dharur Franziska Meier', Wojeiech Galuba', Angel Chang', Zsolt Kira², Vladlen Koltun Jitendra Malik Manolis Savva', Dhruv Batra 1.2 FFacebook Al Research, GGeorgia Tech, 'Intel Research, SSimon Fraser University UC Berkeley
Abstract
We introduce Habitat 2.0 (H2.0), simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios. We make comprehensive contributions all levels of the embodied AI stack data, simulation, and benchmark tasks. Specifically, present: (i) ReplicaCAD: an artist-authored, annotated, reconfigurable 3D dataset of apartments (matching real spaces) with articulated objects (e.g. cabinets and drawers that can open/close); (ii) H2.0: high-performance physics-enabee 3D simulator with speeds exceeding 25,000 simulation steps per second (850 real-time) an 8-GPU node, representing x00 speed-ups prior work; and, (iii) Home Assistant Benchmark (HAB): suite of common tasks for assistive robots (tidy the house, stock groceries, set the table) that test range of mobile manipulation capabilities. These large-scale engineering contributions allow us to systematically compare deep reinforcement learning (RI.) scale and classical sense-plan-ace (SPA) pipelines in long-horizon structured tasks, with an emphasis on generalization to new objects, receptacles, and layouts. We find that (1) flat RL policies struggle HAB compared to hierarchical ones; (2) a hierarchy with independent skills suffers from 'hand-off problems', and (3) SPA pipelines are more brittle than RL policies.
Figure 1: A mobile manipulator (Fetch robot) simulated in Habitat 2.0 performing rearrangement tasks in ReplicaCAD apartment (left) opening drawer before picking up item from it, and (right) placing object into the bowl after navigating the table. Best viewed in motion http://hhbbbia..oggg//o/saaaaaaa
1 Introduction
Consider home assistant robot illustrated in Fig. mobile manipulator (Fetch [1)) performing tasks like stocking groceries into the fridge, clearing the table and putting dishes into the dishwasher,
"Correspondence to: aszot 30gatech. edu
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

fetching objects on command and putting them back, etc. Developing such embodied intelligent systems a goal of deep scientific and societal value. So how should we accomplish this goal?
Training and testing such robots in hardware directly is slow, expensive, and difficult to reproduce. We aim to advance the entire 'research stack' for developing such embodied agents in simulation (1) data: curating house-scale interactive 3D assets (e.g. kitchens with cabinets, drawers, fridges that can open/close) that support studying generalization to unseen objects, receptacles, and home layouts, (2) simulation: developing the next generation of high- nigh-perffrmance photo-realistic 3D simulators that support rich interactive environments, (3) tasks: setting up challenging representative benchmarks to enable reproducible comparisons and systematic tracking of progress over the years. To support this long-term research agenda, we present:
ReplicaCAD: an artist-authored fully-interactive recreation of 'FRL--apartment" spaces from the Replica dataset [2] consisting of 111 unique layouts of single apartment background with 92 authored objects including dynamic parameters, semantic class and surface annotations, and efficient collision proxies, representing 900+ person-hours of professional 3D artist effort. ReplicaCAD (illustrated in figures and videos) was created with the consent of and compensation to artists, and will be shared under Creative Commons license for non-commercial use with attribution (CC-BY-NC).
Habitat 2.0 (H2.0): segh-orrformnce physics- enabled 3D simulator, representing approximately 2 years of development effort and the next generation of the Habitat project [3] (Habitat 1. 0). H2.0 supports piecewise-rigia objects (e.g. door, cabinets, and drawers that can rotate about an axis or slide), articulated robots (e.g- mobile manipulators like Fetch [1], fixed-base arms like Franka [4], quadrupeds like AlienGo [5]), and rigid-body mechanics (kinematics and dynamics). The design philosophy of H2.0 is to prioritize performance (or speed) over the breadth of simulation capabilities. H2.0 by design and choice does not support non-rigid dynamics (deformables, fluids, films, cloths, ropes), physical state transformations (cutting, drilling, welding. melting), audio or tactile sensing many of which are capabilities provided by other simulators [6-8]. The benefit of this focus is that we were able to design and optimize H2.0 to be exceedingly fast simulating a Fetch robot interacting in ReplicaCAD scenes at 1200 steps per second (SPS), where each 'step' involves rendering 1 RGBD observation (128x 128 pixels) and simulating rigid-body dynamics for 1/30 sec. Thus, 30 SPS would be considered 'real time' and 1200 SPS is 40x real-time. H2.0 also scales well - achieving 8,200 SPS x273x real-time) multi-process on single GPU and over 25,000 SPS (850x real-time) on single node with 8 GPUs For reference, existing simulators typically achieve 10-400 SPS (see Tab. 1). These 100x simulation-speedup correspond to cutting experimentation time from 6 months to under 2 days, unlocking experiments that were hitherto infeasible, allowing us to answer questions that were hitherto unanswerable. As we will show, they also directly translate to training-time speed-up and accuracy improvements from training agents (for object rearrangement tasks) on more experience.
Home Assistant Benchmark (HAB): suite of common tasks for assistive robots (Tidy House, P Prepare Groceries, Set Table) that are specific instantiations of the generalized rearrangement problem [9]. Specificallyy, mobile manipulator (Fetch) is asked to rearrange a list of objects from initial to desired positions- - picking/placing objects from receptacles (counter, sink, sofa, table), opening/closing containers (drawers, fridges) necessary. We use the GeometricGoal specification prescribed by Batra et al. [9] - i.e., initial and desired 3D (center-of- -aas)))) position of each target
;- The choice of GeometricGoal is deliberate we aim create the PointNav [10] equivalent for mobile manipulators. As witnessed in the navigation literature, such a task becomes the testbed for exploring ideas [11-19] and a starting point for more semantic tasks [20-22]. The robot operates entirely from onboard sensing - head- and arm-mounted RGB-D cameras, proprioceptive coint-position sensors (for the arm), and egomotion sensors (for the mobile base)-and may not access any privileged state information (no prebuilt maps, no 3D models of rooms or objects, no physically-implan implausible sensors providing knowledge of mass, friction, articulation of containers, etc.). Notice that an object's ' ofnter-of-mass mass provides no information about its size or orientation. The target object may be located inside a container (drawer, fridge), on top of supporting surfaces (shelf, table, sofa) of varying heights and sizes, and surrounded by clutter; all of which must be sensed and maneuvered. Receptacles like drawers and fridges start closed, meaning that the agent must open and close articulated objects to succeed. An episode is considered successful if all target objects are placed within 15cm of their desired positions (without considering orientation). The robot uses continuous end-effector control for the arm and velocity control for the base. We deliberately focus on gross motor control (the base and arm) and not fine motor control (the gripper), following
object i to be rearranged (s?,,)) .i) i=1"
2

---

Readering
Physics
Scene
Speed
Tabwary
Supports
Library
Supperts
Complexity
(stopshee)
Habitat
Magnum
scans
none
contimuous navigation (navmesh)
buikding-sca scale
3,000
A12-THD8
Unity
Unity
Unity
righd dynamics, animated interactions
room-saale
ManipulaTTIOR [33]
Unity
Unity
Unity
A12 112-THORR manipulation
room scale
30 30-40
ThreeDWorrld [7)
Unity
Unity
Unity (Pays2) + HLEX
ngiid particee dynamics nidioairiuulaeed dynamics
rooeshouse seale
3- 168
SAPIEN [34]
OpenGiLOOpuX
contigurable
PhysX
object level 200
4000
RL.Reach
CoppeliaSian (Openkit.)
Gourand shading
CoppelisSim (BulleuODE))
rigidiariculated dynamics
table-nop
iGibson [36]
PyRender
shadlng
PyBullet
rigid/aniculated dynamics
Habitot (H2.0)
Magstum
scans
PBR shading
Bullet
digdiarticulated dynanics novmesh
house-scale
1,200
Table 1: High-level comparison of different simulators. Note:
Speeds were taken directly from respective
publications or obtained via direct personal correspondence with the authors when not publicly available (indicated by 1). Benchmarking was conducted by different teams on different hardware with different underlying 3D assets simulating different capabilities. Thus, these should be considered qualitative comparisons representing what user expects to experience on a single instance of the simulator (no parallelization..
the "abstracted grasping' recommendations from [9]. Specifically, once the end-effector reaches 15em (or closer) to an object, a discrete grasp action becomes available that, if executed, snaps the object into its parallel-jaw gripper 2. We conduct systematic study of two distinct techniques monolithic "sensors-to-acee to-atttott policies trained with reinforcement learning (RI.) at scale, and classical sense- plan-act pipelines (SPA) [26] with a particular emphasis on systematic generalization to new objects, receptacles, apartment layouts (not just robot starting pose). Our findings include:
1. Flat vs hierarchical:
Monolithic RL policies successfully learn diverse individual skills (pick/place, navigate, open/close drawer). However, crafting combined reward function and learning scheme that elicits chaining of such skills for the long-horizon HAB tasks remained out of our reach. We saw significantly stronger results with a hierarchical approach that assumes knowledge of a perfect task planner (via STRIPS [27]) to break it down into a sequence of skills. Hierarchy cuts both ways: However, a hierarchy with independent skills suffers from 'hand-off' problems' where succeeding skill isn't set up for success by the preceding one e.g., navigating to bad location for subsequent manipulation, only partially opening a drawer to grab an object inside, or knocking an object out of reach that is later needed.
3. Brittleness of SensePlanAct: For simple skills, SPA performs just as well as monolithic RL. However, it is significantly more brittle since it needs to map all obstacles in the workspace for planning. More complex settings involving clutter, challenging receptacles, and imperfect navigation can poorly frame the target object and obstacles in the robot's ' camera, leading to incorrect plans.
We hope our work will serve as benchmark for many years to come. H2.0 is free, open-sourced under the MIT license, and under active development." 3 We believe it will reduce the community's s reliance on commercial lock-ins [28, 29] and non oon-photorraaliticc simulation engines [30-32].
2 Related Work
What is a simulator? Abstractly speaking, a simulator has two components: (1) a physics engine that evolves the world state over time S; - St+1. and (2) renderer that generates sensor observations from states: - og. The boundary between the two is often blurred as a matter of convenience. Many physics engines implement minimal renderers to visualize results, and some rendering engines include integrations with a physics engine. PyBullet [37], MuJoCo [28], DART [38], ODE [39], PhysX/FleX [40, 41], and Chrono [42] are primarily physics engines with some level of rendering, while Magnum [43]. ORRB [44], and PyRender [45] are primarily renderers. Game engines like Unity [46] and Unreal [47] provide tightly coupled integration of physics and rendering. Some simulators [3, 48, 49] involve largely static environments - the agent can move but not change the state of the environment (e.g. open cabinets). Thus, they are heavily invested in rendering with fairly lightweight physics (e.g. collision checking with the agent approximated as a cylinder).
How are interactive simulators built today? Either by relying on game engines [6. 50, 51] or via a 'homebrew' integration of existing rendering and physics libraries [7, 34, 36, 52]. Both options have problems. Game engines tend be optimized for human needs (high image-resolution, ~60 FPS, persistent display) not for Al's needs [53] (10k+ FPS, low-res, 'headless' deployment on
"To be clear, H2.0 fully supports the higidbbddy mechanics of grasping: the abstract grasping is a level simplification that can be trivially undone. Grasping. in-hand manipulation. and goal-directed releasing of grasp are all challenging open research problems [23-25] that we believe must further mature in the fixed-based close-range setting before being integrated into long-horizon home-scale rearrangement problem. Start using Habitat 2.0 with the tutorial at https://iiaabiaatoo///oos//////
3

---

cluster). Reliance on them leads to limited control over the performance characteristics. On the other hand, they represent decades of knowledge and engineering effort whose value cannot be discounted. This is perhaps why 'homebrew' efforts involve high-level (typically - integration of existing libraries. Unfortunately but understandably, this results in simulation speeds of 10-100s of SPS, which is orders of magnitude sub-optimal. H2.0 involved deep low-level (C++) integration of rendering (via Magnum [43]) and physics (via Bullet [37]), enabling precise control of scheduling and task-aware optimizations, resulting in substantial performance improvements.
Object rearrangement. Task- and motion- planning [54] and mobile manipulation have a long
history in Al and robotics, whose full survey is beyond the scope of this document. Batra et al. [9] provide a good summary of the historical background of rearrangement, a review of recent efforts, a general framework, and a set of recommendations that we adopt here. Broadly speaking, our work distinguished from prior literature by a combination of the emphasis on visual perception, lack of access to state, systematic generalization, and the experimental setup of visually ecologically-realinn home-scale environments. We now situate w.r.t. a few recent efforts. [55] study replanning in the presence of partial observability but do not consider mobile manipulation. [52] tackle 'interactive navigation', where the robot can bump into and push objects during navigation. but does not have an arm. Some works [56-58] abstract away gross motor control entirely by using symbolic interaction capabilities (e.g. a 'pick up X' action) or a 'magic pointer' [9]. We use abstracted grasping but not abstract manipulation. [19] develop hierarchical methods for mobile manipulation, combining RL policies for goal- geuereennnttion and motion motion-planninn planning for exccuting them. We use the opposite combination of planning and learning- using task-plamnn planning generate goals and RL for skills. [33] is perhaps the most similar to our work. Their task involves moving a single object from one location another, excluding interactions with container objects (opening a drawer or fridge to place an object inside). We will see that rearrangement of multiple objects while handling containment is a much more challenging task. Interestingly, our experiments show evidence for the opposite conclusion reached therein monolithic end-to-end trained RL methods are outperformed by a modular approach that trained stage-wise to handle long-horizon rearrangement tasks.
and
3 Replica to ReplicaCAD: Creating Interactive Digital Twins of Real Spaces
We begin by describing our dataset that provides a rich set of indoor layouts for studying rearrange- ment tasks. Our starting point was Replica [2], dataset of highly photo-realistic 3D reconstructions at room and building scale. Unfortunately. static 3D scans are unsuitable for studying rearrangement tasks because objects in static sean cannot be moved manipulated.
Figure 2: Left: The original Replica scene. Right: the artist recreated scene ReplicaCAD. All objects (furniture, mugs) including articulated ones (drawers, fridge) in ReplicaCAD are fully physically simulated and interactive.
Asset Creation. ReplicaCAD is an artist-created, fully-interactive recreation of 'RRL-apartment" spaces from the Replica dataset [2]. First, team of 3D artists authored individual 3D models (geometry, textures, and material specifications) faithfully recreate nearly all objects (furniture, kitchen utensils, books, etc.; 92 in total) in all 6 rooms from the FRL-apartment spaces well as an accompanying static backdrop (floor and walls). Fig. compares a layout of ReplicaCAD with the original Replica scan. Next, each object was prepared for rigid-body simulation by authoring physical parameters (mass, friction, restitution), collision proxy shapes, and semantic annotations. Several objects (e.g. refrigerator, kitchen counter) were made 'articulated" through sub-part segmentation (annotating fridge door, counter cabinet) and authoring of URDF files describing joint configurations (e.g. fridge door swings around hinge) and dynamic properties (e.g. joint type and limits). For each

---

large furniture object (e.g. table), we annotated surface regions (e.g. table tops) and containment volumes (e.g. drawer space) enable programmatic placement of small objects on top of or within.
Human Layout Generation. Next, a 3D artist authored an additional 5 semantically plausible 'macro variations' of the scenes producing new scene layouts consisting only of larger furniture from the same 3D object assets. Each of these macro variations was further perturbed through 20 "micro variations' that re-positioned objects - e.g. swapping the locations of similarly sized tables or a sofa and two chairs. This resulted in a total of 105 scene layouts that exhibit major and minor semantically meanniiguull variations in furniture placement and scene layout, enabling controlled testing of generalization. Illustrations of these variations can be found in Appendix A.
Procedural Clutter Generation. To maximize the value of the authored assets we also develop a pipeline that allows us to generate new clutter procedurally. Specifically, we dynamically populate the annotated supporting surfaces (e.g- table-top, shelves in cabinet) and containment volumes (e.g. fridge interior, drawer spaces) with object instances from appropriate categories (e.g., plates, food items). These inserted objects can come from ReplicaCAD or the YCB dataset [59]. We compute physically-stabbe insertions of clutter offline (i.e. letting an inserted bowl 'settle' on a shelf) and then load these stable arrangements into the scene dynamically at run-time.
ReplicaCAD is fully integrated with the H2.0 and a supporting configuration file structure enables simple import, instancing, and programmatic alternation of any of these interactive scenes. Overall, ReplicaCAD represents 900+ ceosons of professional 3D artist effort so far (with augmentations in progress). It was created with the consent of and compensation under a Creative Commons license for non-commercial use with attribution (CC-BY-NC). Further ReplicaCAD details and statistics are in Appendix A.
artists, and will be shared
Habitat 2.0 (H2.0): a Lazy Simulator
H2.0's design philosophy is that speed is more important than the breadth of capabilities. H2.0 achieves fast rigid-bodi simulation in large photo-realistic 3D scenes by being lazy and only simulat- ing what is absolutely needed. We instantiate this principle via key ideas localized physics and rendering (Sec. 4.1,, interleaved physics and rendering (Sec, 4.2), and simplify-and-rent (Sec. B.1). We also describe motion planning integration in Appendix B.2.
4.1 Localized Physies and Rendering
Realistic indoor 3D scenes can span houses with multiple rooms (kitchen, living room). hundreds of objects (sofa, table, mug) and 'containers' (fridge, drawer, cabinet), and thousands of parts (fridge shelf, cabinet door). Simulating physics for every part at all times is slow and unnecessary. We leverage Bullet's built-in island sleep system to minimize simulation overhead for idle objects. In addition, we make several optimizations: (1) We employ a navigation mesh to move the robot base kinematically (which has been shown transfer well to real the world [60]) rather than simulating wheel-ground contact. (2) For multi-body articulated furniture, we remove static parts (e.g. the walls and floor of a cabinet) from the Bullet multi-body and instead load these as separate static rigid objects. This improves the sleeping behavior of the entire simulation, for example, an idle object resting on the floor of the cabinet can sleep even while the cabinet door is moving. sleeping state of objects to optimize rendering by caching and re-using scene graph transformation matrices and frustum-culling results.
(3) We use the
4.2 Interleaved rendering and physics
Most physics engines (e.g. Bullet) run on the CPU, while rendering (e.g. via Magnum) typically occurs on the GPU. After our initial optimizations. we found each to take nearly equal compute- compute-time. This represents a glaring inefficiency as illustrated in Fig. 3, at any given time either the CPU is sitting idle waiting for the GPU or vice-versa. Thus, interleaving them leads to significant gains. However, this is complicated by a sequential dependency state transitions depend on robot actions T : (st, at) + St+1, robot actions depend on the sensor observations: æ Or - at and observations depend on the state O 8, - 0,. Thus, it ostensibly appears that physics and rendering outputs (s#+1- o₂) cannot be computed in parallel from &, because computation of a, cannot begin till o, is available. We break this sequential dependency by changing the agent policy to be π [ oo-1) instead of [a; or). Thus, our agent predicts the current action a, not from the current observations o, but from an observation from timestep ago O essentially 'living in the past and acting in the future'.
5

---

This simple change means that we can generate 8141 on the CPU the same time as o is being generated on the GPU.
Sequential
Agent is interacting
Agent is navigating
Phystes (CPU;
Rencering (GRU)
Physacs (CPU)
Rendering
(GPU)
This strategy not only increases simulation throughput, but also offers two other fortuitous benefits increased biological plausibility and improved sim2real transfer potential. The for- mer is due to closer analogy to all sensors (bi- ological or artificial) having a sensing latency (e.g.. the human visual system has approximately 150ms latency [61]). The latter is due to a line of prior work [62-64] showing that introducing this latency in simulators improves the transfer of learned agents reality.
9
9
PH4k
Spece1
Quickt
Interleaved
Agent is interacting
Agent is navigating
Physizz (CPU;
Physiios (CHU)
Rendering (SPU))
Rondenin (GPU))
-t-1
Ptok
Steke1
Ptek
Wall Clock Time
Figure 3: Interleaved physics and rendering. Top shows the normal sequential method of performing physics (st, at) + S1+1 then rendering $c+1 + 0c+1. Bottom shows H2.0's interleaved physics and rendering.
4.3 Benchmarking
We benchmark using Fetch robot, equipped with (up to) two RGB-D cameras (128x 128 pixels) in ReplicaCAD scenes under three scenarios: (1) Idle-1 with the robot initialized in the center of the living room somewhat far from furniture or any other object and taking random actions and equipped with a single RGB camera, (2) RGB-D, Idle-RGB with two RGB-D cameras, (3) Interact: with the robot initialized fairly close to the fridge and taking actions from pre-computed trajectory that results in representative interaction with objects and equipped with two RGB-D cameras. Each simulation step consists of I rendering pass and 4 physics-steps, each simulating V/20 see for a total of 1/30 sec. New joint position goals are set every 1/30 sec and a joint controller computes the joint torques to achieve the joint goals for the current joint state every 1/120 sec. This is a fairly standard experimental configuration in robotics (with 30 FPS cameras and 120 Hz control). In this setting, simulator operating at 30 steps per (wallclock) second (SPS) corresponds to 'real time'.
Benchmarking was done on machines with dual Intel Xeon Gold 6226R CPUs - 32 cores/64 threads (32C/64T) total and 8 NVIDIA GeForce 2080 Ti GPUs. For single-GPU benchmarking processes are confined to 8C/16T of one CPU, simulating an 8C/16T single GPU workstation. For single-GPU multi-proces, benchmarking, 16 processes were used. For multi-GPU benchmarking, 64 processes were used with 8 processes assigned to each GPU, We used python-3.8 and gcc-9.3 for compiling H2.0. We report average SPS over 10 runs and 95% confidence-interval computed via standard error of the mean. Note that 8 processes do not fully utilize 2080 Ti and thus multi-process multi-GPU performance may be better machines with more CPU cores.
I Process
1 GPU
8 GPUs
Idle LxRGB
Idle 2xRGB-D
Interact 2x RGB-D
Idie LXRGB
Idie
Interact
Idic
Idie
Interact 2xRGB-D
2x 2xRBBDD
2x D
LxRGB
2x RGB-D
H2.0 (Full)
1191
=
669 111
510
18
8186
147
1926
119 1660
15
25734
1
9542
171
7699
1:77
render opts.
781
364
282
43
6709
1076
1035
#3
18844
#785
6397
5517
4M1
physics opts.
271
21
252
17
358
16
2290
15
1270
130
1606
15
7942
1500
5535
141
6119
1.1
opts.
242
177
224
2223
814
7192
3965
4829
430
Table 2: Benchmarking H2.0 performance: simulation steps per second (higher better) over 10 runs and a 95% confidence-intend interval In Idle. the agent is executing random actions but not interacting with the scene, while Interact uses precomputed trajectory and thus results in representative interaction with objects. To put these numbers into context, see Tab. 1. Reproduce these numbers at http://aaaahiiaaoordd//hhaaaaaaa
Table 2 reports benchmarking numbers for H2.0. We make a few observations. The ablations for H2.0 (denoted by 1- render opts', '-physics opts', and '-all opts.' show that principles followed in our system design lead to significant performance improvements.
Our 'Idle- 1 XRG' setting is similar to the benchmarking setup of iGibson [36], which reports 100 SPS. In contrast, H2.0 single-prn single-process with all optimizations turned off is 240% faster (242 vs 100 SPS). H2.0 single-process with optimizations on is ~ 1200% faster than iGibson (1191 vs 100 SPS). The comparison to iGibson is particularly illustrative since uses the 'same' physics engine (PyBullet) as H2.0 (Bullet). We can clearly see the benefit of working with the low-level C++ Bullet rather than PyBullet and the deep integration between rendering and physics. However, we note the comparison between the two benchmarks is not exact since the robot type, number of objects, and

---

object assets are different. A direct comparison against other simulators is not feasible due to different capabilities, assets, hardware, and experimental settings. But a qualitative order-of -f-ganitudde survey is illustrative A12-THOR [6] achieves 60/30 SPS in idle/interact, SAPIEN [34] achieves 200/400 SPS (personal communication), TDW [7] achieves 5 SPS in interact, and RLBench [35] achieves between and 60 SPS depending on the sensor suite (personal communication), Finally, H2.0 scales well achievinng 8,186 SPS (272x x real-time) multi-process on a single GPU and 25,734 SPS (850x real-time) on a single node with 8 GPUs. These x00x simulation-speedde experimentation time from 6-month cycle to under 2 days. iGibson also supports multi-process parallization for speeding simulation speeds beyond the reported single process numbers in the current version of the paper [36], and we recommend following updates of their work for more details.
correspond to cutting
The Pick Task: a Base Case of Rearrangement
We first carry out systematic analyses on a relatively simple robotic manipulation task: picking up one object from cluttered 'receptacle'. This forms a 'base case' and an instructive starting point that we eventually expand to the more challenging Home Assistant Benchmark (HAB) (Sec. 6).
Task Definition:
Pick (s"). Fig. 4 illustrates an episode in the pick task. Our agent (a Fetch robot (1)) spawned close a receptacle (a table) that holds multiple objects (e.g. cracker box, bowl). The task for the robot is to pick up target object with center-of-maat coordinates s E: R (provided in robot's coordinate system) as efficiently as possible without excessive collisions. We study systematic gener- alization to new clutter layout the receptacle, to new objects, and to new receptacles. Agent em- bodiment and sensing. Fetch [1] is a wheeled base with a 7-DoF arm manipulator and a parallel-jaw gripper, equipped with two RGBD cameras (90' FoV, 128 x 128 pixels) mounted on its 'head" and arm. It can sense its proprioceptive-stace dim), end-effector position (3-dim), and base-egomotion
arm joint angles (7-
(6-dim, also known as GPS+Compass in the navigation
literature [3]). Note: the episodes in Pick are constructed such that the robot does not need to move its base. Thus, the egomotion sensor does not play role in Pick but will be important in HAB tasks (Section 6).
Action space: gross motor control. The agent performs end-effector control at 30Hz. At every step, it outputs the desired change in end-effector position (dz, 8y, 82); the desired end-effector position is fed into an inverse kinematics solver from PyBullet [37] to derive desired states for all joints, which are used to set the joint motor targets, achieved using PD control. The maximum end- effector displacement per step is 1.5cm, and the maximum impulse of the joint motors is IONs with a position gain of Kp=0.3. In Pick, the base is fixed but in HAB, the agent also emits linear and angular velocities for the base.
0
S
Figure 4: Fetch with head and cameras picking up a howl from the counter.
Abstracted grasping. The agent controls the gripper by emitting scalar. If this scalar is positive and the gripper is not currently holding an object and the end-effector is within 15cm of an object, then the object closest the end-effector is snapped into the parallel-jaw gripper. The grasping is perfect and objects do not slide out. If the scalar is negative and the gripper is currently holding an object, then the object currently held in the gripper is released and simulated as falling. In all other cases, nothing happens. For analysis of other action spaces see Appendix F.6.
Evaluation. An object is considered successfully picked if the arm returns known 'resting position" with the target object grasped. The agent fails the accumulated contact force experienced by the arm/body exceeds threshold of 5k Newtons. If the agent picks up the wrong object, the episode terminates. Once the object is grasped, the drop action is masked out meaning the agent will never release the object. The episode horizon is 200 steps.
Methods. We compare two methods representing two distinctive approaches to this problem: 1. MonolithieRI: a sensors-to-actions policy trained end-to-end with reinforcement learning (RL). The visual input is encoded using a CNN, concatenated with embeddings of propriaceptive-sensing and goal coordinates, and fed to recurrent actor-critic network, trained with DD-PPO [11] for 100
7

---

Million steps of experience (see Appendix C for details). This baseline translates our community's s most-successful paradigm yet from navigation to manipulation.
2. SensePlanAct (SPA) pipeline:
Sensing consists of constructing an accumulative 3D point-cloud of the scene from depth sensors, which is then used for collision queries. Motion planning is done using Bidirectional RRT [65] in the arm joint configuration space (see Appendix D). The controller was described in "Action Space' above and is consistent with MonolithicRL. We also create Sateeelllant--riiieeddeee
(SPA-Prív), that uses privileged information perfect knowledge of scene geometry (from the simulator) and perfect controller (arm is kinematically set to desired joint poses). The purpose of this baseline is to provide an upper-bound on the performance of SPA.
Systematic Generalization. With H2.0 we can com- pare how learning based systems generalize com- pared to SPA architectures. Tab. 3 shows the results of a systematic generalization study of 4 unseen ob- jects, 3 unseen receptacles, and 20 unseen apartment layouts (from unseen 'macro variation' in Replica- CAD). In training the agent sees objects from the YCB dataset kitchen and food categories (chef can, cracker box, sugar box, tomato soup can, tuna fish cap. pudding box, gelatin box, potted meat can, and bowl). During evaluation it is tested on unseen objects (apple, orange, mug, sponge). Likewise, the agent is trained on the counter, sink, light table, cabinet, fridge, dark table, and sofa receptacles (view in Fig. 11) but evaluated on the unseen receptacles of tv stand, shelves, and chair (view in Fig. 12).
Unseen
Method
Seen
Layouts
Objects
Receptacles
MonolithicRI.
91.7 11.1 86.3 11.4 74.7 11.
52.7 120
SPA
70.2 21.9 72.7 ai8
72.7 214
60.3 #20
SPA-Priv
77.0 +1.7 80.0 +1.0 79.2 #1.7 60.7 +20
Table 3: Pick generalization analysis: success rates with mean and standard error on 600 episodes (and across 3 seeds for MonolithicRL).
MonolithicRL generalizes fairly well from seen to unseen layouts (91.7 + 86.3%), significantly outperforming SPA (72.7%) and even SPA-Priv (80.0%). However, generalization to new objects is challenging (91.7 + 74.7%) as a result of the new visual feature distribution and new object obstacles. Generalization to new receptacles is poor (91.7 - 52.7%). However, the performance drop of SPA (and qualitative results) suggest that the unseen receptacles (shelf, armchair, tv stand) may be objectively more difficult to pick up objects from since the shelf and armchair are tight constrained areas whereas the majority of the training receptacles, such as counters and tables, have no such constraints (see Fig. 12). We believe the performance of MonolithicRI, will improve as more receptacles 3D assets become available since the training distribution was only 4 receptacles. We cannot make any such claims for SPA.
In the supplementary we also analyze different sensor input modalities (Appendix F.I), the surprising success of "blind" policies (Appendix F.2), the effect of different camera placements (Appendix F.3), different action spaces (Appendix F.6), the effect of the time delay on performance (Appendix F.5), and qualitative evidence of self- ooff-tracking (Appendix F.4).
6 Home Assistant Benchmark (HAB)
We now describe our benchmark of common household assistive robotic tasks. We stress that these tasks illusirate the capabilities of H2.0 but do not delineate them lot more is possible but not feasible to pack into a single coherent document with clear scientific takeaways.
Task Definition. We study three (families of) long-range tasks that correspond to common activities:
1. Tidy House: Move 5 objects from random (unimpeded) locations back to where they belong (see Fig. 19a). This task requires no opening or closing and no objects are contained.
Start: 5 target objects objects spawned in 6 possible receptacles (excluding fridge and drawer).
Goal: Each target object is assigned a goal in different receptacle than the starting receptacle.
Task length: 5000 steps.
2. PrepareGrocer in the fridge (see Fig. 19b). This task requires no opening or closing and no objects are contained.
Remove 2 objects from the fridge to the counters and place one object back
Start: 2 target objects in the fridge and one on the left counter. The fridge fully opened.
Goal: The goal for the target objects in the fridge are on the right counter and light table. The goal
for the other target object in the fridge.
Task length: 4000 steps
8

---

3. Table: Get a bowl from a drawer, fruit from fridge, place the fruit in the bowl on the table (see Fig. 19c).
Start: A target bowl object is in one of the drawers and a target fruit object in the middle fridge shelf. Both the fridge and drawer start closed.
Goal: The goal for the bowl is the light table, the goal for the fruit is top of the bowl. Both the fridge and drawer must be closed.
Task length: 4500 steps.
The list is in increasing order of complexity from no interaction with containers (Tidy House), to picking and placing from the fridge container (Prepare Groceries), to opening and closing containers (Set Table). Note that these descriptions are provided purely for human understanding; the robot operates entirely from a GeometricGool specification [9] given by the initial and desired
3D (center-of-mast position of each target object i to be moved (s.. st)
N and se is a constant (arm resting) location. - Thus, Pick (s.) is each task episode, we
special case where sample a ReplicaCAD layout with YCB [59] objects randomly placed on feasible placement regions (see procedural clutter generation in Section 3). Each task has 5 clutter objects per receptacle. Unless specified, objects are sampled from the 'oood' and 'kitchen' YCB item categories in the YCB dataset.
The agent is evaluated on unseen layouts and configurations of objects, and so cannot simply memorize. We characterize task difficulty by the required number of rigid-body transitions (e.g., picking up a bowl, opening a drawer). The task evaluation, agent embodiment, sensing, and action space remain unchanged from Section 5, with the addition of base control via velocity commands. Details on episode statistics, as well as the evaluation protocols are in Appendix G.
Methods. We extend the methods from Sec. 5 to better handle the above long-horizon tasks with high-level STRIPS planner using a parameterized set of skills: cick, place, Open fri rigde door, Close Fri dge door, Open drawer, Close drawer, and Navi gate. The full details of the planner implementation and how methods are extended are in Appendix H. Here, we provide a brief overview.
1. MonolithicRL: Essentially unchanged from Sec. 5, with the exception of accepting a list of start and goal coordinates (sf. st)""> as opposed to just sq.
2. TiskPPaaninn+SKkillRRLL (TP+SRL): hierarchical approach that assumes knowledge of a perfect task planner (implemented with STRIPS (27]) and the initial object containment needed by the task planner to break down a task into a sequence of parameterized skills: Navigate, Pick, Place, Open fridge door, Close fridge door, Open drawer, Close drawer. Each skill is functionally identical to MonolithicRL in Sec. 5 - taking as input a single 3D position, either se or sf. For instance, in the Set task, let (a+, aa) and (1), bb) denote the start and goal positions of the apple and bowl, respectively. The task planner converts this task into: Transport Bowl
Open Drawer
Drawer
Close Drawer
Navi Navigate(")) Open drawer(b"), Picke Pbck(b"), Navigate("))
Place(b"), Navigate Close drawer(b"),
Navi (a ) Open fri doar(a Navi (ate Place(a"). Navigate Close fridg rridge door(a")
Open Fridge
Transpoct
Close Fridge
Simply listing out this sequence highlights the challenging nature of these tasks.
3. TassPlaanigg+SSnsellaaaae (TP+SPA): Same task planner as above, with each skill imple- mented via SPA from Sec. 5 except for Navigate where the same learned navigation policy from TP+SPA is used. TP+SPA-Pris is analogously defined. Crafting an SPA pipeline for opening/closing unknown articulated containers is an open unsolved problem in robotics involving detecting and tracking articulation [66, 67] without models, constrained full-body planning [68-70] without hand engineering constraints, and designing controllers to handle continuous contact [71, 72] making it out of scope for this work. Thus, we do not report TP+SPA on Set Table.
Results and Findings. Figure 5 shows progressive success rates for different methods on all tasks. Due to the difficulty of the full task, for analysis, the X-axis lists the sequence of agent-enviromment interactions (pick, place, open, close) required to accomplish the task, same as that used by the task-planner." The number of interactions is a proxy for task difficulty and the plot is analogous to precision-recall curves (with the ideal curve being straight line at 100%). Furthermore, since
'This sequence from the task plan is useful for experimental analysis and debugging. but does not represent the only way to solve the task and should be disposed in future once methods improve the full task.

---

MonelthicAL
TP+SRL
TP+SPA
TP+SPAPP
TA+SAL TOrace Navi
TP-SPA (Oradle Novi
TP+SPA (Oradi Navi
racs
NCc
PCC
TO0 -
Sucoee bate
(a) Tidy House
(b) TrepareGrocerii
(c) Set Table
Figure 5: Success rates for Home Assistant Benchmark tasks. Due the difficulty of full HAB tasks, analyze performance as completing a part of the overall task. For the TP methods that use an explicit navigation skill, we indicate with an arrow the interaction names where navigation occurs and include versions for learned and oracle navigation. Results are on unseen layouts with mean and standard error computed for 100 episodes.
navigation is often executed between successive skills, we include versions of the task planning methods with oracle navigation skill. We make the following observations (See Appendix I for skill learning curves and SPA failure statistics):
1. MonolithicRL performs abysmally. We were able to train individual skills with RL to reasonable degrees of success (see Appendix 1.2). However, crafting a combined reward function and learning scheme that elicits chaining of such skills for a long-horizon task, without any architectural inductive bias about the task structure, remained out of our reach despite prolonged effort.
2. Learning a navigation policy to chain together skills is challenging as illustrated by the performance drop between learned and oracle navigation. In navigation for the sake of navigation (PointNav (10),, the agent is provided coordinates of the reachable goal location. In navigation for manipulation (Navigate), the agent is provided coordinates of target object's center-of-mass but needs to navigate to an unspecified non-unique suitable location from where the object is manipulable.
3. Compounding errors hurt performance of task planning methods. Even with the relatively eas- ier skills in Tidy House in Figure 5a all methods with oracle navigation gradually decrease in performance as the number of required interactions increases.
Sense-plan-act variants scale poorly to increasing task complexity. In the easiest setting, Tidy House with oracle navigation (Figure 5a), TP+SPA performs better than TP+SRL. However, this trend is reversed with learned navigation since TP+SPA methods, which rely on egocentric perception for planning, are not necessarily correctly positioned to sense the workspace. In the more complex task of P Prepare Groceries (Figure 5b), TP+SRL outperforms TP+SPA both with and without oracle navigation due to the perception challenge of the tight and cluttered fridge. TP+SPA fails to find a goal configuration 3x more often and fails to find a plan in the allowed time 3x more often in Prepare Groceries than Tidy House.
7 Societal Impacts, Limitations, and Conclusion
ReplicaCAD was modeled upon apartments in one country (USA). Different cultures and regions may have different layouts of furniture, types of furniture, and types of objects not represented in ReplicaCAD; and this lack of representation can have negative social implications for the as- sistants developed. While H2.0 is fast simulator, we find that the performance of the overall simulation+trainingg loop is bottlenecked by factors like synchronization of parallel environments and reloading of assets upon episode reset. An exciting and complementary future direction holistically reorganizing the mnndering-physics--RR there is idle GPU time when rendering is faster than physics, because inference waits for both o and 8141 to be ready despite not needing $t-1- This is done because existing RL training systems expect the reward rt to be returned when the agent takes an action Ge, but r is typically a function of s;, ar. and $1+1. Reorganizing the rendering++hyyiis--R. interplay is exciting problem for future work.
interplay as studied by [73-78]. As illustrated in Figure 3,
We presented the ReplicaCAD dataset, the Habitat 2.0 platform and home assistant benchmark. 3D simulator that enables efficient experimentation H2.0 is fully interactive, rearranging richly interactive 3D environments. Coupled with the involving embodied Al agents ReplicaCAD data these improvements allow us to investigate the performance of RL policies against classical MP approaches for the suite of challenging rearrangement tasks we defined. We hope that the Habitat 2.0 platform will catalyze work embodied AI for interactive environments.
10

---

8 Acknowledgements
Funding in direct support of this work: The Georgia Tech portion of this work supported by state funds from Georgia Tech (AS, ZK), NSF (DB), AFRL (DB), DARPA (DB), ONR YIPs (DB), ARO PECASE (DB), Amazon (DB). The SFU portion of this work is supported by a CIFAR Al Chair (AC), a Canada Research Chair (MS), and NSERC Discovery Grants (AC,MS). This work was also supported by FAIR (AC, EU, YZ, JTT, NM, MM, DC, OM, AG, FM, WG) and Intel (VK).
Additional revenues related to this work: NSF (ZK), DARPA (ZK), ONR (ZK), NGA (ZK), Samsung (ZK). Airbus (ZK), consulting for Marble Inc (ZK), paid talk by Data Science Connect (ZK), sponsored research funding by FAIR (AC.MS).
References
[1] Fetch robotics. Fetch. http://fett ch robot ccm/ 2020.
[2] Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel,
Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:1906.05799
2019.
[3]
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, et al. Habitat: A Platform for Embodied Al Research, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9339-9347, 2019.
[4] Franka. Franka emika specification. https: / www, franka de. 2020.
[5] Unitree robotics. Aliengo. ht https: www. unitree com, 2020.
[6] Eric Kolve, Roozbeh Mottaghi, Winson Han, VanderBilt, Luca Weihs, Alvaro Herrasti, Daniel Gordon,
Yuke Zhu, Abhinav Gupta, and Ali Farhadi. A12-Thor: An interactive 3D environment for visual AI.
arXiv preprint arXiv: 2017.
[7] Chuang Gan, Jeremy Schwartz, Seth Alter, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber. Megumi Sano, al. DhrldDWorld: A platform for interactive multi-modal physical simulation. arXiv preprint arXiv:2007.044 04.45444444 2020.
[8] Daniel Seita, Pete Florence, Jonathan Tompson. Erwin Coumans, Vikas Sindhwani, Ken Goldberg. and Andy Zeng. Learning to Rearrange Deformable Cables, Fabries, and Bags with Transporter Networks. In IEEE International Conference on Robotics and Automation (ICRA), 2021. [9] Dhruv Batra, Aneel X Chang, Sonia Chernova, Andrew J Davison, Jia Deng, Vladlen Koltun, Sergey Levine, Jitendra Malik, Igor Mordatch, Roozbeh Mottaghi, Manolis Savva, and Hao Su, Rearrangement: A challenge for embodied AI. arXiv preprint 1,0099,, 2020.
[10] Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik. Roozbeh Mottaghi, Manolis Savva, et al. On evaluation of embodied navigation agents, arXiv preprint arXiv:1807 80770657 2018.
[11] Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. DD-PPO: Learning near- perrept pointgoal navigators from 2.5 billion frames. In International Conference Learning Representations (ICLR), 2020.
[12] Erik Wijmans, Irfan Essa, and Dhruv Batra. How to train pointgoal navigation agents on a (sample and compute) budget. arXiv preprint arXiv:2012.061 2020.
[13] Joel Ye, Dhruv Batra, Erik Wijmans, and Abhishek Das. Auxiliary tasks speed up learning pointgoal
navigation. arXiv preprint arXiv:2007.04500
2020.
[14] Yilun Du, Chuang Gan, and Phillip Isola. Curious representation learning for embodied intelligence.
arXiv preprint arXiv:2105.0 000000 2021.
[15) Peter Karkus, Shaojun Cai, and David Hsu. Differentiable slam- net: Learning particle slam for visual navigation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[16] Claudia Pérez-D' Arpino, Can Liu, Patrick Goebel, Roberto Martín-Martin and Silvio Savarese. Robot navigation in constrained pedestrian environments using reinforcement learning. Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2021.
[17] Santhosh K. Ramakrishuan, Ziad Al-Halah, and Kristen Grauman. Occupancy anticipation for efficient exploration and navigation. In ECCV, 2020.
[18] Somil Bansal, Varun Tolani, Saurabh Gupta, Jitendra Malik, and Claire Tomlin. Combining optimal control and lcarning for visual navigation in novel environments. In Conference on Robot Learning (CoRL), 2019.
[19] Fei Xia, Chengshu Li, Roberto Martín- Martin-Martín, Or Litany, Alexander Toshev, and Silvio Savarese. Relmo- gen: Leveraging motion generation in reinforcement learning for mobile manipulation. In Proceedings IEEE International Conference on Robotics and Automation (ICRA), 2021.
[20] Dhruv Batra, Aaron Gokaslan, Aniruddha Kembhavi, Oleksandr Maksymets, Roozbeh Mottaghi, Manolis Savva, Alexander Toshev, and Erik Wijmans. Objectnav revisited: On evaluation of embodied agents navigating to objects. arXiv preprint arXiv:2006.1 13171, 2020.

---

[21] Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. Room-across Multilingual vision-and-language navigation with dense spatiotemporal grounding. arXiv preprint urXiv:2010.07954, 2020.
[22] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and- navigation: Interpreting navigation instructions in real environments. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3674-3683, 2018.
[23] Adityyavairavan Murali, Arsalan Mousavian, Clemens Eppner; Chris Paxton, and Dieter Fox. 6-dof grasping for target-driven object manipulation in clutter. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 6232-6238. IEEE, 2020.
[24] Jeannette Bohg. Antonio Morales. Tamim Asfour, and Danica Kragic. Data-driven grasp synthesis
survey. IEEE Trunsactions on Robotics, 30(2):28--30, 2013.
[25] Kaiyu Hang, Miao Li, Johannes A Stork, Yasemin Bekinoglu, Florian Pokorny, Aude Billard, and Danica Kragic. Hierarchical fingertip space: A unified framework for grasp planning and in-hand grasp adaptation. IEEE Transactions on robotics, 32(4);960-99) 900-92,. 2016.
[26]
Robin R Murphy. Introduction to Al robotics. MIT press, 2019.
[27]
Richard Fikes and Nils Nilsson. Strips: A new approach to the application of theorem proving to problem solving. Artificial intelligence, 2(3-4):189-208, 1971.
[28] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model control. In
2012 IEEE/RSJ International Conference Intelligent Robots and Systems, pages 5026 5033. IEEE, 2012.
[29]
Nvidia. Isaac Sim. https:: //evee/////eee//ee///// www...iieii.ii....oe...... ccm wttta/ccccccccaacaeeeeeeaeaecaea wwsssaac-mmmm 2020.
[30] Greg Brockman, Vicki Cheung, Ladwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym. arXiv preprint arXiv: 16X6::66660014444 2016..
2016.
[31] Mare G Bellemare, Yavar Naddaf. Joel Veness, and Michael Bowling. The arcade learning environment:
An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253-279, 2013. [32] Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez. Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdolmalcki, Josh Merel, Andrew Lefrancq, et al. Deepmind control suite. arXiv preprint arXiv:1801.00 18000,0090,..1111 2018.
[33] Kiana Ehsani, Winson Han, Alvaro Herrasti, Eli VanderBilt, Luca Weihs, Eric Kolve, Aniruddha Kem- bhavi, and Roozbeh Mottaghi. ManipulaTHOR: A framework for visual object manipulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021.
[34] Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang. Yifu Yuan, He Wang. Li Yi, Angel X. Chang, Leonidas J. Guibas, and Hao Su. SAPIEN: A simulated part-based based interactive environment, In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[35] Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J Davison. Rlbench: The robot learning benchmark & learning enviromment. IEFE Robotics and Automation Letters, 5(2):301 2020. [36] Bokui Shen, Fei Xia, Chengshu Li, Roherto Martin-Martin, Linxi Fan, Guanzhi Wang. Shyamal Buch, Claudia Arpino, Sanjana Srivastava, Lyne Tchapmi, Kent Vainio. Li Fei-Fei, and Silvio Savarese. iGibson, a simulation environment for interactive tasks in large realistic scenes. arXiv preprint, 2020.
[37] Erwin Coumans and Yunfei Bai. PyBullet, Python module for physics simulation for games, robotics and machine learning. http: //ppbullplll let , org, 2016-2019
[38] Jeongscok Lee, Michael X Grey, Schoon Ha, Tobias Kunz, Sumit Jain, Yuting Ye, Siddhartha S Srinivasa, Mike Stilman, and C Karen Liu, Dart: Dynamic animation and robotics toolkit. Journal of Open Source Software, 3(22):500, 2018
[39] R Smith. ODE: Open Dynamics Engine. http: /ttp:://wwww ode. org/ 01 2009.
[40] Nvidia. PhysX. https: //t:://dvveee /wweeeeeeeeeeeeeeeeeeeeeeeeeeeee wwwiiiiiiii... cww www../////maawr...oooo -aas-phhxxxx-- overview.
[41] Nvidia. FleX. https / httev//vlll/////e//ee//e nnvi d a.
wtt.daa....a..ooeeoeaoo..ee.oaaa cww/fflx. 2020.
[42] Hammad Mazhar, Toby Heyn, Arman Pazouki, Dan Melanz, Andrew Seidl, Aaron Bartholomew. Alessan-
dro Tasora, and Dan Negrut. CHRONO: A parallel multi- library for rigid-body flexible-body. and fluid dynamics. Mechanical Sciences, 4:49-64, 64, 02 2013. doi: 4--1994/--4449-001 URL
https: haap:://pree /ttp:///o/ecccccccccco rono org/.
[43] Vladimír Vondruš and contributors. Magnum. https: : / //maguum...m graphi os, 2020.
[44]
Lilian Weng Macick Chociej. Peter Welinder. Orrb: Openai remote rendering backend. In eprint arXiv,
2019. URL https: ///arxiv...i/////// org/ abs/ 19006.96...0 11633.
[45]
Matthew Matl. Pyrender. https: http:p::////tttbb/pp/pp cttp com/ /ommt 1/ hpppp/rredrrrrr///r/ee 2020.
[46] Unity Technologies. Unity. htttts::///uttt
com/.
[47]
Epic Games, Unreal Engine. https:ttpo..o / /www.//................. www.waencganggnnnn
com/.
[48]
Manolis Savva, Angel X. Chang. Alexey Dosovitskiy, Thomas Funkhouser, and Vladlen Koltun. MINOS:
Multimodal indoor simulator for navigation in complex environments. 1712v:/712.03933 2017. [49] Yi Wu, Yuxin Wu, Georgia Gkioxari, and Yuandong Tian. Building generalizable agents with a realistic
and rich 3d enviromment. arXiv preprint arXiv: 1801,02209 2018.
[50] Claudia Yan, Dipendra Misra, Andrew Bennnett, Aaron Walsman, Yonatan Bisk, and Yoav Artzi, Chalet: 2018.
Cornell house agent learning environment. arXiv preprint
12

---

[51] Xavier Puig, Kevin Ra, Marko Boben. Jiaman Li, Tingwu Wang. Sanja Fidler. and Antonio Torralba. VortualHome:: Simulating household activities via programs. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognitiion, pages 8494-8502, 2018.
[52] Fei Xia, William B Shen, Chengshu Li, Priya Kasimbeg, Micael Edmond Tchapmi, Alexander Toshev, Roberto Martin-Martin, and Silvio Savarese. Interactive gibson benchmark: A benchmark for interactive navigation in cluttered environments, IEEE Robotics and Automation Letters, 5(2):713-75 2020.
[53] HeeSun Choi, Cindy Crump, Christian Duriez, Asher Elmquist, Gregory Hager, David Han, Frank Hearl, Jessica Hodgins, Abhinandan Jain, Frederick Leve, Chen Li, Franziska Meier, Dan Negrut, Ludovic Righetti, Alberto Rodriguez, Jie Tan, and Jeff Trinkle. On the use of simulation in robotics: Opportunities, challenges, and suggestions for moving forward. Proceedings of the National Academy of Sciences, 118(1), 2021. ISSN 0027-8424. doi: 10.1073/pnas. 19003/// 1907856118 18. URL https: /ww.. pnas org/ content // e1 907856118.
[54] Caclan Reed Garrett, Rohan Chitnis, Rachel Holladay, Beomjoon Kim, Tom Silver, Leslie Pack Kaclbling, and Tomás Lozano-Pérez. Integrated task and motion planning. arXiv preprint axXiv2010.00088. 2020. [55] Caclan Reed Garrett, Chris Paxton, Tomás Lozano-Péred Leslic Pack Kaelbling, and Dieter Fox. Online replanning in belief space for partially observable task and motion problems. In IEEE International Conference on Robotics and Automation (ICRA), 2020.
[56] Dipendra Misra, Andrew Bennelt, Valts Blukis, Eyvind Niklasson, Max Shatkhin, and Yoav Artzi. Mapping instructions to actions in 3d environments with visual goal prediction. arXiv preprint 1XXv:1809,0077 2018.
[57] Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk. Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 10740-10749, 2020.
[58] Luca Weihs, Matt Deitke, Aniruddha Kembhavi, and Roozbeh Mottaghi. Visual room rearrangement. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, 2021.
[59] Berk Calli, Arjun Singh. Aaron Walsman, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. The YCB object and model set: Towards common benchmarks for manipulation research. In 2015 international conference on advanced robotics (ICAR), pages 510-517. IEEE, 2015.
[60] Abhishek Kadian, Joanne Truong, Aaron Gokaslan, Alexander Clegg, Erik Wijmans, Stefan Lec, Manolis Savva, Sonia Chemova, and Dhruv Batra. Sim2real predictivity: Does evaluation in simulation predict real-world performance? IEEE Robotics and Automation Letters, 5(4):6670-6677, 2020.
[61] Simon Thorpe. Denis Fize, and Catherine Marlot. Speed of processing in the human visual system. nature, (6582)2555 522. 1996.
[62] Sandeep Singh Sandha. Luis Garcia, Bharathan Balaji, Fatima M Anwar, and Mani Srivastava. Sim2real transfer for deep reinforcement learning with stochastic state transition delays. CoRL 2020. 2020.
[63] Gabriel Dulac-Amold, Nir Levine, Daniel J. Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, and Todd Hester. An empirical investigation of the challenges of real-world reinforcement learning. arXiv preprint, 2020.
[64] Jie Tan. Tingnan Zhang. Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. Sim-to-real: Learning agile locomotion for quadruped robots. RSS 14, 2018.
[65] Steven M LaValle. Planning algorithms. Cambridge university press, 2006.
[66] Tanner Schmidt, Richard A Newcombe, and Dieter Fox. Dart: Dense articulated real-time tracking. In
Robotics: Science and Systems, volume 2. Berkeley, CA, 2014.
[67] Richard Sahala Hartanto, Ryoichi Ishikawa, Menandro Roxas, and Takeshi Oishi. Hand -ot--mottoouuuuo motion-guided articulation and segmentation estimation. In 2020 29th IEEE International Conference on Robot and Human Inteructive Communication (RO-MAN), pages 807-813. IEEF, 2020.
[68] Dmitry Berenson, Siddhartha Srinivasa, and James Kuffner. Task space regions: A framework for pose- constrained manipulation planning. The International Journal of Robotics Research, 30( 1334511145-144 2011.
[69] Felix Burget, Armin Hornung, and Maren Bennewitz. Whole-body motion planning for manipulation of articulated objects. In 2013 IEEE International Conference on Robotics and Automation, pages 1656-1662. IEEE, 2013.
[70] Zachary Kingston, Mark Moll, and Lydia Kavraki. Sampling-based methods for motion planning with constraints. Annual review of control, robotics, and autonomous systems, 1:159-185, 2018.
[71] Wim Meeussen, Melonee Wise, Stuart Glaser. Sachin Chitta, Conor McGann, Patrick Mihelich, Eitan Mppseenn Marius Muja, Victor Eruhimov, Tully Foote, et al. Autonomous door opening and plugging with a personal robot. In 2010 IEEE International Conference on Robotics and Automation, pages 729-736. 736. IEEE, 2010.
[72] Advait Jain and Charles C Kemp. Pulling open doors and drawers: Coordinating an omni dirrccional base and compliant and Automation, pages 1807-1814, IEEF, 2010.
with equilibrium point control. In 2010 IEEE International Conference on Robotics
[73] Steven Dalton, luri Frosio, and Michael Garland. Accelerating reinforcement learning through gpu atari emulation. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
13

---

[74]
Adam Stooke and Pieter Abbeel. ripyt: A research code base for deep reinforcement learning in pytorch. arXiv preprint urXiv:190.015000 2019.
[75] Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan. Vlad Maih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, lain Dunning, et al. Impala: Scalable distributed deep-rl with importance weighted letor-lcarner architectures. In International Conference on Machine Learning, pages 4407-1416, PMLR, 2018.
[76] Lasse Espeholt, Raphaël Marinier, Piotr Stanczyk, Ke Wang, and Marcin Michalski. Seed rl: Scalable and efficient deep-rl with accelerated central inference. arXiv preprint arXiv:/900.0000 /910,0659/, 2019.
[77] Aleksei Petrenko, Zhehui Huang, Tushar Kumar, Gaurav Sukhatme, and Vladlen Koltun. Sample factory: Egocentric 3D control from pixels at 100000 FPS with asynchronous reinforcement learning. In International Conference on Machine Learning, pages 7652-7662. PMLR. 2020.
[78] Brennan Shackleu, Erik Wijmans, Aleksei Petrenko, Manolis Savva, Dhruv Batra, Vladlen Koltun. and Kayvon Fatahalian. Large batch simulation for deep reinforcement learning. In International Conference on Learning Representations (ICLR), 2021. fKa.
net. net forum?
[79] Binomial LLC. Basis universal. https:/ attps::/giithis com www.c//e//niiimammiococm htt.///miaaal///aaaasssaaaaaaaa universal. 2020.
[80] Ioan A Sucan, Mark Moll, and Lydia E Kavraki. The open motion planning library. IEEE Robotics & Automation Magazine, 19(4):72-82) 2012.
[81] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov, Proximal policy optimization algorithms, arXiv preprint arXiv:1707.06347. 2017.
[82] Diederik Kingma and Jimmy Ba.
Adam: A method for stochastic optimization. arXiv preprint
arXiv: 1412,6980 2014.
[83] David Coleman, Ioan Sucan, Sachin Chitta, and Nikolaus Correll. Reducing the harrier to entry of complex robotic software: moveit! case study. arXiv preprint 4rXfvv1404.3785, 2014.
[84] James Kuffner and Steven M Vallle. Rrt-connect: An efficient approach to single-query path planning.
In Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), volume 2, pages 995-1001. IEEE, 2000. [85] Yoshiaki Kuwata, Gaston A Fiore, Justin Teo, Emilio Frazzoli, and Jonathan P How. Motion planning for urban driving using rrt. 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1681-1686. IEEE, 2008
[86] Nathan Ratliff, Matt Zucker, J Andrew Bagnell, and Siddhartha Srinivasa. Chomp: Gradient optimization techniques for efficient motion planning. In 2009 IEEE International Conference on Robotics and Automation, pages 489-494, IEEE, 2009.
[87] John Schulman, Yan Duan, Jonathan Ho, Alex Lee. Ibrahim Awwal, Henry Bradlow, Jia Pan, Sachin Patil, Ken Goldberg, and Pieter Abbeel. Motion planning with sequential convex optimization and convex collision checking. The International Journal of Robotics Research, 13(9))225111270) 2014.
[88] Carlos Hernandez, Mukunda Bharatheesha, Wilson Ko, Hans Gaiser, Jethro Tan, Kanter van Deurzen, Maarten de Vries, Bas Van Mil, Jeff van Egmond, Ruben Burger, al. Team delft's 's rohot winner of the amazon picking challenge 2016. In Robot World Cup. pages 613-624. Springer, 2016.
[89] Mustafa Mukadam, Jing Dong, Xinyan Yan, Frank Dellaert, and Byron Boots. Continuous-time gaussian process motion planning via probabilistic inference. The International Journal of Robotics Reveurch, 37 (11):1319-11) 1040, 2018.
[90] Brian lchter, James Harrison, and Marco Pavone. Learning sampling distributions for robot motion planning. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 7087-7094.
IEEE, 2018
[91] Brian Hou, Sanjiban Choudhury, Gilwoo Lee, Aditya Mandalika, and Siddhartha S Srinivasa. Posterior sampling for anytime motion planning graphs with expensive-to-evaluute edges. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 4266-4272. IEEE, 2020.
[92] Fahad Islam, Chris Paxton, Clemens Eppner. Bryan Peele, Maxim Likhachev, and Dieter Fox. Alternative paths planner (app) for provably fixed-time manipulation planning in remi-structures environments. arXiv preprint KLV:20012.149700 2020.
[93] Michael Pantic, Lionel Ott, Cesar Cadena, Roland Siegwart, and Juan Nieto. Mesh manifold based ricmannian motion planning for omnidirectional micro acrial vehicles. arXiv preprint arXiv:2 10313, 2021.
[94] Jonathan D Gammell, Siddhartha S Srinivasa, and Timothy D Barfoot. Informed rıt:: Optimal sampling- based path planning focused via direct sampling of admissible ellipsoidal heuristic. In 2014 IEEE/RS International Conference on Intelligent Robots and Systems, pages 2997-3004. IEEE, 2014.
[95] Jonathan D Gammell, Siddhartha S Srinivasa, and Timothy D Barfoot. Batch informed trees (bit"); Sampling based optimal planning via the heuristically guided search of implicit random geometric graphs. In 2015 IEEE international conference on robotics and automation (ICRA). pages
IEEE,
20155
[96] Daniel Kappler, Franziska Meier, Jan Issac, Jim Mainprice, Cristina Garcia Cifuentes, Manuel Wüthrich, Vincent Berenz, Stefan Schaal, Nathan Ratliff, and Jeannette Bohg. Real-time perception meets reactive motion generation. IEEE Robotics and Automation Letters, 3(3):1664-187 2018
14

---

[97] Morgan Quigley. Ken Conley, Brian Gerkey, Josh Faust. Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y Ng. Ros: open-source robot operating system. In ICRA workshop open source software, volume 3. page Kobe, Japan. 2009.
[98] Aleksandra Faust, Kenneth Oslund, Oscar Ramirez, Anthony Francis, Lydia Tapia, Marek Fiscr, and James Davidson. Prm-rl: Long-range robotic navigation tasks by combining reinforcement learning and sampling-basee planning. In 2018 IEEE International Conference Robotics and Automation (ICRA), pages 5113-5120. IEEE, 2018.
[99] Mohak Bhardwaj. Byron Boots, and Mustafa Mukadam. Differentiable gaussian process motion planning. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 10598-10604. IEEE, 2020.
[100] Dmitry Berenson, Siddhartha S Stinivasa, Dave Ferguson, and James Kuffner. Manipulation planning on constraint manifolds. 2009 JEEE international conference on robotics and automation, pages 625-632.
IEEE, 2009.
[101] Naoki Yokoyama, Schoon Ha, and Dhruv Batra. Success weighted by complction time: A dynamics-aware
evaluation criteria for embodied navigation. arXiv preprint 1xXiv:2003008/)) 2021.
[102] Ilya Kostrikov, Denis Yarats, and Rob Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. International Conference on Learning Representations (ICLR),
2021.
[103] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-basee localization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
[104] Akanksha Atrey, Kaleigh Clary, and David Jensen, Exploratory not explanatory: Counterfactual analysis of saliency maps for deep reinforcement learning. In International Conference on Learning Representa-
tions (ICLR), 2020. URL ht /openrevis ew. net./ forum d-rk1 3m RFDB.
[105] Julius Adebayo, Justin Gilmer, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Conference on Neural Information Processing Systems (NeurIPS). 2018.
15

---

Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See second paragraph of Sec. 7.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See the first paragraph of Sec. 7.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] Code and setup instructions can be found at https://github.. htt.////cccbooorrsrreece.ceeceee
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they chosen)? YYs]] All methods and training details are described in detail in Sec. H.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] Results in Sec. are over three random seeds and 600 episodes each. Results in Sec. F.1 and Sec. F.3 are over 10 random seeds and 500 episodes each. Finally, results in Sec. 6 are over 100 episodes.
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] Compute resources for the benchmark are described in Section 4, for RL training in Appendix C.2, and for motion planning in Appendix D.
you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] Cited [59] for use of the YCB objects.
(b) Did you mention the license of the assets? [Yes] ReplicaCAD is released under the Creative
Commons license and H2.0 is open-sourced under the MIT license
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes] The Repli- caCAD dataset will be made publicly available for free prior publication under the Creative Com- mons license. Download instructions can be found at hutps://gtthub.com//eee http:///ww....ooooaaooooooeer/// lab.
(d) Did you discuss whether and how consent was obtained from people whose data you're us- ing/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion or offensive content? [N/A] The released ReplicaCAD dataset includes furniture layouts and common kitchen items. There is no identifiable offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent participant compensation? [N/A]
16


Reviews:
[{"review_id": "b0h_CIZcQvO", "reviewer": "Reviewer_Xe9G", "summary": "This paper presents a new simulation environment for robotic pick-and-place tasks. The main contribution is a significant engineering effort to accelerate simulation around an existing physics engine, Bullet. The work includes some models for simulation and three tasks to test solutions. The paper also evaluates “flat” reinforcement learning, and hierarchical structure, with learned subcomponents or motion planning ones.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The presented work presents a version of the simulator and novel models. However, intellectually, I do not appreciate considerably novel contributions. Different forms of the features presented here have been presented by others before (models of scenes and objects, interactive simulation, rearrangement tasks,…). In that sense, the paper does not present original ideas or simulation concepts. \n\nQuality: The text, images, and code are well executed. I appreciate the work of a large engineering group, as it is highlighted explicitly several times (see my comment on repeating the invested time in the “limitations” section).\n\nClarity: The paper is clearly written. However, there are some concepts that are presented in a way that sounds like novel contributions of this work instead of part of the codebase this work extends (like Bullet functionalities for sleeping objects, or the common rigid body physics simulation). It would be great to 1) make clear in the text what is really novel in this work, 2) compare to state of the art in those aspects. Any element that is listed as a contribution, should be compared in the related work.\n\nSignificance: The code presented here (the new simulation environment) could be useful for the community. The manuscript does contain some additional information. \n\nIn summary, I see the engineering value of the work, but the intellectual novelty is not high. \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I see some limitations and unclear parts in the current manuscript that I list below. I hope this helps to improve the manuscript.\n\nIt is a bit unclear what is the number of apartments. “111 unique layouts of a single apartment background”. I’m interpreting this as “one apartment, 111 different ways of arranging the furniture”? In that case, it sounds limited with respect to AI2Thor, RoboThor (that enables multiple arrangements) and iGibson (that provides 15 environments and the possibility to change the objects). TDW includes 100 rooms. From a user’s perspective, why not unifying all of those? I get that AI2Thor is a bit harder because it is Unity, but iGibson is also Bullet? 92 objects do not sound very large as dataset. AI2Thor includes more than 100 objects. iGibson annotated 150 models. While valuable, I think to be a strong contribution, a dataset of objects should include many more objects, e.g., ShapeNet, PartNet… \n\nIt is a bit strange to continuously highlight the time invested in the different parts (hours of work, years of work). I don’t think this should be a factor to decide on the value of something. I would prefer to remove references to that, especially assuming that a large company is behind this effort, and making the “hours of work” a feature to highlight would clearly benefit large corporations with large teams or the budget to hire contractors, which could be an ethical concern.\n\nWhen explaining Habitat 2.0, the text explains the capabilities of any physics simulator, and that makes it hard to attribute credit. First, I think it should be necessary to cite the first time (line 35) the true source of the code, which is Couman’s Bullet because H2.0 is not advancing the field of rigid body simulation, the core equations are the same. Second, highlighting rigid body simulation capabilities in the way they are listed may be misleading: any rigid body physics simulator can do all of this, and if it supports simulating kinematic chains it can simulate any robot. Are all the listed robots modeled and included in H2 or are they just random examples? I couldn't find in the code. Please, clarify.  \n\nThe number of tasks is somewhat also not to pair with other benchmarks like metaworld, or RLBench. The way of defining the tasks is also very ad-hoc: there is a new term used “GeometricGoal”, but this is the common way of defining pick and place tasks in robotics: place the object at this location with some threshold. How scalable is the definition of tasks? Does it have to be manually defined for each new task? Can the tasks be defined more semantically instead of geometrically?\n\nThe related work section is lacking depth. Currently, there is a lot of simulation environments out there and it is increasingly difficult for users to understand their pros and cons, so I really think this section is critical. For example, I would recommend analyzing in detail the “closest neighbors”: AI2Thor (and variants), Sapien, iGibson, ThreeDWorld, RLBench. Right now, in two paragraphs, all physics simulators, renderers, and game engines are mentioned, but this feels more like a list than an in-depth comparison. The paper mentions several “points” that this work contributes but the related work focuses only on one: speed. In that sense, it feels like this is really the only difference to existing work. The subsections of the related work are also strange as they try to explain something instead of placing the current work in a scholarly context. The comparing table is a great idea but the information included right now is not very useful: it includes the names of the rendering libraries and the type of rendering (not explained or referred in the paper) but the only critical number is the speed, lacking a comparison in the other mentioned contributions (the scenes/objects, and the benchmark tasks). Also, the cited paper [33] is really similar, it would be great to go deeper in the comparison. Although it is only arxiv, I would recommend including a comparison to “The ThreeDWorld Transport Challenge” that is also extremely similar. \n\nFor related work, would be possible to include name+number instead of only the number? E.g. “Garret et al. [55] study…” instead of “[55] study…”. This facilitates readability.\n\nJoint type and limits are not dynamic properties, they are kinematics.\n\nIt is a bit unclear to me why highlighting and novel the “Localized Physics”. Bullet (as most mature physics simulators) already optimize in this front. Bullet uses the well-known concept of “islands” of objects to simulate and activate/sleep bodies. This is a default functionality. The only thing I can infer from this text is that this project may have improved a bit on that? Still, the way it is explained (“ if a robot is picking a mug from a living-room table, why must we check for collisions between the kitchen fridge shelf and objects on it?”) is to be misleading as it sounds like a novel contribution while it is only a (valuable) engineering fine-tuning of an existing concept and implementation. The second contribution listed there, “using a navigation mesh to move the robot base kinematically”: how does this affect if there are objects on the floor, for example, one of the picked objects is dropped? Can the objects be dropped? Maybe the “perfect grasping” impedes that? It would be interesting to deactivate the “perfect grasping” functionality in a small experiment (maybe the pick one) and evaluate the performance. This would tell us how realistic the results are.\n\nWhy is there an association between the “simulation step” and the “control step”? Is the control signal changed at 120Hz? Is there a controller running? If not, it is just physics simulation, right? Please, clarify. \n\nIt would be great to have a discussion on parallelization. What is parallelized? The same environment? How is that possible? Copies of the same environment? I wonder how the use of parallelizing computers and GPUs relate to other simulators, for example, to Mujoco, concretely with respect to their analysis here: http://www.mujoco.org/performance.html and in related papers.\n\nDoes the robot have access to the absolute location of the base? This is given as “realistic proprioception” but I’m not sure this is the default on any robot. GPS won’t provide this level of accuracy, even if it would work in an apartment.\n\nWhy measuring the accumulated contact force in the experiments and use it to stop? What is that physically? Is the accumulation over the entire arm at a time step (in that case, 5000N seems way off the payload) or over time? If it is over time, there is no limit per time step?\n\nI’m confused about the Sense-Plan-Act paradigm listed as “classical”. The alternative tested does task planning and uses RL-learned skills but is also doing the same three “primitives”. I went back to Murphy’s book and I do not understand the terminology in the paper. Maybe the RL option can be also explained in the same terms as the cited Murphy book? Is the main difference not just that the “act” part is executed by a learned policy? For example, in line 286, the solution explained is a “Sense-Act” (Murphy calls this “reactive paradigm”).\n\nHow are collisions implemented? Are there fully implemented? Some visualizations of the trajectories show the arm going through objects. Also, how what is the field of view of the cameras? Are they realistic?\n", "review_text": "", "extra_scores": {"ethical_concerns": "Beyond the continuously listed “hours/months of work”, which I consider not a great element to highlight from a company, I do not see other ethical concerns on this work.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "_nEaFEIYd21", "reviewer": "Reviewer_Pdk8", "summary": "The paper introduces a high performance simulation platform called Habitat 2.0 (H2.0) for training virtual robots and benchmarks RL and classical robotics policies on home assistant benchmark (HAB) that consists of three long-range tasks:  `tidying house` (moving 5 objects from random reachable locations back to where they belong), `Preparing Groceries` which is removing 2 objects from the fridge to the counters and place one back in the fridge and `setting table` which is about getting a bowl from the drawer, fruit from the fridge and placing fruit inside the bowl on the table. \n\nThe platform also involves a lot of engineering infrastructure development to increase the performance of the simulator leading to an impressive 850x improvement compared to real-time. It is worth noting that the grasping is abstracted i.e. whenever the gripper is within the 15cm range from the object, the object snaps into the gripper.  \n\nUsing this simulator platform hey benchmark RL and classical robotics pipeline which they call as sense-plan-act (SPA) and show that SPA based policies are brittle but RL based policies need hierarchical planners as pure image to action based RL policies fail to solve long-range tasks due to various challenges involving the length of the task and inability to come up with a sensible reward function. However, RL with hierarchical planners aren't perfect either and can suffer from inability to cope with the complexity of the environment. Therefore, the simulation platform provides a humbling peek into the inability of current state of the art algorithms. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The paper shows that pure image to action policies --- called Monolithic RL --- are simply not capable to solving tasks on their own. This is because of the complexity of the task that is well beyond the reach of current SOTA RL algorithms as well as the reward function. In general, for long-range tasks, coming up with the right objective remains elusive.\n\n- Comparison between TP+SRL --- which is policies involving TaskPlanning + SkillsRL where TaskPlanner (STRIPS) is used to break the task down into atomic tasks which are all learned via RL --- and TP+SPA --- (STRIPS planner) and sense plan act policy involving classical robotics pipeline to solve the tasks --- doesn't reveal a straightforward pattern. On the `TidyHouse` task, TP+ SPA performs better than TP+ SRL while the trend is reversed for the `PrepareGroceries` task. \n\nIt is humbling to see that even for seemingly simpler tasks with near-perfect planners guiding the chain of skills, it's not possible to fully solve the tasks requiring multiple interactions with the environments. This is even more surprising considering that the grasping has been abstracted in this work. The issue seems to be either compounding errors or agents reaching positions where the are unable to view the required workspace in their viewport. Providing an oracle navigation helps understand the gap as they tend to have better success rates. \n\nThe engineering effort in building up the pipeline and platform to carry out tasks at extreme high frame-rates is very commendable. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "While exploring the robotics set-up, the paper seems to be mostly focussing on the understanding of the RL algorithms and cares less about the real problems involved in doing learning for robotics e.g. grasping is abstracted with considerable and generous tolerances of 15cm and that if the gripper is within this tolerance the objects snaps into it. Most often in the real world, the issues arise due to unstable or impossible grasping and that is abstracted completely in this work. In the end even if we have perfect planner designing generalisble low-level controllers is pretty hard. This work seems to completely sidestep that part. Obviously this makes the problem even more challenging but I'd have liked to see some work in that direction too. The success rate would go down quite a bit but it would be worth plotting. Moreover, as more contacts are made, the simulator will slow down too. Table 1 shows comparison against various other simulators but they don't all necessarily have similar settings so it is not fair. \n\nAlso, when they simulate things at 30Hz (time step 1/30) there is no motion blur simulation which could also affect the policy. I wonder if there was any decision choices made for that I didn't see that in the paper. In the real world motion blur is certainly something that is going to affect the performance of the algorithm. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2 "}, "extra_text": {}}, {"review_id": "PTslu8txEv1", "reviewer": "Reviewer_dawT", "summary": "This paper presents the development of a simulator of realistic home environments with robotics support, including physics simulation, photorealism and curated assets. The most relevant novel feature is the increase in simulation speed by orders of magnitude, which enables significantly faster training times. The simulator, which is openly available, also unifies a number of advances introduced by other simulators from the embodied AI community, such as converting real scenes into CAD models for enabling interaction, now into a high performance platform with a well designed and scalable software design. Overall, this paper presents a large engineering effort and investment into developing software for a simulator and environment assets that the community can benefit from. ", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper has the following contributions:\n\n-----(1) ReplicaCAD.----- A dataset of simulation environments created by CAD modelling after real spaces from the replica dataset. These assets are the outcome of extensive design and manual curation, including textures, physical parameters, objects classes, etc, enabling realistic object randomization. The specific idea of producing CAD-based environments is not novel, a point that perhaps deserves to be highlighted more in the paper. That doesn't diminish its importance, as the coupling between the simulator H2.0 and these assets enables comprehensive use of the proposed system. The assets seem to be higher quality than in related work. Clearly the outcome of a large investment as noted in the paper. \n\n-----(2)----- H2.0 high performance photorealistic simulator with physics engine and robot support. To me this is the main contribution of the paper in terms of novel development, as a new tool available to the community and that supports significantly higher simulation speeds. Answering novel research questions might benefit from this speed up in training times. This contribution is mainly covered in Section 4. I would have liked to see even more details on this section, as being the central breakthrough. \n\n* Fig. 3 shows the main point on how authors design concurrent physics stepping and rendering. Since this causes the Observation to be delayed one cycle, I found interesting the argument of biological plausibility (line 214), perhaps it would be interesting to even have this delay being an adjustable parameter. \n\n* I didn't find many details regarding the 1GPU vs 8GPU case. Is the 8GPU case just parallel environments or is it one environment distributed among the 8 GPU? Please offer more details.\n* H2.0 takes the approach of keeping physics sim in the processor (using Bullet) and rendering on the GPU, and achieves high speed by using a number of performance optimizations. Another approach is distributed physics simulation in GPUs. For completeness, could the paper cover the pros and cons of these two approaches?\n\n* One of the optimizations is not simulating the physics of contact between the floor and the robot's wheels. I understand the paper claims this has been shown to be a good approximation, but I still worry about the potential of transfering to the real robot. Mixing full physics sim with simplified sim can create a regime where it's hard to know what segments are realistic or not, and therefore hard to debug systems when thinking on transferring to real platforms. Of course we can also think about research purely on the simulator, for which kinematics states are enough to make progress on particular research questions. Same comment applies to the grasping simplification (which is also present in other simulators). While it's true that software-wise this is trivially undone, the main question might be not if H2.0 is able to simulate (actually Bullet), but instead how realistic it is and if that segment of manipulation would behave according to reality.\n\n\n\n-----(3)----- Benchmark with household tasks. The paper proposes a small set of well defined robot tasks that can be fully simulated in H2.0. The tasks are mainly instances of the rearrangement problem, a particular way of looking at task and motion planning problems. I think these tasks are a good set of basic problems to support interesting research questions that can be investigated within the simulator. It is challenging to claim how solving the tasks in the simulator would transfer to real robot deployment. This is a broader research question that doesn't diminish the value of the proposed work, but it's worth to acknowledge. The paper also uses the simulator to study algorithms and models for mobile manipulation. From my point of view this section doesn't present a novel contribution in itself, as it treats topics long discussed in other robotics fields, but it's a solid way to verify that the proposed simulator H2.0 is in fact practically useful to investigate these questions. \n\nOther comments:\n\n* I'm not sure using the terminology from the robotics paradigm of sense-plan-act (SPA) to be distinct from the reinforcement learning (RL) policy is the best way to highlight how these two overall approaches are different. In SPA, you refer to the pipeline sensor to motion planner to controller, and in some sense a policy also has a sensor input, to forward pass of the policy, to execution (sometimes policy output is still given to a similar low level platform controller). In robotics SPA refers to the overall flow of information and decision making (from sensor, to decision, to action execution). The most frequently used SPA uses a \"motion planner\", while RL uses the \"forward pass of a neural network policy). This is just a thinking point about the terms.\n\n* The paper is very clear and the details and board set of considerations are well thought. The writing makes frequent use of parentheses, which seemed too much but actually made the writing very concrete, grounding the general comments into specific aspects that people in the field will get right away. \n\n* While the presented experiments show the usability of the simulator and assets for this type of problems, I think it would also be important to report more quantitatively what quality to expect from the physics simulation in comparison with the real world objects. The paper reports to have authored the physical parameters of the assets but just from reading the paper I'm not sure what level of realism to expect if I wanted to use H2.0 for robotics research.\n\n* One of the optimizations for speeding up the simulation is to use object sleeping for objects beyond a certain distance from the robot agent. How can the approach be adapted to multi-agent systems? For example, more than one robot in the scene.\n\n* I'm not sure if this paper should be on the main paper submission track on in the benchmark track. \n\n* The video should have been submitted with supplementary materials, not only a link on the paper.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I don't have concerns here.", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "-3zr9XCjYYc", "reviewer": "Reviewer_ibKn", "summary": "This paper presents a simulator for embodied agents for manipulation tasks in house-scale indoor environments. The paper also presents a realistic dataset of household scenarios, and home assistant benchmark (HAB), a suite of tasks designed to evaluate the performance of classical and learning based methods on rearrangement problems.", "questions": "", "limitations": "", "rating": 10, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n---------\n\n**S1** Habitat 2.0 (H2.0) represents a significant advance on multiple fronts for the embodied AI community. By curating the ReplicaCAD dataset, providing a simulation package that speeds up training by over 100x, and by introducing a large-scale benchmark of tasks that are currently out-of-reach of deep RL techniques, this paper makes comprehensive contributions to the community and paves way for a potentially large volume of exciting future work.\n\n**S2** The engineering and artistic efforts on ReplicaCAD and H2.0 are impressive. The embodied AI community has a dearth of interactive datasets; existing datasets that are rich are either small scale (AI2-THOR has room-scale scenes) or have very few scenes (iGibson has 14-15 scenes in total). Most simulators that are rich in terms of physics rely on underlying engines (e.g., Unity or Nvidia Flex) and end up being slow. H2.0 provides a highly optimized (CPU) physics engine and interleaves CPU-GPU usage for physics and graphics engines, enabling scaling conventional RL to extremely long training experiences / episodes.\n\n**S3** This paper points out interesting limitations with the combinatorial generalization of learned atomic \"skills\" (Fig. 5). While atomic skills are learned with high success rates, they do not seem to chain well, and errors from early-on skills in the execution pipeline cascade, often resulting in task failure.\n\n**S4** The work is extremely timely, tackles important yet-unsolved problems, and does so comprehensively. This paper was an unconventional, yet extremely well-written; making it an enjoyable read.\n\n\nWeaknesses\n----------\n\nI have the following concerns that I hope to see discussed in the author response phase.\n\n**W1** *Beyond success rate*: Currently, the sole metric that techniques are being compared upon is \"success rate\" (i.e., the percentage of test scenarios that end up being 'successful'). While success rate indicates a gross performance of a method, it does not capture the efficiency with which the task is carried out in general. Policies that achieve goals quickly may be preferable to ones that are sucessful, but do so at the expense of additional time. I would like to see more discussion around this in the author response. I believe this to be an important factor given the goal of the benchmark is to spur future work in rearrangement-style tasks involving manipulation.\n\n\n**W2** *SensePlanAct (SPA) baseline*: Are there any insights as to why SPA-Privileged has a lower success rate in two of the settings in Table 3, when compared to MonolithicRL; despite access to perfect sensing and perfect control? The only (speculative) reason that comes to mind is that bidirectional RRD fails to compute a feasible motion plan before the episode terminates.\n\n\n**W3** *Solvability*: Is there a mechanism to ensure that each task in the benchmark is \"solvable\"? If so, is there an \"oracle\" that would produce a solution to each task? (This might be a useful tool to gather demonstrations for techniques that leverage expert demonstrations)\n\n\nMinor remarks\n-------------\n\nThe following comments do not impact my overall evaluation of the paper. I believe these to be easily addressed in a minor revision.\n\n- MonolithicRL: Were subsets of input modalities ablated upon, to analyze the imact of each sensing modality on \"pick\" success rate? (i.e., how would a depth-only policy compare to a RGB-D policy, etc.)\n\n- Grasping - are there scenarios where unintended grasp releases cause unforseen / unsolvable scenarios? For example, a policy may arbitrarily choose to release an object before it reaches a target receptacle, and the object ends up in a location where it is inaccessible to the end effector.\n\n- SensePlanAct (SPA) baseline: The \"sensing\" phase is claimed to accumate a 3D pointcloud. How is this constructed? Using an off-the-shelf SLAM system? Does this require relative pose for each input image to be known a priori?\n\n\nJustification for score\n-----------------------\n\nGiven the perceived impact such work would have on a significant fraction on the ML community, I would strongly argue in favour of accepting this contribution to Neurips. I believe this work to have immense scope for following-up on, particularly given the promises around permissibly-licensed releases of code and data. While I enlist a few concerns with this work, I believe the strengths comfortably overshadow these concerns, and that it might be difficult to pack more into what is already a content-rich 9-page scientific document.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I liked the limitations and societal impact section -- I believe it touches upon major societal ramifications of the work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents the second version of a well-known and well used simulator, environment and benchmark for embodied computer vision, in particular navigation tasks. 

The paper received 4 expert reviews with a wide spread (minimum rating 4, maximum rating), and was initially on the fence. The main weaknesses raised where a lack of methodology, as the paper mainly describes an engineering contribution.

In the discussion phase, a consensus emerged on the usefulness of this kind of contribution for the community, as the impact of the new simulator can be estimated to be large.

The AC concurs and recommends acceptance.
Decision: Accept (Spotlight)