Multi-skill Mobile Manipulation for Object Rearrangement
Authors: Jiayuan Gu
Devendra Singh Chaplot
Hao Su
Jitendra Malik
Venue: ICLR 2023 notable top 25% (2023)
Keywords: mobile manipulation
reinforcement learning

Abstract: We study a modular approach to tackle long-horizon mobile manipulation tasks for object rearrangement, which decomposes a full task into a sequence of subtasks. To tackle the entire task, prior work chains multiple stationary manipulation skills with a point-goal navigation skill, which are learned individually on subtasks. Although more effective than monolithic end-to-end RL policies, this framework suffers from compounding errors in skill chaining, e.g., navigating to a bad location where a stationary manipulation skill can not reach its target to manipulate. To this end, we propose that the manipulation skills should include mobility to have flexibility in interacting with the target object from multiple locations and at the same time the navigation skill could have multiple end points which lead to successful manipulation. We operationalize these ideas by implementing mobile manipulation skills rather than stationary ones and training a navigation skill trained with region goal instead of point goal. We evaluate our multi-skill mobile manipulation method M3 on 3 challenging long-horizon mobile manipulation tasks in the Home Assistant Benchmark (HAB), and show superior performance as compared to the baselines.

Published as a conference paper at ICLR 2023
MULTI-SKILL MOBILE MANIPULATION FOR OBJECT REARRANGEMENT
Jiayuan Gu', Devendra Singh Chaplot", Hao Su' Jitendra Malik 2.3 UC San Diego, Meta AI Research, 'UC Berkeley
ABSTRACT
We study a modular approach to tackle long-horizon mobile manipulation tasks for object rearrangement, which decomposes full task into sequence of sub- tasks. To tackle the entire task, prior work chains multiple stationary manipula- tion skills with point-goal navigation skill, which are learned individually on subtasks. Although more effective than monolithic end-to-end RI. policies, this framework suffers from compounding errors in skill chaining, e.g., navigating to bad location where stationary manipulation skill can not reach its target to manipulate. To this end, we propose that the manipulation skills should in- clude mobility to have flexibility in interacting with the target object from mul- tiple locations and at the same time the navigation skill could have multiple end points which lead to successful manipulation. We operationalize these ideas by implementing mobile manipulation skills rather than stationary ones and training navigation skill trained with region goal instead of point goal. We evaluate multi-skill mobile manipulation method M3 on challenging long- long-horizon mobile manipulation tasks in the Home Assistant Benchmark (HAB), and show superior performance as compared to the baselines.
INTRODUCTION
Building AI with embodiment is an important future mission of AL. Object rearrangement (Batra et al. 2020) is considered as a canonical task for embodied AI. The most challenging rearrangement tasks (Szot et al. 2021 Ehsani et al.. 2021 Gan et al. (021) are often long-horizon mobile ma- nipulation tasks, which demand both navigation and manipulation abilities, e.g., to move to certain locations and pick or place objects. It is challenging to learn a monolithic RL policy for complex long-horizon mobile manipulation tasks, due to challenges such as high sample complexity, compli- cated reward design, and inefficient exploration. A practical solution tackle a long-horizon task is to decompose it into set of subtasks, which are tractable, short-horizon and compact in state or action spaces. Each subtask can be solved by designing or learning skill, so that sequence of skills can be chained to complete the entire task (Lee et al. 2018 Clegg et al., 2018 [ee et al., 2019) 2021]. For example, skills for object rearrangement can be picking or placing objects, opening or closing fridges and drawers, moving chairs, navigating in the room, etc.
Achieving successful object rearrangement using this modular framework requires careful subtask formulation such that skills trained for these subtasks can be chained together effectively. We define three desirable properties for skills to solve diverse long-horizon tasks: achievability, composabil- ity, and reusability. Note that assume each subtask is associated with set of initial states. Then, achievability quantifies the portion of initial states solvable by a skill. pair of skills are composable if the initial states achievable by the succeeding skill can encompass the terminal states of the preceding skill. This encompassment requirement is necessary to ensure robustness to mild compounding errors. However, trivially enlarging the initial set of subtask increases learning dif- ficulty and may lead to many unachievable initial states for the designed/learned skill. Last, skill reusable if it can be directly chained without or with limited fine-tuning (Clegg al., 2018 Lee et al. 2021). According to our experiments, effective subtask formulation is critical though largely overlooked in the literature.
Project wehsite: wwcsicchhttsss/ssttss..ooo......
Codes: https///gtthbbb.ommmllaaaaannag
mani caiipplee wtppptttttiioo ationn

---

Published as a conference paper at ICLR 2023
Teaul MEN
Teere chens
aqusadios
tiripaic
rexpeiatus
Intid Alo
Terrinlsbik
Teaul SPIN
Teruial cass.
Heghnngent saiipties
(a) Method Overview
(b) The Home Assistant Benehmark
Figure 1: provides an overview of our multi-skill mobile manipulation (M3) method. The inactive part the robot is colored gray. Previous approaches exclusively activate either the mobile platform or manipulator for each skill, and suffer from compounding errors in skill chaining given limited composability of skills. We introduce mobility to manipulation skills, which effectively enlarges the feasible initial set, and a region-goal navigation reward to facilitate learning the navigation skill. Tb illustrates one task (SerTable) in the Home Assistant Benchmark (Szot et al., 2021), where the robot needs to navigate in the room, open the drawers or fridge, pick multiple objects in drawers fridge and place them on the table. Best viewed in motion at the project website'.
In the context of mobile manipulation, skill chaining poses many challenges for subtask formulation. For example, imperfect navigation skill might terminate at bad location where the target object is out of reach for a stationary manipulation skill (Szot et al. 2021). To tackle such "hand-off" problems, investigate how formulate subtasks for mobile manipulation. First, we replace stationary (fixed-base) manipulation skills with mobile counterparts, which allow the base to move when the manipulation is undertaken. We observe that mobile manipulation skills are more robust to compounding errors in skill chaining, and enable the robot to make full use of its embodiment better accomplish subtasks, e.g., finding better location with less clutter and fewer obstacles to pick an object. We emphasize how to generate initial states of manipulation skills as trade-off between composability and achievability in Sec4..1 Second, we study how translate the start of manipulation skills to the navigation reward, which is used to train the navigation skill to connect manipulation skills. Note that the goal position mobile manipulation plays a very different role from that in point-goal (Wijmans al. 2019 Kadian al. 2020) navigation. On the one hand, the position of a target object (e.g., on the table or in the fridge) is often not directly navigable; on the other hand, navigable position close to the goal position can be infeasible due to kinematic and collision constraints. Besides, there exist multiple feasible starting positions for manipulation skills, yet previous works such as [Zoo et al. (2021) train the navigation skill to learn single one, which is selected heuristically and may not be suitable for stationary manipulation. Thanks to the flexibility of our mobile manipulation skills, we devise region-goal navigation reward address those issues, detailed in Sec 4.2
In this work, we present our improved multi-skill mobile manipulation method M3, where mobile manipulation skills are chained by the navigation skill trained with our region-goal navigation re- ward. It achieves an average success rate of 63% on 3 long-horizon mobile manipulation tasks in the Home Assistant Benchmark (Szot et al. 2021). as compared to 50% for our best baseline. Fig provides an overview of our method and tasks. Our contributions are listed as follows:
1. We study how to formulate mobile manipulation skills, and empirically show that they are more robust to compounding errors in skill chaining than stationary counterparts;
2. We devise region-goal navigation reward for mobile manipulation, which shows better perfor- mance and stronger generalizability than the point-goal counterpart in previous works;
3. We show that improved multi-ske skill mobile manipulation pipeline can achieve superior per- formance on long-horizon mobile manipulation tasks without bells and whistles, which can serve as strong baseline for future study.
2

---

Published as a conference paper at ICLR 2023
2
RELATED WORK
2.1 MOBILE MANIPULATION
Rearrangement BBttr et al., 2020) is "to bring a given physical environment into specified state".
We refer readers to Batra et al. (2020) for a comprehensive survey. Many existing RL tasks can be considered as instances of rearrangement, e-g., picking and placing rigid objects (Zhu et al. 2020
al. 2020) or manipulating articulated objects (Urakami et al. 2019; Mu et al., 2021). However,
they mainly focus stationary manipulation (Urakami et al. 2019 Zhu al. [020 Yu et al. 2020)
or individual, short-horizon skills (Mu et al. 2021). Recently, y. several benchmarks like Home Assis- tant Benchmark (HAB) (Szot al. 2021p. ManipulaTHHO (Ehsani et al.
2021) and ThrleDWorld
Transport Challenge (Gan et al.) (021), I), are proposed to study long-horizon mobile manipulation tasks. They usually demand that the robot rearranges household objects in room, requiring explo- ration and navigation (Anderson et al. 2018 Chaplot et al., (020) between interacting with objects entirely based on onboard sensing, without any privileged state or map information.
Mobile manipulation [RAS 2022) refers to "robotic tasks that require synergistic combination of navigation and interaction with the environment". It has been studied long in the robotics com- munity. Ni et al. (202T) provides summary of traditional methods, which usually require perfect paaaning (TAMP) (Srivastava et al.) 2014 Garrett et al. [202]] 2020). TAMP relies on well-designed state proposals (grasp poses, robot positions, etc.) to sample feasible trajectories, which is computationally inefficient and unscalable for complicated scenarios.
knowledge the environment. One example
Learning-based approaches enable the robot to act according to visual observations, Xia et al. (2021) proposes hierarchical method for mobile manipulation in iGibson (Xia et al.) 2020). which pre- dicts either a high-level base or arm action by RL policies and exccutes plans generated by motion- planning to achieve the action. However, the arm action space is specially designed for a primitive action pushing. [Sun et al. (2022) develops a real-world RL. framework to collect trash on the floor, with separate navigation and grasping policies. [Esaai et al. (2021);
al. (2021 ) train an end-to- (Ehsani et al. 202 However, the reward function used train such an end-to-end policy usually demands careful tuning. For example, Ni et al. (2021) shows that minor modification (a penalty for disturbance avoidance) can lead to a considerable performance drop. The vulnerability of end-to-end RL approaches restricts scalability. Most prior works in both RL and robotics separate mobile the platform and manipulator, to "reduce the difficulty to solve the inverse kinematics problem of kinematically redundant sys- tem" (Sereinig al. 2020: (andakaluum & Ang Jr, 2022). [Wang et al. (2020) trains an end-to-end RL policy based on the object pose and proprioception to simultaneously control the base and arm. It focuses picking a single object up in simple scenes, while our work addresses morghorrzon rearrangement tasks that require multiple skills.
); Ni
end RL policy to tackle mobile pick-and-place in ManipulaTHOn
Szot et al. (2021) adopts different hierarchical approach for mobile manipulation. It uses task- planning (Fikes & Nilsson 1971) to generate high-level symbolic goals, and individual skills are trained by RL to accomplish those goals. It outperforms the monolithic end-to-end RL policy and the classical sense-plan-at robotic pipeline. It is scalable since skills can be composited to solve different tasks, and benefit from progress in individual skill learning (Yu al.] 2020) Mu et al. 2021]. Moreover, different from other benchmarks, the HAB features continuous motor control (base and arm), interaction with articulated objects (opening drawers and fridges), and complicated scene layouts. Thus, we choose the HAB as the platform to study morizooi mobile manipulation.
2.2 SKILL CHAINING FOR LONG-HORIZON TASKS
Szot et al. 2002 observes that sequentially chaining multiple skills suffers from "hand-off" prob- lems, where a preceding skill terminates at a state that the succeeding skill has either never seen during training or is infeasible to solve. [ee et al. (018)) proposes to learn a transition policy to con- nect primitive skills, but assumes that such policy can be found through random exploration. Lee] et al. (2021 I) regularizes the terminal state distribution of a skill to be close to the initial set of the fol- lowing skill, through reward learned with adversarial training. Most prior skill chaining methods focus on fine-tuning learned skills. In this work, we instead focus on subtask formulation for skill chaining, which directly improves composability and reusability without additional computation.
3

---

Published as a conference paper at ICLR 2023
3 PRELIMINARY
3.1 HOME ASSISTANT BENCHMARK (HAB)
The Home Assistant Benchmark (HAB) Szoo al. 2021 includes 3 long-horizon mobile ma- nipulation rearrangement tasks (TidyHouse, PrepareGroceries Setlable) based on the ReplicaCAD dataset, which contains rich set of 105 indoor scene layouts. For each episode (instance of task), rigid objects from the YCB (Calli et al. 2015) dataset are randomly placed on annotated supporting surfaces of receptacles. to generate clutter in a randomly selected scene. Here we provide brief description of these tasks.
TidyHouse: House:
Move 5 objects from starting positions to goal positions. Objects and goals are located in open receptacles (e.g., table, kitchen counter) rather than containers. Complex scene layouts, diverse receptacles, dense clutter all pose challenges. The task implicitly favors collision-free be- havior since latter target object might be knocked out of reach when former object is moved by the robot.
PrepareGroceríes: Move objects from the fridge to the counters and move object from the counter to the fridge. The fridge is fully open initially. The task requires picking and placing an object in a cluttered receptacle with restricted space.
SetTable: Move a bowl from drawer to table, and move fruit from the fridge to the bowl on the table. Both the drawer and fridge are closed initially. The task requires interaction with articulated objects as well as picking objects from containers.
All the tasks demand onboard sensing instead of privileged information (e.g., ground-truth object positions and navigation map). All the tasks use the GeometricGioll (Batra al. 2020) specification (No: 8)), which describes the initial 3D (center-of-ng -fasss)) position 8g of the target object and the goal position For example, TidyHouse is specified by 5 tuples {(s) (t)), !=....
3.2 SUBTASK AND SKILL
In this section, we present the definition of subtask and skill in the context of reinforcement learning. A long-horizon task can be formulated as a Markov decision process (MDP) defined by a tuple (8, A, Rt P.I) of state space S, action space A. reward function R(s, s'),), transition distribution ((s'|s, a). initial state distribution I. subtask w a smaller MDP Aur Ra P,Iw) derived from the original MDP of the full task. A skill (or policy), which maps state S to an action a E A, is learned for each subtask by RL. algorithms.
Szot al. (2021) introduces several parameterized skills for the HAB: Pick, Place, Open fridge, Close fridge, Open drawer, Close drawer, Navigate. Each skill takes a 3D position as input, either 80 8.. See Appendix [ for more details. Here, provide brief description of these skills.
Pick(sp): pick the object initialized at so Place(s.): place the held object at 8, Open [container](s): open the container containing the object initialized at or the goal position Close [container]ss] close the container containing the object initialized at or the goal position Navigate(s): navigate to the start other skills specified by
Note that 8n is constant per episode instead of tracked object position. Hence, the target object may not be located at so at the beginming of a skill, e.g., picking an object from an opened drawer. Next, we will illustrate how these skills are chained in the HAB.
3.3 SKILL CHAINING
Given a task decomposition, a hierarchical approach also needs to generate high-level actions to select a subtask and perform the corresponding skill. Task planning (Fikes & Nilsson [97I] can be applied to find sequence of subtasks before execution, with perfect knowledge of the environment. An alternative is to learn high-level actions through hierarchical RL. In this work, we use the subtask sequences generated by perfect task planner (Szot et al. 2021). Here we list these sequences, to highlight the difficulty of tasks
"o be precise, the tasks studied this work are partially observable Markov decision process (POMDP). We only list the subtask sequence of TidyHouse for one object here for illustration. The containers are denoted with subscripts fr (fridge) and dr (drawer) if included in the skill.
4

---

Published as a conference paper at ICLR 2023
TidyHouse(sh.. s Navigate ()) Pick(s)) + Navigate(s) (+t)
+ Place st)
PrepareGroceriess.n sp. s, s s. sq. s²)): Navigate,, (sp)
Picka (sp) + Navigate
7
Place(s!) + Na (0) Pickn, (6) + Navigate (s
- Place (s2) + Navigate (0))
Pick(sØ) sg) + Navigate
place (s
SetTable s/. s4 sp. ): Navigate de
(sp)
Open (x0)
Pickar(s
Navigate
(lace(s)) Navigate dr (s)) Closear (8)) + Navigate (5 + Openg (s + Navigate, (3))
Picks (5)) Navigate
Place (22)
Navigate, (5))
Closen (t))
4 SUBTASK FORMULATION AND SKILL LEARNING FOR MOBILE MANIPULATION
Following the proposed principles (composability, achievability, reusability), we revisit and refor- mulate subtasks defined in the Home Assistant Benchmark (HAB). The core idea to enlarge the initial states of manipulation skills to encompass the terminal states of the navigation skill, given our observation that the navigation skill is usually more robust initial states. However, manipulation skills (Pick, Place. Open drawer, Close drawer) in Szot et al. (2021). are stationary. The compos- ability of a stationary manipulation skill restricted, since its feasible initial states are limited due to kinematic constraints. For instance, the robot can not open the drawer if it is too close or too far from the drawer. Therefore, these initial states need to be carefully designed given the trade-off between composability and achievability, which is not scalable and flexible. On the other hand, the navigation skill, which is learned to navigate to the start of manipulation skills, is also restricted by stationary constraints, since it is required to precisely terminate at small set of "good" locations for manipulation. To this end, we propose to replace stationary manipulation skills with mobile counterparts. Thanks to mobility, mobile manipulation skills can have better composability without sacrificing much achievability. For example, a mobile manipulator can learn to first get closer to the target and then manipulate, to compensate for errors from navigation. indicates that the initial states can be designed in a more flexible way, which also enables us to design a better navigation reward to facilitate learning.
In the context of mobile manipulation, the initial state of skill consists of the robot base position, base orientation, and joint positions. For simplicity, we do not discuss the initial states of rigid and articulated objects in the scene, which are usually defined in episode generation. Moreover, we follow previous works (Szot et al. [002 [Lee et al,, 202T) to initialize the arm its resting position and reset it after each skill in skill chaining. Such a reset operation is common in robotics (Garrett) (ct al.) (2020). Each skill is learned to reset the arm after accomplishing the subtask as in Szot et al.
(202)). Furthermore, for base orientation, we follow the heuristic in Szot et al. [202 to make the robot face the target position Sy or 8..
4.1 MANIPULATION SKILLS WITH MORILITY
We first present how initial base positions are generated in previous works. For stationary manip- ulation, a feasible base position needs to satisfy several constraints, e-g., kinematic (the target is reachable) and collision-free constraints. [Szot al. (202T) uses heuristics determine base posi- tions. For Pick, Place without containers (fridge and drawer), navigable position closest to the target position is selected. For Pick, Place with containers, a fixed position relative the container is selected. For Open, Close, a navigable position is randomly selected from a handcrafted region relative to each container. Noise added to base position and orientation in addition, and infeasible initial states are rejected by constraints. See Fig for examples.
The above example indicates the difficulty and complexity to design feasible initial states for sta- tionary manipulation. One naive solution is to enlarge the initial set with infeasible states, but this can hurt learning as shown later in See 5.4 Besides, rejection sampling can be quite inefficient in this case, and Szot et al. (2021) actually computes fixed number of feasible initial states offline.
Manipulation Skills with Mobility. To this end, we propose to use mobile manipulation skills instead. The original action space (only arm actions) is augmented with base actions. We devise a unified and efficient pipeline to generate initial base positions. Concretely, we first discretize the floor map with a resolution of 5 5cm?, and get all navigable (grid) positions. Then, different candidates are computed from these positions based on subtasks. Candidates are either within a radius (e.g., 2m) around the target position for Pick, Place, region relative to the container for
5

---

Published as a conference paper at ICLR 2023
(a) Pick(stationary)
(b) Pick(mobile)
(c) Close drawer
(d) Close fridge
Figure 2: Initial base positions of manipulation skills. We only show the examples for Pick, Close drawer, Close fridge, as Place, Open drawer, Open fridge share the same initial base positions respectively. Positions are visualized as green points on the floor. The target object in Pick is highlighted by circle in cyan. Note that the initial base position of Pick sick(statinnar) is a single navigable position closest to the object.
Open, Close. Finally, feasible position is sampled from the candidates with rejection and noise. Compared to stationary manipulation, the rejection rate of our pipeline much lower, and thus can be efficiently employed on-the-fly during training. See Fig 2 for examples.
4.2 NAVIGATION SKILL WITH REGION-GOAD NAVIGATION REWARD
The navigation skill is learned connect different manipulation skills. Hence, it needs to terminate within the set of initial achievable states manipulation skills. We follow (Szo et al.] (202T) to randomly sample a navigable base position and orientation as the initial state of navigation skill. The challenge how to formulate the reward function, which implicitly defines desirable terminal states. A common navigation reward (Wijmans al.) 2019) the negative change of geodesic distance to a single 2D goal position on the floor. (Szot et al. (2021) extends it for mobile manipulation, which introduces the negative change of angular distance to the desired orientation (facing the target). The resulting reward function, (s( a), for state s and action is the following [Eq
ax - (g) >ang AangAang
(apt" A
+ Asucee (at"" ge] Talack
A (g) = deco (z .g) - di=o (x/=1r, 9). where doe0 (zhnae, .g) is the geodesic distance between the current base position ztase and the 2D goal position dg)0(g) is short for dx .g). Aang = de domg = |10, 0* [|1 1 8" ll where fg the current base orientation, and H" is the target orientation. Note that the 2D goal on the floor is different from the 3D goal specification for manipulation subtasks. (,000=<1)
an indicator of whether the agent is close enough to the 2D goal, where D is threshold. I agge "DA," and @ thresholds for geodesic and angular distances. Tstack is a slack penalty. Xangs Asuce hyper-parameters.
an indicator of navigation success, where D
This reward has several drawbacks: A single 2D goal needs to be assigned, which should be an initial base position of manipulation skills. It is usually sampled with rejection, as explained in 4.1 ignores the existence of multiple reasonable goals, introduces ambiguity to the reward (hindering training), and leads the skill to memorize (hurting generalization). 2) There is hyper- parameter D, which defines the region where the angular term Aang is considered. However, it can lead the agent learn the undesirable behavior of entering the region with a large angular distance, e.g., backing onto the target.
Region-Goal Navigation Reward. To this end, we propose a region-goal navigation reward for training the navigation skill. Inspired by object-goal navigation, between the robot and a region of 2D goals on the floor instead of a single goal. Thanks to the flexibility of our mobile manipulation skills, we can simply reuse the candidates (Sec 44. 1 for their initial base positions as the navigation goals. However, these candidates are not all collision-free Thus, add a collision penalty Teol Acat the reward, where C₁ the current collision force and Xcol is weight. Besides, we simply remove the angular term, and find that the success reward sufficient encourage correct orientation. Our region- goal navigation reward is follows:
the geodesic distance
4.cc/4.1
(x(*, a) = A + Asuee [12ee (( ((o))(((d))""""" ge] Teal Tslack
The gendesic distance to region can be approximated by the minimum of all the geodesic distances to grid positions within the region.

---

Published as a conference paper at ICLR 2023
5 EXPERIMENTS
5.1
EXPERIMENTAL SETUP
We use the ReplicaCAD dataset and the Habitat 2.0 simulator (Szot et al. 2021) for our experiments. The ReplicaCAD dataset contains 5 macro variations, with 21 micro variations per macro variation [ unseen layouts. For the rest of the 4 macro variations, we split 84 scenes into 64 scenes for training and 20 scenes to evaluate the generalization of unseen configurations (object and goal positions). For each task, we generate 6400 episodes (64 scenes) for training, 100 episodes (20 scenes) to evaluate cross-confiuuration generalization, and another 100 episodes (the hold-out macro variation) to evaluate cross-layout generalization. The robot Fetch (Robotics) 2022) mobile manipulator with 7-DoF' arm and parallel-jaw gripper. See Appendix B for more details about the setup and dataset generation.
We hold out 1 macro variation to evaluate the generalization
Observation space: The observation space includes head and arm depth images (128 x 128), arm joint positions (7-dim), end-effector position (3-dim) in the base frame, goal positions (3-dim) in both base and end-effector frames, as well as a scalar to indicate whether an object is held. The goal position, depending subtasks, can be either the initial or desired position of the target object. We assume a perfect GPS+Compass sensor and proprioceptive sensors as in [Szot et al. (202I), which used to compute the relative goal positions. For the navigation skill, only the head depth image and the goal position in the base frame are used.
Action space: The action space is a 10-dim continuous space, including 2-dim base action (linear forwarding and angular velocities), 7-dim arm action, and 1-dim gripper action. Grasping is abstract as in Batra et al.] (2020): (Szot et al. (202T); :) |Ehsani et al. (202T). the gripper action is positive, the object closest to the end-effector within 15cm will be snapped to the gripper; if negative, the gripper will release any object held. For the navigation skill, we use a discrete action space, including a stop action, in Yokoyama et al. (202T): Szot et al. (202T). A discrete action will be converted to continuous velocities to move the robot, while arm and gripper actions are masked out.
prpper-paramttess We train each skill by the PPO (Schulman et al. 2017) algorithm. The visual observations are encoded by 3-layer CNN as in Szot et al. (2021). The visual features are concate- nated with state observations and previous action, followed by a l-layer GRU and linear layers to output action and value. Each skill is trained with 3 different seeds. See Appendix C. for details.
Metrics: Each HAB task consists of a sequence of subtasks to accomplish, as illustrated in Sec 3.3) The completion of subtask is conditioned the completion of its preceding subtask. We report progressive completion rates of subtasks, and the completion rate of the last subtask is thus the success of the full task. For each evaluation episode, the robot is initialized position and orientation without collision, and its arm is initialized at the resting position. The completion rate averaged over 9 different runs
random base
5.2 BASELINES
We denote our method by M3, short for a multi-skill mobile manipulation pipeline where mobile manipulation skills (M) are chained by the navigation skill trained with our region-goal navigation reward (R). We compare our method with several RL baselines. All baselines follow the same experimental setup in See 5.1 unless specified. We refer readers to /Sztt et al. (2021) for sense- plan-act baseline, which shown to be inferior to the skill chaining pipeline emphasized in this work. Stationary manipulation skills and point-goal navigation reward are denoted by S and P.
Monolithic RL (mono): This baseline is an end-to-end RL policy trained with a combination of reward functions of individual skills. See Appendix for more details.
Stationary manipulation skills point-goal navigation reward (S+P): This baseline is TaskPlan- SkillSKil (TP+SRL) introduced in Szot al.) (202T), where stationary manipulation skills are chained by the navigation skill trained with the point-goal navigation reward. Compared to the orig- inal implementation, we make several improvements, including better reward functions and training schemes. For reference, the original success rates of all HAB tasks nearly zero. See Appendix A for more details.
*Fach macro variation has a different, semantically plausible layout of large furniture (e-g., kitchen counter and fridge) while cach micro variation is generated through perturbing small furniture (e.g., chairs and tables). $3 seeds for RI. training multiplied by 3 seeds for initial states
7

---

Published as a conference paper at ICLR 2023
TidyHouse
PrepareGroceries
SerTable
Cross-configuratin
200
K20071
62- K20000
Cross- -ayout
2.4
0
tasks. The x-axis represents progressive subtasks. The y-axis represents the completion rate of each subtask. The mean and standard error for 100 episodes over seeds are reported. Best viewed zoomed.
Figure 3: Progressive completion rates for HAB Szot al. 2021
Mobile manipulation skills point-goal navigation reward (M+P): Compared to our M3, this baseline does not the region-goal navigation reward. It demonstrates the effectiveness of pro- posed mobile manipulation skills. Note that the point-goal navigation reward is designed for the start of stationary manipulation skills.
5.3 RESULTS
Fig shows the progressive completion rates of different methods on all tasks. Our method M3 achieves an average success rate of 1..2% in the cross-configuration setting, and 55.0% in the cross- layout setting, over all tasks. It outperforms all the baselines both settings, namely mono (1.8% S+P /5/14 1.1%% and M+P (64.9% First, all the modular approaches show much better performance than the monolithic baseline, which verifies the effectiveness of modular approaches for long-horizon mobile manipulation tasks. Mobile manipulation skills are in general superior to stationary ones (M+P vs.S+P). Fig 4 provides an example where mobile manipulation skills can compensate for imperfect navigation. Furthermore, our region-goal navigation reward can reduce the ambiguity of navigation goals to facilitate training (see training curves in Appendix C). Since it does not require the policy to memorize ambiguous goals, the induced skill shows better generalizability, especially in the cross-layout setting (55.0% for M3 v8.36.2% for M+P).
5.4 ABLATION STUDIES
We conduct several ablation studies to show that mobile manipulation skills are more flexible to formulate than stationary ones, and to understand the advantage of our navigation reward.
Can initial states be trivially enlarged? We conduct experiments to understand to what extent can enlarge the initial states of manipulation skills given the trade-off between achievability and composability. In the S(L)+P experiment, we simply replace the initial states of stationary manipulation skills with those of mobile ones. The success rates of stationary manipulation skills on subtasks drop by large margin, e.g., from 95% to 45% for Pick on TidyHouse. Fig 5 shows that S(I.+P (37.7% 1%) i inferior both S+P (57.4% (%) and M+P (64.9% 2%)). indicates that stationary manipulation skills have much smaller set of feasible initial states compared to mobile ones, and including infeasible initial states during training can hurt performance signíficantly. We also study the impact of initial state distribution mobile manipulation skills in Appendix F
Is the collision penalty important for the navigation skill? Our region-goal navigation reward benefits from unambiguous region goals and the collision penalty. We add the collision penalty the point-gor navigation reward [Eq[[) in SCP(()) and M+P(C) experiments. Fig shows that the colli- sion penalty significantly improves the success rate: S+P(C) (65.2%/44.66 4.6%) vs S+P (57.4%/3 1111%
8

---

Published as a conference paper at ICLR 2023
P
(a) Stationary Manipulation (S+P)
(b) Mobile Manipulation (M+P)
Figure 4: Qualitative comparison between stationary and mobile manipulation. In this example, the point-goal navigation skill terminates between two drawers (1st image). Mobile manipulation man- ages to open the correct drawer containing the bowl (last image in the bottom row) while stationary manipulation gets confused and finally opens the wrong drawer (last image in the top row). More qualitative results can be found in Appendix H and on our project website
Tidy House
PrepareCinceries
SetTable
Cross-cole Cllss-Configurationn
5
Cross-layout
MINCT WAICT
Figure 5: Progressive completion rates for HAB tasks. The x-axis represents progressive subtasks. The y-axis represents the completion rate of each subtask. Results of ablation experiments are presented with solid lines. The mean and standard error for 100 episodes over 9 seeds are reported.
and M+P(C) (67.9% Ms.M+P (64.9% A collision-aware navigation skill can avoid disturbing the environment. e.g., accidentally closing the fridge before placing an object in it. Be- sides, M+P(C) is still inferior to our M3 (71.2%/55.09% It implies that reducing the ambiguity of navigation goals helps learn more robust and generalizable navigation skills.
CONCLUSION AND LIMITATIONS
In this work, we present a modular approach to tackle long-horizon mobile manipulation tasks in the Home Assistant Benchmark (HAB), featuring mobile manipulation skills and the region-goal navigation reward. Given the superior performance, our approach can serve as a strong baseline for future study. Besides, the proposed principles (achievability, composability, reusability) can serve guideline about how to formulate meaningful and reusable subtasks. However, work is still limited to abstract grasp and other potential simulation defects. We leave fully dynamic simulation and real-world deployment future work.

---

Published as a conference paper at ICLR 2023
REFERENCES
Peter Anderson, Angel Chang, Devendra Singh Chaplot, Alexey Dosovitskiy, Saurabh Gupta, Vladlen Koltun, Jana Kosecka, Jitendra Malik, Roozbeh Mottaghi, Manolis Savva, al. On evaluation of embodied navigation agents. arXiv preprint arXiv: (aln::88700677 2018.
Dhruv Batra, Angel X Chang, Sonia Chernova, Andrew Davison, Jía Deng, Vladlen Koltun, Sergey Levine, Jitendra Malik, Igor Mordatch, Roozbeh Mottaghi, et al. Rearrangement: challenge for embodied ai. arXiv preprint arXiv:2011.0199 2020.
Berk Calli, Arjun Singh, Aaron Walsman, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. The ycb object and model set: Towards common benchmarks for manipulation research. In 2015 international conference on advanced robotics (ICAR), pp- 510-517. IEFE, 2015.
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdi- nov. Learning explore using active neural slam. In International Conference on Learning Representations (ICLR), 2020.
Alexander Clegg, Wenhao Yu, Jie Tan, C Karen Liu, and Greg Turk. Learning to dress: Synthesizing human dressing motion via deep reinforcement learning. ACM Transactions on Graphics (TOG), 37(6):1-10, 2018.
Kiana Ehsani, Winson Han, Alvaro Herrasti, Eli VanderBilt, Luca Weihs, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Manipulathor: framework for visual object manipulation. 2021 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 4495- 4504, 2021.
Richard E Fikes and Nils Nilsson. Strips: A new approach to the application of thcorem proving to problem solving. Artificial intelligence, 2(3-4):189-22 1971.
Chuang Gan, Siyuan Zhou, Jeremy Schwartz, Seth Alter, Abhishek Bhandwaldar, Dan Gutfreund, Daniel LK Yamins, James DiCarlo, Josh McDermott, Antonio Torralba, et al. The threedworld transport challenge: A visually guided task-and-motion planning benchmark for physically real- istic embodied ai. arXiv preprint arXiv:2103. 14025, 2021.
Caelan Reed Garrett, Tomás Jorzno-Pééez, and Leslie Pack Kaelbling. PddIstream: Integrating symbolic planners and blackbox samplers via optimistic adaptive planning. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 30, pp. 440-448, 2020.
Caelan Reed Garrett, Rohan Chitnis, Rachel Holladay, Beomjoon Kim, Tom Silver, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Integrated task and motion planning. Annual review of control, roboties, and autonomous systems, 4:265-299 293, 2021.
Abhishek Kadian, Joanne Truong, Aaron Gokaslan, Alexander Clegg, Erik Wijmans, Stefan Lee, Manolis Savva, Sonia Chernova, and Dhruv Batra. Sim2real predictivity: Does evaluation in simulation predict real-world performance? IEEE Robotics and Automation Letters, 5(4):6670- 2020.
Youngwoon Lee, Shao-Hua Sun, Sriram Somasundaram, Edward S Hu, and Joseph Lim. Com- posing complex skills by learning transition policies. In International Conference on Learning Representations, 2018
Youngwoon Lee, Jingyun Yang, and Joseph J Lim. Learning to coordinate manipulation skills via skill behavior diversification. In International Conference on Learning Representations, 2019.
Youngwoon Lee, Joseph Lim, Anima Anandkumar, and Yuke Zhu. Adversarial skill chaining for long-horizon robot manipulation via terminal state regularization. In Conference on Robot Learning. 2021.
Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Cathera Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhiwei Jia. and Hao Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations. In Thirty-ffiil Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
10

---

Published as a conference paper at ICLR 2023
Tianwei Ni, Kiana Ehsani, Luca Weihs, and Jordi Salvador. Towards disturbance- visual mobile manipulation. arXiv preprint 1721:221/2.12612, 2021.
IEEE RAS.
Mobile manipulation.
https://wwit /tt::://wwwww. ieee ras. rar.
mobile-mani mobile-manipulation 2022. Accessed: 2022-05-18.
Fetch
Robotics. Autonomous mobile robots that improve productivity.
http://
Eetchrobotics. com/ 2022. Accessed: 2022-05-18.
Thushara Sandakalum and Marcelo H Ang Jr. Motion planning for mobile manipulators a system- atic review. Machines, 10(2):97, 2022.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707003347 2017.
Martin Sereinig, Wolfgang Werth, and Lisa-Marie Faller. A review of the challenges in mobile ma- nipulation: systems design and robocup challenges. e Elektrotechnik und Informatiionstechiik, 137(6)2297-388, 2020.
Siddharth Srivastava, Eugene Fang, Lorenzo Riano, Rohan Chitnis, Stuart Russell, and Pieter Abbeel. Combined task and motion planning through an extensible planner-independen interface layer. In 2014 IEEE international conference robotics and automation (ICRA), pp. 639-646. IEEE, 2014.
Charles Sun, Jedrzej Orbik, Coline Manon Devin, Brian H Yang, Abhishek Gupta, Glen Berseth, and Sergey Levine. Fully autonomous real-world reinforcement learning with applications to mobile manipulation. In Conference on Robot Learning, pp. 308-319. PMLR, 2022.
Andrew Szot, Alex Clegg. Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Chaplot, Oleksandr Maksymets, Aaron Gokaslan, Vladimir Von- drus, Sameer Dharur, Franziska Meier, Wojciech Galuba, Angel Chang, Zsolt Kira. Vladlen Koltun, Jitendra Malik, Manolis Savva, and Dhruv Batra. Habitat 2.0: Training home assistants to rearrange their habitat. In Advances in Neural Information Processing Systems (NeurlPS), 2021.
Yusuke Urakami, Alec Hodgkinson, Casey Carlin, Randall Leu, Luca Rigazio, and Pieter Abbeel. Doorgym: A scalable door opening environment and baseline agent. arXiv preprint /9X8v:/908.01 2019.
Cong Wang, Qifeng Zhang, Qiyan Tian, Shuo Li, Xiaohui Wang, David Lane, Yvan Petillot, and Sen Wang. Learning mobile manipulation through deep reinforcement learning. Sensors, 20(3): 939, 2020.
Luca Weihs, Matt Deitke, Aniruddha Kembhavi, and Roozbeh Mottaghi. Visual room rearrange- ment. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pp. 5922-5931, 2021.
2021.
Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. Dd-ppo: Learning near- searecerfect pointgoal navigators from 2.5 billion frames. In International Conference on Learning Representations, 2019.
Fei Xia, William Shen, Chengshu Li, Priya Kasimbeg, Micael Edmond Tchapmi, Alexander Toshev, Roberto Martín-Martín, and Silvio Savarese. Interactive gibson benchmark: A benchmark for interactive navigation in cluttered environments. IEEE Robotics and Automation Letters, 5(2): 713-720, 2020.
Fei Xia, Chengshu Li, Roberto Martin-Martin, Or Litany, Alexander Toshev, and Silvio Savarese. Relmogen: Integrating motion generation in reinforcement learning for mobile manipulation. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 4583 4590. IEEE, 2021.
Naoki Yokoyama, Sehoon Ha, and Dhruv Batra. Success weighted by completion time: A dynamics- aware evaluation criteria for embodied navigation. In 2021 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 1562-1569. IEEE, 2021.

---

Published as a conference paper at ICLR 2023
Tianhe Yu. Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference Robot Learning, pp. 1094-1100. PMLR, 2020.
Yuke Zhu, Josiah Wong, Ajay Mandlekar, and Roberto Martin-Martin. robosuite: modular simu- lation framework and benchmark for robot learning. In arXiv preprint arXiv:2009. /2293, 2020.
12

---

Published as a conference paper at ICLR 2023
A OVERVIEW
Compared to the original implementation (Szot et al., 2021). our implementation benefits from re- paired assets (Sec B). improved reward functions and better training schemes (Sec C). Other differ- ences include observation and action spaces. We introduce in observations the target positions in the base frame addition those in the end-effector frame. The arm action is defined in the joint configuration space (7-dim) rather than the end-effector Euclidean space (3-dim with no orientation).
B DATASET AND EPISODES
Szot et al. (202T) keeps updating the ReplicaCAD dataset. The major fix is "minor furniture layout modifications in order to better accommodate robot access to the full set of receptacles The agent radius is also decreased from 0.4m to 0.3m to generate navigation meshes with higher connectivity. Besides, Szot et al. (2021) also improves the episode generator to ensure stable initialization of objects. Those improvements eliminate most unachievable episodes in the initial version. The episodes used in our experiments are generated with the ReplicaCAD v1.4 and the latest habitat- lab
Cross-configuration and cross-layout settings are the same except for scene layouts. In the cross- configuration setting, test scene layouts (micro variations) are different but similar to training ones. In the cross-layout setting, test scene layouts (macro variations) are significantly different from training ones. Each macro variation has different, semantically plausible layout of large furniture (e.g., kitchen counter and fridge) while each micro variation is generated through perturbing small furniture (e.g., chairs and tables). Thus, the cross-layout setting demands stronger generalization on scene layouts.
For TidyHouse, each episode includes clutter objects and 5 target objects along with their goal positions, located at 7 different receptacles (chair, tables, tv stand, two kitchen counters, sofa). For PrepareGroceries, each episode includes 21 clutter objects located at 8 different receptacles (the 7 receptacles used in TidyHouse and the top shelf of the fridge) and 1 clutter object located at the middle shelf of the fridge. target objects are located at the middle shelf, and each their goal positions is located one of two kitchen counters. The third target object is located kitchen counters, and its goal position is at the middle shelf. SerTable generates episodes similar to PropereGs,,eries, except that two target objects, bowl and apple, initialized at one of drawers and at the middle fridge shelf respectively. Each of their goal positions is located at one of two tables.
of two
C SKILL LEARNING
Each skill trained to accomplish a subtask and reset its end-effector at the resting position. The robot arm is first initialized with predefined resting joint positions, such that the corresponding resting position of the end-effector (0.5, 1.0. 0.0) in the base frame P The initial end-effector position is then perturbed by Gaussian noise (NO,, 0.025) clipped at 0.05m. The base position is perturbed by Gaussian noise (0,00 0.1) truncated at 0.2m. The base orientation is perturbed by a Gaussian noise N 0.25) truncated at 0.5 radian. The maximum episode length is 200 steps for all the manipulation skills, and 500 steps for the navigation skill. The episode terminates on success or failure. We use the same reward function for both stationary and mobile manipulation skills, unless specified.
For all skills, dú, is the distance between the end-effector and the object, d= is the distance between the end-effector and the resting position, dh is the distance between the end-effector and prede- fined manipulation handle (a 3D position) of the articulated object, d is the distance between the joint position of the articulated object and the goal joint position. A d² 1) d% (t) stands for the (negative) change in distance between a and b. For example, A the change distance
https:: :t:p::://itti hhapb//co//accooookeeeeeeaaaaaaa httpoo/ooooeere//aa....aaaaaaaad
The positive and y axes point forward and upward Habitat.
13

---

Published as a conference paper at ICLR 2023
between the end-effector and the object. Tholding indicates if the robot is holding (correct) object or handle. Ssee indicates the task success. Ce refers to the current collision force, and Cl: stands for the accumulated collision force.
The 7-dim arm action stands for the delta joint positions added to the current target joint positions of the PD controller. The input scaled by 0.025 (radian). The 2-dim base action stands for linear and angular velocities. The base movement in the Habitat 2.0 is implemented by kinematically setting the robot's base transforma- tion. The collision between the robot base and navigation meshes is taken into consideration. The input base action assumed be normalized [-1,1]. and will be scaled by (navigation skill) or 1.5 (manipulation skills). For the navigation skill, we follow SSzo et al. (2021) to use discrete action space and translate the discrete action into the continuous one. Concretely, the (normalized) linear velocity from -0.5 to is discretized into 4 choices (f (f-0.5, 0, 0.5, 1 1}), and the (normalized) angular velocity from
action assumed to be normalized to [-1, 1], and will be
is discretized into choices (((-1, -0.5, 0, 0.5, 1)). The stop action corresponds to the discrete action representing zero velocities.
Pick(so)
Objective: pick the object initialized at 80 Initial base position (noise is applied in addition):
SStationary: the closest navigable position 8o
Mobile: randomly selected navigable position within 2m of so
Ipick indicates whether the correct object is picked and Hurong indicates whether wrong object is picked.
Reward:
4A Iholding + Lpick + hooldinggg + 2.51lsuce max0000011,, 0.2) - Ilc,. su00) llurong - 11 >0.09| Looldnn 0.002
Success: The robot is holding the target object and the end-effector is within 5cm the resting position. Iyuce hooddnn A d% < 0.05
Failure:
5(Ch0 1: The accumulated collision force is larger than 5000.V.
Turong 1: A wrong object is picked.
Iddg >0.09 hoodinng
The held object slides off the gripper.
Observation space:
Depth images from head and arm cameras.
The current arm joint positions.
The current end-effector position in the base frame.
Whether the gripper holding anything.
The starting position so in both the base and end-effector frame.
Action space: The gripper is disabled to release.
Place(s.)
Objective: place the held object at s.
Initial base position (noise is applied addition):
-Stationary: the closest navigable position 8.
Mobile: randomly selected navigable position within 2m of s.
Reward: Lplece indicates whether the object is released within 15cm of the goal position, and Marrop
indicates whether the object is released beyond 15cm.
T1 4A hholinn + Iplace + 41" lthllingg + 2.5ll - Laroo 1ído 00.09 lhooding - 0.002
m0.00.,, 0.2) - -4ic
Success: The object is within 15cm of the goal position and the end-effector within 5cm of the resting position. T dg. < 0..5 Thholdnng di 0.05
14

---

Published as a conference paper at ICLR 2023
Failure:
15000 1: The accumulated collision force is larger than 7500N.
Tdrop 1: The object is released beyond 15cm of the goal position.
000000 hodding The held object slides off the gripper.
Observation space:
Depth images from head and arm cameras.
The current arm joint positions.
The current end-effector position in the base frame.
Whether the gripper is holding anything.
The goal position in both the base and end-effector frame.
Action space: The gripper is disabled to grasp after releasing the object.
Open drawer(s)
Objective: open the drawer containing the object initialized 8. The goal joint position of the drawer is 0.45m.
Initial base position (noise is applied in addition):
Stationary: navigable position randomly selected within [0.80, 0.335 X [0.95, 0.35] gion in front of the drawer.
Mobile: a navigable position randomly selected within a [0.3, 0.66 x 1.55, 0.6] region in front of the drawer.
Reward: Lopen dg < 0.05 indicates whether the drawer is open. llletease indicates whether the handle is released when the drawer is open. llgrasp indicates whether the correct handle is grasped. Obase is the (2-dim) base action.
2 Mopen ggrasp
2A holding Tratease + 24 Nepen 2.5T suce
hrrong
Iia >0.2 Tholding Taut 0.004||aaaaa
The drawer is open, and the end-effector within 15em of the resting position. Isuen Lopen A Ltholding A d' < 0.15
Success:
Failure:
Turong 1: The wrong object or handle is picked. Wholding 2
Lihh 2.2
1: The grasped handle slides off the gripper.
Lout 1: The robot moves of a predefined region (a 2m 3m region in front of the drawer).
1lppentt -lopentt (Atopee 1:
The drawer is not open after being opened.
The gripper releases the handle when the drawer is not open (Itopen
Ag >= 0.1: The drawer is opened too fast.
Observation space:
Depth images from head and arm cameras.
The current arm joint positions.
The current end-effector position in the base frame.
Whether the gripper is holding anything.
The starting position in both the base and end-effector frame.
Close drawer(s)
Objective: close the drawer containing the object initialized at 8. The goal joint position is Om..
Initial joint position: ga [0.4, 0.5), where qu is the joint position of the target drawer, random subset of other drawers are slightly open (g/. 0.1).
Initial base position (noise is applied in addition):
15

---

Published as a conference paper at ICLR 2023
Stationary: navigable position randomly selected within [0.3, -0.35) x [0.45, 0.35] region in front the drawer.
Mobile: a navigable position randomly selected within a [0.3, -0.6] x [1.0, 0.6] region in front of the drawer.
Reward: It almost the same as Open drawer by replacing open with close. Iolose = da < 0.1.
Success: The drawer is closed, and the end-effector is within 15cm of the resting position.
Failure: It is almost the same Open drawer by replacing open with close, except that the last constraint Ag >= 0.1 is not included.
Open fridge(s)
Objective: open the fridge containing the object initialized 8. The goal joint position is g - Initial base position (noise is applied in addition): navigable position randomly selected within [0.933. 1.5] [1.833, 1.5] region in front of the fridge.
Reward: Lopen g - 9a 0.15, where qa is the joint position (radian) of the fridge. To avoid the robot from penetrating the fridge due to simulation defects, we add collision penalty but excludes collision between the end-effector and the fridge.
Tt 24 Stopen + Lgrasp - +22 hoodding + Preeaae + Aoetopen + 2.51l
-Ic,,5000 Iurong [at 2022 holddngg - Ilout 0.004 0.004][abasel
Success: The fridge is open, and the end-effector is within 15cm of the resting position. Lsuce Topen A Utholding A die 0.15
Failure:
Twrong 1: The wrong object handle is picked.
I002 holding 1: The grasped handle slides off the gripper.
Lout 1: The robot moves of predefined region 2m x 3.2m region in front of the fridge).
The fridge is not open after being opened. The gripper releases the handle when the fridge is not open (Ltopen 1).
Iijorente ()topen(() 1:
Observation space:
Depth images from head and arm cameras.
The current arm joint positions.
The current end-effector position in the base frame.
Whether the gripper holding anything.
The starting position in both the base and end-effector frame.
Close fridge(s)
Objective: close the fridge containing the object initialized at s. The goal joint position is y 0.
Initial joint position: Ja [ 0.15, 2.356], where ja is the joint position of the target fridge.
Initial base position (noise applied in addition): navigable position randomly selected within a [0.933, 1.55 [1.833, 1.5] region in front of the fridge.
Reward: It is almost the same Close fridge by replacing open with close. Iatose dg < 0.15.
Success: The fridge close, and the end-effector within 15em of the resting position.
Navigate(s) (point-goal)
Objective: navigate the start of other skills specified by
Reward: refer to Eq rstack = 0.002, D = 0.9, Xang = 0.25, Asuee 2.5
Success: The robot is within 0.3 meter of the goal, 0.5 radian of the target orientation, and has called the stop action the current time step.
Observation space:
Depth images from the head camera.
The goal position 8, in the base frame.
16

---

Published as a conference paper at ICLR 2023
(a) Pick(stationary)
(b) Pick(mobile)
(c) Placo(stationary)
(d) Place(mobile)
(e) Open drawer(stationary)
(l) Open drawer(mobille
(g)
Close (h) Close drawer(mobile)
drawer(stationary)
2.4
(i Open fridge
Close fridge
(k) Navigation (point-gond goal) (I) Navigation (region-goal)
Figure 6: Training curves for skills. The y-axis represents the success rate of the subtask (including resetting the end-effector its resting position). Best viewed zoomed.
Navigate(s) (region-goal)
Objective: navigate the start of other skills specified by s
Reward: refer to Eq Tslack = 0.002, Tool = min ma.000.011 0.2), Asuce = 2.5
Success: The robot is within 0.1 meter of any goal in the region, 0.25 radian the target orienta- tion at the current position, and has called the stop action the current time step.
Observation space:
Depth images from the head camera. The goal position s. in the base frame.
C.I PPO HYPER-PARAN PARAMETRS
Our PPO implementation is based on the habitat-lab/ The visual encoder is simple CNN The coefficients of value and entropy losses are 0.5 and respectively. We use 64 parallel environments and collect 128 transitions per environment update the networks. We use mini-batches, 2 epochs per update, and a clipping parameter of 0.2 for both policy and value. The gradient norm is clipped at 0.5. We the Adam optimizer with learning rate of 0.0003. The linear learning rate decay is enabled. The mean of the Gaussian action predicted by the policy network is activated by tanh. The (log) standard deviation of the Gaussian action, which is input-independent parameter, is initialized as -.0.. Fig shows training curves of skills.
C.2 OTHER IMPLEMENTATION DETAILS
The PPO algorithm implemented by the habitat-lab does not distinguish the termination of the en- vironment (MDP) and the truncation due to time limit. We fix this issue in our implementation. Furthermore, we separately train all the skills for each HAB task to avoid potential ambiguity. For example, the starting position of an object in the drawer is computed when the drawer is closed at
hhttps://gi com http:://g....ttoooooooooooeeeee/ hea.aseoooorrrrrachaaaaaaattaeee l /blbbb// ob/ maa /habi tat_
basel ineelines/aee weweeiine/r//moaaa s/ simpl e_cnn_py
17

---

Published as a conference paper at ICLR 2023
the beginning of episode. However, the skill Pick needs pick this object up when the drawer is open and the actual position of the object is different from the starting position. It is inconsistent with other cases when the object is in open receptacle or the fridge. We observe such ambiguity can hurt performance. See Fig o for all task-specific variants of skills.
D MONOLITHIC BASELINE
For the monolithic baseline, monolithic RL policy is trained for each HAB task. During training, the policy only handles one randomly selected target object, e.g., picking and placing one object in TidyHouse. During inference, the policy is applied to each target object. We use the same ob- servation space, action space and training scheme as those for our mobile manipulation skills. The main challenge how to formulate reward function for those complicated oon-hhoiizon HAB tasks that usually require multiple stages. We follow (Szot et al. (202T) to composite reward functions for individual skills, given the sequence of subtasks. Concretely, at each time step during training, we infer the current subtask given perfect knowledge of the environment, and use the reward function of the corresponding skill. To ease training. we remove the collision penalty and do not terminate the episode due to collision. Besides, subtask. Thanks to our improved reward functions and better training scheme, our monolithic RL baseline is much better than the original implementation in Szot al. (202)). However, although able to move the object to its goal position, the policy never learns to release the object to complete the subtask Place during training. subtask in a long sequence and previous subtasks all require the robot not to release. To boost its performance, we force the gripper to release anything held at the end of execution during evaluation.
the region-goal navigation reward for the navigation
might be due to exploration difficulty since Place is the last
E EVALUATION
E.1 SEQUENTIAL SKILL CHAINING
For evaluation, skills are sequentially executed the order of their corresponding subtasks, as de- scribed in Sec 3.3 The main challenge is how to terminate skill without privileged information. Basically, each skill will be terminated if its execution time exceeds its max episode length (200 steps for manipulation skills and 500 steps for the navigation skill). The termination condition of Pick is that an object is held and the end-effector is within 15cm of the resting position, which can be computed based proprioception only. The gripper is disabled to release for Pick. The ter- mination condition of Place is that the gripper holds nothing and the end-effector is within 1 15cm of the resting position. The gripper is disabled to grasp for Place. Besides, anything held will be released when Place terminates. For Open and Close, we use heuristic from Szot et al. (202T): the skill will terminate the end-effector within 15cm of the resting position and has moved at least 30em away from the resting position during execution. Navigate terminates when it calls the stop action. Furthermore, since the manipulation skills only learn to reset its end-effector, we apply an additional operation to reset the whole arm after each skill. This reset operation is achieved by setting predefined joint positions as the target of the robot's 's PD controller.
E.2 PROGRESSIVE COMPLETION RATE
In this section, we describe how progressive completion rates are computed. The evaluation protocol is the same as Szot et al. (20TT) (see its Appendix F), and here we phrase it in a way more friendly to readers with little knowledge of task planning and Planning Domain Definition Language (PDDL). To partially evaluate a HAB task, we divide a full task into a sequence of stages (subgoals). For example, TidyHouse
be considered to consist of pick 0, place O, pick 1, Each stage can correspond to multiple subtasks. For example, the stage pick_i includes Navigate(sh) and p Thus, to be precise, the completion rate is computed based on stages instead of subtasks. We define a set of predicates to measure whether the goal of stage is completed. A stage goal is completed if all the predicates associated with are satisfied. The predicates are listed as follows:
ho 1 ding (target_ob i): The robot is holding the i-th object. at (target_obi_ng i, target_gooaaaa pos i): The i-th object is within 15cm of its goal position.
18

---

Published as a conference paper at ICLR 2023
TidyHouse
PrepareGroceries
SerTable
Tross-onfiiuration
M1314
62-
Cross- -ayout
MITHE
Figure 7: Progressive completion rates for HAB (Szot et al. 2021 tasks. The x-axis represents
progressive subtasks. The y-axis represents the completion rate of each subtask. Results of ablation experiments are presented with solid lines. The mean and standard error for 100 episodes over seeds are reported.
opened_dramd (target_marker |i); The target drawer is open (the joint position is larger than 0.4m).
(target_marker i): The target drawer is close (the joint position is smaller than 0.1m).
opened_fr (target_marker i): The target fridge is open (the joint position is larger than radian).
closed_ fridge (target_marker li): The target fridge is close (the joint position is smaller than 0.15 radian).
During evaluation, we evaluate whether the current stage goal is completed at each time step. the current stage goal is completed,
progress to the next stage. Hence, the completion rate monotonically decreases. Listings 1 2 S present the stages defined for each HAB task and the predicates associated with each stage. Note that the stage goal place i only indicates that the object has been released at its goal position, but the placement can be unstable (e.g., the object falls down the table), which can lead to the failure of the next stage. Besides, due abstract grasp, it is difficult to place the object stably since the pose of the grasped object can not be fully controlled. Therefore, we modify the objective of SerTable to make the task achievable given abstract grasp. Coneretely, instead of placing the fruit in the bowl, the robot only needs to place the fruit picked from the fridge goal position on the table.
F MORE ABLATION STUDIES
In this section, we study the impact of different initial state distributions on mobile manipulation skills. We study the impact of different initial state distributions on mobile manipulation skills. We enlarge initial states by changing the distributions of the initial base position (the radius around the target) and orientation. For reference, the maximum radius around the target is set to 2m in the main experiments (See 5. Several experiments are conducted: M(S)+R, M(L1)+R, M(L2)+R, M(L3)+R. M(S)+R, M(L1)+R and M(L2)+R stand for the experiments where the maximum radii around the target are set to 1.5m, 2.5m and 4m respectively. M(L3)+R keeps the radius as 2m, but samples the initial base orientation from [-T, æ.), instead of using the direction facing towards the target. Fig shows the quantitative results. Enlarging the initial states in general leads to per- formance degradation. Compared to M3 11.2%%/55.00 M(L1)+R (67. 41..4%//997999 7.7% and M(I3))+
19

---

Published as a conference paper at ICLR 2023
TidyHouse
PrepareGroceries
SerTable
Cross-configuraie
5-3
31+-0-7
62-
Cross- -ayouu
Figure 8: Progressive completion rates for HAB (Szot et al. 2021 tasks. The x-axis represents progressive subtasks. The y-axis represents the completion rate of each subtask. Results of ablation experiments are presented with solid lines. The mean and standard error for 100 episodes over seeds are reported.
(67.5% 4%) show moderate performance drop. M(L2)+R 55.22%33.99% shows the largest per- formance drop, which indicates that mobile manipulation skills are not able to handle long-range navigation yet. Moreover, M(S)+R (69.5% 1%) performs on par with M3. It implies that there usually exists "sweet spot" of the initial state distribution for mobile manipulation skills as trade- off between achievability and composability.
Besides, we extend the S(L)+P experiment described in S.c where we simply replace the initial states of stationary manipulation skills with those of mobile ones. We reject the initial states that the target is not reachable due to the kinematic constraint. The constraint checked via inverse kinematics (IK). The extended experiment is denoted by S(L+IK)+P. Fig S shows the quantitative results. The overall success rate of S(L+IK)+P is 44.7% in the cross-configurationnnnsss layout setting. It indicates that increasing the feasible initial states help stationary manipulation skills compared to S(L)+P (37.7% 11.11/)..))....... but still has large performance drop compared to S+P (57 55.4%/33. 1%). One possible reason is that although the target might be IK-reachable, it can be hard achieve with stationary manipulation skills due to collision with other objects. However, mobile manipulation skills can first navigate to better locations with fewer obstacles in the front.
G MORE QUANTITATIVE METRICS
In this section, we present more quantitative metrics in addition to progressive completion rates for the main experiments on 3 HAB tasks. We report the number of successfully placed objects and the average distance between objects and goals in Table | and These metrics are analogous to IXEDSTRICT and $ in Weihs et al. (02TT.
Method
TidyHouse (5)
PrepareGroceries (3)
SetTable (2)
S+P
4.58/4.56
2.33/1.54
1.45/1.03
M+P
4.45/4.54
2.45/1.79
1.71/1.08
M3 (ours)
4.64/4.60
2.56/2.44
1.71/1.22
Table 1: The number of successfully placed objects for HAB tasks. The metrics in the cross- c setting are reported. The number of objects to place is shown along with the name of each task.
20

---

Published as a conference paper at ICLR 2023
Method
TidyHouse
PrepareGrrocrrie
SerTable
S+P
0.245/0.256
0.338/0.982
3.506/6.481
M+P
1.296/0.82) 0.198/0.174
0.302/0.689
4.165/4.641
M3 (ours)
0.239//0337
5.208/6.345
Table 2: Average distance between objects and goals for HAB tasks. The metries in the cross-
configuration/crossllaa layout setting are reported. Note that the average distance is sensitive to outliers.
(a) Stationary manipulation and point-goal navigation (S+P)
(b) Mobile manipulation and point-goal navigation (M+P)
(c) Mobile manipulation and region-goal navigation (M3)
Figure 9: Qualitative comparison in TidyHouse. In this example, the point-goal navigation skill terminates behind the TV (Ist image). The arm blocked the TV in stationary manipulation (last image in the top row). The robot manages to move backward and avoid being blocked in mobile manipulation (last image the middle row). The region-ge goal navigation skill instead terminates in front of the TV (1st image in the bottom row).
H MORE QUALITATIVE RESULTS
Fig 10 show more qualitative comparison of different methods. Their animated versions can be found on our project website.
21

---

Published as a conference paper at ICLR 2023
(a) Point-goal navigation
(b) Region-goal navigation
Figure 10: Qualitative comparison in PrepareGroceries. this example, the point-goal navigation skill accidentally close the fridge (top row). The region-goal navigation skill is able avoid dis- turbing the environment due to the collision penalty (bottom row).
Stationary manipulation
Mobile manipulation
Figure 11: Qualitative comparison in SerTable. In this example, the navigation skill terminates at the position where the robot can not reach the target object in the fridge in stationary manipulation. (top row). The robot can move closer to the object and then pick it, to compensate for the navigation skill in mobile manipulation (bottom row).
22

---

Published as a conference paper at ICLR 2023
pick_
"holding (target_objl0)*
3 place_0:
4 (not_holdingg)"
5 "at (target_obi_
6 pick_l:
7 "holding (targe_ooll
(target_obi__os pos
place 1:
"not_ "not_holding()"
"at (target_obi_pos Larget__goal_pss0000
12 "at [target_obj__os| [
13 pick 2:
(targetoo(t)
"a at [targe__oetpooo|tttrgggggeollll_
"at [
place_2:
"not_ "oot__holding()*
19 at (target_obj_pos] | [
20 at (target_obj__os
"at
pick
"holding (targ
at ( 0, (argttggggtgggggapo00000
"at (target_obi_pos 1, target Laarge__gaal_paee [ooal_poss|]]
"at (target_obi_pes 2, target_gooll_posl222
place_33 3:
28 (not_holding())
" at (target_obj_pos |
at [[trgetttcttipeppp////gggggeeoep
"at [farreeet_ct_ooe]22/aa
at [tt_rgee_ttjiaaaaa33rrreeeeeee__
pich 4i
(tolgiin
3
at [aaggettcttiooooooooooggggeeeeee
36
at [aangett_eiiposs__ttarereeeeee_
" (target_obj__oss targoggggaorggg_ppppppp
"at (target_cbi_ppal targett
place_4: 4:
"not_ (not_holdingg))" |0rpagggggt_ggalppp0000
"at (target_obj_pos| |, [aaaggget_ogooooooooogggggggoggo
"at (target_abj__os||_ _ htt._pooo__tttrget__oooo_._oo
42
"at ( 2, htgpo/___oootttttggga_a_aaaa___a [2)
"at ( gagge_ooppp__ttargeee__ooooogggp (3)
"at (target_obi target [aat
Listing 1:
Stage goals and their associated predicates defined for TidyHouse. The stages listed
the order for progressive evaluation.
23

---

Published as a conference paper at ICLR 2023
1 pick_0 0:
2 "holding (target_objl0)
3 place_0:
"not "not_holdingg)"
"at (target_obi_pos tarset_golll__os000
6 pick_ _:
7 "holding (target_bji)))
8 "at ( 0, taretggggggglll_p0000
9 place_1:
"not_ ()
"at (target__bjj_oos 0, target__eal__pss00
"at (target_obi_pos| 1, "t taroet_goal__pos]])
pick 2:
(targe__o2=
15 "at (target_obj__os |,
[ttaaget__cojoossstrtttttttoooo trrggett_gaa__pppss 10)"
16
"at (target_obi_pos| 11, | tarset__oool__po||]]]]] 11
place_22
"not_ "not_holdiig(()
"at (target_obii_pgg 00, tttgtttrgggaaallll 1ot_gee__cal_pp_00000
pos
at [target_obi_pos 1 target__oaal_ps||]]
"at (target_obi_pos/ 2, tarsst__goal_pss12)
Stage goals and their associated predicates defined for PrepareGroceries. The stages are listed in the order for progressive evaluation.
Listing 2:
24

---

Published as a conference paper at ICLR 2023
open 0:
drawer (target_marker](0) 0)
3 pick_0:
4 "holding (target_objl0)*
5 place_0 0:
6 "not__olld" moot_holding()"
7 (target_obi_pos por
close 0:
"at (tareeemac
celosed_drawer
open_1:
12 @elosed_drawer (tare
at (targgettcthhppsssoo/eeeeeee___
"openea
pick_
e lowoodddrawerttateeeeeeeeee (tareet_mmmmteeerrr000
at (target_obj 10, target Saarrge__ggalllaee pos|0)"
19
(togging
20
place_1:
closse__dawew drw.er (tw.gaatteetttettttteteeeeeeeeee
at ootp_ppp__ppollaarrrtooooooooooo
nnot (oo__holding()"
"tt [target_obj_pos] 1, target__goal__sslll
close_1:
26
27
at (targer_eiioooss10/gggggggeee__e
at oaggoeee
Listing 3: Stage goals and their associated predicates defined for SerTable. The stages are listed in the order for progressive evaluation.
25


Reviews:
[{"review_id": "wdg-u9OOZc", "reviewer": "Reviewer_d9hA", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This submission proposes a modular solution for the task of multi-skill manipulation for object rearrangement. The addressed topic is an emerging one in the domain of embodied AI. The work mainly aims to tackle the issue of inaccurate terminal positions of the navigation skill. To mitigate the errors of positions (e.g., not satisfying some kinematic constraints), the new solution adopt a region-goal navigation reward, instead of point-goal reward in existing methods. The idea has been validated on ReplicaCAS and Habitat simulator. The experimental results are positive. ", "summary_of_the_review": "Embodied AI is an emerging and challenging research task. This work attempts to contribute an improved model for manipulation-based object rearrangement. I have several concerns regarding the current version that makes me hesitate to recommend acceptance.\n\nThe technical significance of the proposed method should be clarified. For example, Section 4.2 only briefly describe the proposed region-goal navigation reward, without much convincing explanation of the key choices (such as \"remove the angular term\" since \"find that the success reward is sufficient to encourage correct orientation\"). This makes the draft somewhat more like a technical report. The corresponding experiments faithful report the success rates, from which the main claims are drawn. A few qualitative examples (e.g., Figure 4) are provided for better illustration. However, in-depth analysis is still missing. I would strongly suggest to make the sections of introduce, related work and preliminary knowledge more concise, leaving more space for more quantitative studies and ablation studies.", "strength_and_weaknesses": "Strength:\n- The proposed new solution is reasonably improving the performance of a home assisting model. By using region-goal navigation award, the model has higher tolerance for the errors caused in the stage of navigation. \n- The writing is good. I have no complaints about the writing.\n\nWeakness:\n- The experiments are superficially treated. It is crucial for the readers to understand which key designs brought the performance elevation. Although two ablation studies (w.r.t. initial states or collision penalty) are provided, the proof is mainly based on comparing the success rates of different settings or methods. No further in-depth analysis is given.\n- The technical contribution is quite limited. The modification to conventional pipelines is the inclusion of region-goal (rather than point-goal) reward. Though it is proved to be effectiveness by various experiments, one can hardly claim this represents some significant contribution. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "cmB4rBo5fh", "reviewer": "Reviewer_zVvP", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is working on object rearrangement. Comparing to prior work, the proposed method replaces the static manipulation skill with a mobile manipulation skill, which remedies the situation where the navigation target position makes manipulation infeasible due to physical constraints. A region-goal navigation reward is modified based on point-goal navigation reward to train the navigation skill. Experiments on 3 mobile manipulation tasks in HAB show that the proposed modifications can lead to a higher success rate than baselines in most testing settings.", "summary_of_the_review": "This paper proposes two changes to previous work on solving the rearrangement task. The proposed changes are reasonable. The evaluations are clear. However, the changes to previous work are not significantly large. I'm mostly on the borderline with a little inclination to accept the paper.", "strength_and_weaknesses": "Rearrangement can be hard when considering partial observability and various physical constraints. The modular approach makes solving each sub-task easier, while posing challenges to the overall feasibility. The proposed changes in this paper are intuitive and are trying to address such challenges. The reward is carefully designed. The ablations in the experiments are helpful and show clear comparison for each change made. \n\nQuestions / Weakness\n- There are many hard-coded biases injected. For example, the radius selected around the target position for mobile manipulation. How sensitive is the policy to the selection of these values? From the appendix it seems the radius affects the performance a lot. \n- 4.1: ‘Finally, a feasible position is sampled from the candidates with rejection and noise.’ What is the rejection based on?\n- ‘In the S(L)+P experiment, we simply replace the initial states of stationary manipulation skills with those of mobile ones. ‘ What percentage of initial states are not within the reachable workspace of the robot arm? If doing a simple check that uses IK to determine whether it’s reachable by the robot, would there still be a large performance drop?\n- In Fig. 3, M+P is performing much worse than M3. What is the reason for that? As the mobile manipulation should be able to resolve the infeasibility caused by point-goal navigation. Is it more difficult for the mobile manipulation policy to work on a different layout?\n- Committing to a specific task plan without checking the intermediate physical constraints is the reason why static manipulation is failing. The current assumption is that the downstream subtask is always feasible and the subgoal is always achievable. If there are some other constraints, for example, if a re-grasp is needed (i.e. pick-place is not feasible but needs pick-place-pick-place), the subtask will still fail, which has the same underlying problem as static manipulation not able to solve the task because of collision or kinematic constraint. The proposed changes can improve the one on reachability. How would the current framework solve this type of problem in general?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "V26g0HNzIH", "reviewer": "Reviewer_x5CG", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "To tackle long-horizon tasks such as rearrangement, a practical solution is to decompose it into subtasks and complete them sequentially. Each subtask can be solved by designing or learning a skill. However, there exist compounding errors in skill chaining. To tackle this problem, this paper proposes manipulation skills with mobility, which allow the base to move when the manipulation is undertaken, and devise a region-goal navigation reward for mobile manipulation as navigation is the transition between two manipulations. To evaluate the proposed method, this paper conducts experiments on 3 long-horizon mobile manipulation tasks in the Home Assistant Benchmark, showing superior performance compared to the baselines.", "summary_of_the_review": "The motivations of this paper (region-goal navigation and mobile manipulation skills) are interesting and practical. But I still wonder more sufficient experiments with more metrics. \n", "strength_and_weaknesses": "Strength:\n1. The paper is logically clear in describing the method and experiment. The figures and video are vivid to show the key ideas.\n2. The motivation and idea (try to allow the base to move when the manipulation is undertaken) are flexible and interesting. Three principles to formulate subtasks (achievability, composability, reusability) are reasonable.\n3. The experiment results show that the proposed 3M model has improved performance in terms of long-horizon tasks.\n\nWeakness:\n1. The experiment settings could be explained in more details, such as cross-configuration and cross-layout in Figure 3, 5 and 7. The current version only simply introduces these concepts in Sec 5.1. It would be more clear if the authors could provide more detailed introduction.\n2. The experiment metric is only limited to the success rate. Is it possible to evaluate with more metrics like [1]?\n3. The paper simply removes the angular term in navigation reward without illustration or experimental evidence.\n[1]\tWeihs, Luca, Matt Deitke, Aniruddha Kembhavi and Roozbeh Mottaghi. “Visual Room Rearrangement.” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021): 5918-5927.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "0Qn7DUxK7YP", "reviewer": "Reviewer_w7DJ", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors took on a skill-chaining approach to solving long-horizon rearrangement problems and designed region-goal navigation rewards to reduce errors in skill-chaining. Their pipeline is novel and showed encouraging results on difficult multi-skill mobile manipulation problems.", "summary_of_the_review": "Overall, it is a strong paper, taking a novel model-free approach to solving the difficult problem of long-horizon rearrangement by mobile robots.", "strength_and_weaknesses": "The pipeline is novel and performs surprisingly well on long-horizon rearrangement problems with large observation and action space. The experiments are well-designed. The details in their methods and discussions are very informative for researchers in related fields.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%