Multi-skill Mobile Manipulation for Object Rearrangement
Abstract
We study a modular approach to tackle long-horizon mobile manipulation tasks for object rearrangement, which decomposes a full task into a sequence of subtasks.
To tackle the entire task, prior work chains multiple stationary manipulation skills with a point-goal navigation skill, which are learned individually on subtasks.
Although more effective than monolithic end-to-end RL policies, this framework suffers from compounding errors in skill chaining, e .
g ., navigating to a bad location where a stationary manipulation skill can not reach its target to manipulate.
To this end, we propose that the manipulation skills should include mobility to have ﬂexibility in interacting with the target object from multiple locations and at the same time the navigation skill could have multiple end points which lead to successful manipulation.
We operationalize these ideas by implementing mobile manipulation skills rather than stationary ones and training a navigation skill trained with region goal instead of point goal.
We evaluate our multi-skill mobile manipulation method M3 on 3 challenging long-horizon mobile manipulation tasks in the Home Assistant Benchmark (HAB), and show superior performance as compared to the baselines.
Introduction
Building AI with embodiment is an important future mission of AI.
Object rearrangement [2] is considered as a canonical task for embodied AI.
The most challenging rearrangement tasks [1, 3, 4] are often long-horizon mobile manipulation tasks, which demand both navigation and manipulation abilities, e .
g ., to move to certain locations and to pick or place objects.
It is challenging to learn a monolithic RL policy for complex long-horizon mobile manipulation tasks, due to challenges such as high sample complexity, complicated reward design and inefﬁcient exploration.
A practical solution to tackle a long-horizon task is to decompose it into a set of subtasks, which are tractable, short-horizon and compact in state or action spaces.
Each subtask can be solved by designing or learning a skill, so that a sequence of skills can be chained to complete the entire task [5–8].
For example, skills for object rearrangement can be picking or placing objects, opening or closing fridges and drawers, moving chairs, navigating in the room, etc .
Achieving successful object rearrangement using this modular framework requires careful subtask formulation such that skills trained for these subtasks can be chained together effectively.
We deﬁne three desirable properties for skills to solve diverse long-horizon tasks: achievability, composability, and reusability .
Note that we assume each subtask is associated with a set of initial states.
Then, achievability quantiﬁes the portion of initial states solvable by a skill.
A pair of skills are composable if the initial states achievable by the succeeding skill can encompass the terminal states of the preceding skill.
This encompassment requirement is necessary to ensure robustness to mild compounding errors.
However, trivially enlarging the initial set of a subtask increases learning difﬁculty and may lead to [Figure] many unachievable initial states for the designed/learned skill.
Last, a skill is reusable if it can be directly chained without or with limited ﬁne-tuning [6, 8].
According to our experiments, effective subtask formulation is critical though largely overlooked in the literature.
In the context of mobile manipulation, skill chaining poses many challenges for subtask formulation.
For example, an imperfect navigation skill might terminate at a bad location where the target object is out of reach for a stationary manipulation skill [1].
To tackle such “hand-off” problems, we investigate how to formulate subtasks for mobile manipulation.
First, we replace stationary (ﬁxed-base) manipulation skills with mobile counterparts, which allow the base to move when the manipulation is undertaken.
We observe that mobile manipulation skills are more robust to compounding errors in skill chaining, and enable the robot to make full use of its embodiment to better accomplish subtasks, e .
g ., ﬁnding a better location with less clutter and fewer obstacles to pick an object.
We emphasize how to generate initial states of manipulation skills as a trade-off between composability and achievability in Sec 4.1.
Second, we study how to translate the start of manipulation skills to the navigation reward, which is used to train the navigation skill to connect manipulation skills.
Note that the goal position in mobile manipulation plays a very different role from that in point-goal [9, 10] navigation.
On the one hand, the position of a target object ( e . g ., on the table or in the fridge) is often not directly navigable; on the other hand, a navigable position close to the goal position can be infeasible due to kinematic and collision constraints.
Besides, there exist multiple feasible starting positions for manipulation skills, yet previous works such as [1] train the navigation skill to learn a single one, which is selected heuristically and may not be suitable for stationary manipulation.
Thanks to the ﬂexibility of our mobile manipulation skills, we devise a region-goal navigation reward to handle those issues, detailed in Sec 4.2.
In this work, we present our improved multi-skill mobile manipulation method M3 , where mobile manipulation skills are chained by the navigation skill trained with our region-goal navigation reward.
It achieves an average success rate of 63% on 3 long-horizon mobile manipulation tasks in the Home
Assistant Benchmark [1], as compared to 50% for our best baseline.
Fig 1 provides an overview of our method and tasks.
Our contributions are listed as follows:
We study how to formulate mobile manipulation skills, and empirically show that they are more robust to compounding errors in skill chaining than stationary counterparts; 2.
We devise a region-goal navigation reward for mobile manipulation, which shows better performance and stronger generalizability than the point-goal counterpart in previous works; 3.
We show that our improved multi-skill mobile manipulation pipeline can achieve superior performance on long-horizon mobile manipulation tasks without bells and whistles, which can serve as a strong baseline for future study.
Related Work 2.1 Mobile Manipulation
Rearrangement [2] is “to bring a given physical environment into a speciﬁed state”.
We refer readers to [2] for a comprehensive survey.
Many existing RL tasks can be considered as instances of rearrangement, e .
g .,
picking and placing rigid objects [11, 12] or manipulating articulated objects [13, 14].
However, they mainly focus on stationary manipulation [11–13] or individual, shorthorizon skills [14].
Recently, several benchmarks like Home Assistant Benchmark (HAB) [1],
ManipulaTHOR [3] and ThreeDWorld Transport Challenge [4], are proposed to study long-horizon mobile manipulation tasks.
They usually demand that the robot rearranges household objects in a room, requiring exploration and navigation [15, 16] between interacting with objects entirely based on onboard sensing, without any privileged state or map information.
Mobile manipulation [17] refers to “robotic tasks that require a synergistic combination of navigation and interaction with the environment”.
It has been studied long in the robotics community.
[18] provides a summary of traditional methods, which usually require perfect knowledge of the environment.
One example is task-and-motion-planning (TAMP)
TAMP relies on well-designed state proposals (grasp poses, robot positions, etc .)
to sample feasible trajectories, which is computationally inefﬁcient and unscalable for complicated scenarios.
Learning-based approaches enable the robot to act according to visual observations.
[22] proposes a hierarchical method for mobile manipulation in iGibson [23], which predicts either a high-level base or arm action by RL policies and executes plans generated by motion-planning to achieve the action.
However, the arm action space is specially designed for a primitive action pushing .
[24] develops a real-world RL framework to collect trash on the ﬂoor, with separate navigation and grasping policies.
[3, 18] train an end-to-end RL policy to tackle mobile pick-and-place in ManipulaTHOR [3].
However, the reward function used to train such an end-to-end policy usually demands careful tuning.
For example, [18] shows that a minor modiﬁcation (a penalty for disturbance avoidance) can lead to a considerable performance drop.
The vulnerability of end-to-end RL approaches restricts scalability.
Most prior works in both RL and robotics separate mobile the platform and manipulator, to “reduce the difﬁculty to solve the inverse kinematics problem of a kinematically redundant system” [25, 26]. [27] trains an end-to-end RL policy based on the object pose and proprioception to simultaneously control the base and arm.
It focuses on picking a single object up in simple scenes, while our work addresses long-horizon rearrangement tasks that require multiple skills. [1] adopts a different hierarchical approach for mobile manipulation in the HAB [1].
It uses taskplanning [28] to generate high-level symbolic goals, and individual skills are trained by RL to accomplish those goals.
It outperforms the monolithic end-to-end RL policy and the classical sense-plan-act robotic pipeline.
It is scalable since skills can be composited to solve different tasks, and beneﬁt from progress in individual skill learning [12, 14].
Moreover, different from other benchmarks, the HAB features continuous motor control (base and arm), interaction with articulated objects (opening drawers and fridges), and complicated scene layouts.
Thus, we choose the HAB as the platform to study long-horizon mobile manipulation.
2.2 Skill Chaining for Long-horizon Tasks
[1] observes that sequentially chaining multiple skills suffers from “hand-off” problems, where a preceding skill terminates at a state that the succeeding skill has either never seen during training or is infeasible to solve.
[5] proposes to learn a transition policy to connect primitive skills, but assumes that such a policy can be found through random exploration.
[8] regularizes the terminal state distribution of a skill to be close to the initial set of the following skill, through a reward learned with adversarial training.
Most prior skill chaining methods focus on ﬁne-tuning learned skills.
In this work, we instead focus on subtask formulation for skill chaining, which directly improves composability and reusability without additional computation.
Preliminary 3.1 Home Assistant Benchmark (HAB)
The Home Assistant Benchmark (HAB) [1] includes 3 long-horizon mobile manipulation rearrangement tasks ( TidyHouse, PrepareGroceries, SetTable ) based on the ReplicaCAD dataset, which contains a rich set of 105 indoor scene layouts.
For each episode (instance of task), rigid objects from the YCB [29] dataset are randomly placed on annotated supporting surfaces of receptacles, to generate clutter in a randomly selected scene.
Here we provide a brief description of these tasks.
TidyHouse :
Move 5 objects from starting positions to goal positions.
Objects and goals are located in open receptacles ( e .
g ., table, kitchen counter) rather than containers.
Complex scene layouts, diverse receptacles, dense clutter all pose challenges.
The task implicitly favors collision-free behavior since a latter target object might be knocked out of reach when a former object is moved by the robot.
PrepareGroceries : Move 2 objects from the fridge to the counters and move an object from the counter to the fridge.
The fridge is fully open initially.
The task requires picking and placing an object in a cluttered receptacle with restricted space.
SetTable : Move a bowl from a drawer to a table, and move a fruit from the fridge to the bowl on the table.
Both the drawer and fridge are closed initially.
The task requires interaction with articulated objects as well as picking objects from containers.
All the tasks use the GeometricGoal [2] speciﬁcation ( s 0 , s ∗ ) , which describes the initial 3D (centerof-mass) position s 0 of the target object and the goal position s ∗ .
For example, TidyHouse is speciﬁed by 5 tuples { ( s i 0 , s i ∗ ) } i =1 ... 5 .
3.2 Subtask and Skill
In this section, we present the deﬁnition of subtask and skill in the context of reinforcement learning.
A long-horizon task can be formulated as a Markov decision process (MDP) 1 deﬁned by a tuple ( S , A , R, P, I ) of state space S , action space A , reward function R ( s, a, s (cid:48) ) , transition distribution
P ( s (cid:48) | s, a ) , initial state distribution I .
A subtask ω is a smaller MDP ( S , A ω , R ω , P, I ω ) derived from the original MDP of the full task.
A skill (or policy), which maps a state s ∈ S to an action a ∈ A , is learned for each subtask by RL algorithms. [1] introduces several parameterized skills for the HAB: Pick, Place, Open fridge, Close fridge, Open drawer, Close drawer, Navigate .
Each skill takes a single 3D position as input, either s 0 or s ∗ .
See
Appendix C for more details.
Here, we provide a brief description of these skills.
Pick( s 0 ) : pick the object initialized at s[-
-]
Place( s ∗ ) : place the held object at s ∗
Open [container]( s ) : open the container containing the object initialized at s or the goal position s
Close [container]( s ) : close the container containing the object initialized at s or the goal position s
Navigate( s ) : navigate to the start of other skills speciﬁed by s
Note that s 0 is constant per episode instead of a tracked object position.
Hence, the target object may not be located at s 0 at the beginning of a skill, e .
g ., picking an object from an opened drawer.
Next, we will illustrate how these skills are chained in the HAB.
3.3 Skill Chaining
Given a task decomposition, a hierarchical approach also needs to generate high-level actions to select a subtask and perform the corresponding skill.
Task planning [28] can be applied to ﬁnd a sequence of subtasks before execution, with perfect knowledge of the environment.
An alternative is to learn high-level actions through hierarchical RL.
In this work, we use the subtask sequences generated by a perfect task planner [1].
Here we list these sequences, to highlight the difﬁculty of tasks 2 .
TidyHouse ( s i 0 , s i ∗ ) :
Navigate ( s i 0 )
→ Pick ( s i 0 )
→ Navigate ( s i ∗ ) →
Place ( s i ∗ )
PrepareGroceries ( s 10 , s 1 ∗ , s 20 , s 2 ∗ , s 30 , s 3 ∗ ) :
Navigate fr ( s 10 ) → Pick fr ( s 10 ) → Navigate ( s 1 ∗ ) →
[Table]
Subtask Formulation and Skill Learning for Mobile Manipulation
Following the proposed principles ( composability , achievability , reusability ), we revisit and reformulate subtasks deﬁned in the Home Assistant Benchmark (HAB) [1].
The core idea is to enlarge the initial states of manipulation skills to encompass the terminal states of the navigation skill, given our observation that the navigation skill is usually more robust to initial states.
However, manipulation skills ( Pick , Place , Open drawer , Close drawer ) in [1], are stationary.
The composability of a stationary manipulation skill is restricted, since its feasible initial states are limited due to kinematic constraints.
For instance, the robot can not open the drawer if it is too close or too far from the drawer.
Therefore, these initial states need to be carefully designed given the trade-off between composability and achievability , which is not scalable and ﬂexible.
On the other hand, the navigation skill, which is learned to navigate to the start of manipulation skills, is also restricted by stationary constraints, since it is required to precisely terminate at a small set of “good” locations for manipulation.
To this end, we propose to replace stationary manipulation skills with mobile counterparts.
Thanks to mobility, mobile manipulation skills can have better composability without sacriﬁcing much achievability .
For example, a mobile manipulator can learn to ﬁrst get closer to the target and then manipulate, to compensate for errors from navigation.
It indicates that the initial states can be designed in a more ﬂexible way, which also enables us to design a better navigation reward to facilitate learning.
In the context of mobile manipulation, the initial state of a skill consists of the robot base position, base orientation, and joint positions.
For simplicity, we do not discuss the initial states of rigid and articulated objects in the scene, which are usually deﬁned in episode generation.
Moreover, we follow previous works [1, 8] to initialize the arm at its resting position and reset it after each skill in skill chaining.
Such a reset operation is common in robotics [21].
Each skill is learned to reset the arm after accomplishing the subtask as in [1].
Furthermore, for base orientation, we follow the heuristic in [1] to make the robot face the target position s 0 or s ∗ .
4.1 Manipulation Skills with Mobility
We ﬁrst present how initial base positions are generated in previous works.
For stationary manipulation, a feasible base position needs to satisfy several constraints, e .
g ., kinematic (the target is reachable) and collision-free constraints.
[1] uses heuristics to determine base positions.
For Pick,
Place without containers (fridge and drawer), a navigable position closest to the target position is selected.
For Pick, Place with containers, a ﬁxed position relative to the container is selected.
For
Open, Close , a navigable position is randomly selected from a handcrafted region relative to each container.
Noise is added to base position and orientation in addition, and infeasible initial states are rejected by constraints.
See Fig 2 for examples.
The above example indicates the difﬁculty and complexity to design feasible initial states for stationary manipulation.
One naive solution is to enlarge the initial set with infeasible states, but this can hurt learning as shown later in Sec 5.4.
Besides, rejection sampling can be quite inefﬁcient in this case, and [1] actually computes a ﬁxed number of feasible initial states ofﬂine.
Manipulation Skills with Mobility.
To this end, we propose to use mobile manipulation skills instead.
The original action space (only arm actions) is augmented with base actions.
We devise a uniﬁed and efﬁcient pipeline to generate initial base positions.
Concretely, we ﬁrst discretize the ﬂoor map with a resolution of 5 × 5 cm 2 , and get all navigable (grid) positions.
Then, different candidates are computed from these positions based on subtasks.
Candidates are either within a radius ( e .
g ., 2m) around the target position for Pick, Place , or a region relative to the container for Open, Close .
Finally, a feasible position is sampled from the candidates with rejection and noise.
Compared to stationary manipulation, the rejection rate of our pipeline is much lower, and thus can be efﬁciently employed on-the-ﬂy during training.
See Fig 2 for examples.
[Figure]
4.2 Navigation Skill with Region-Goal Navigation Reward
The navigation skill is learned to connect different manipulation skills.
Hence, it needs to terminate within the set of initial achievable states of manipulation skills.
We follow [1] to randomly sample a navigable base position and orientation as the initial state of navigation skill.
The challenge is how to formulate the reward function, which implicitly deﬁnes desirable terminal states.
A common navigation reward [9] is the negative change of geodesic distance to a single 2D goal position on the ﬂoor.
[1] extends it for mobile manipulation, which introduces the negative change of angular distance to the desired orientation (facing the target).
The resulting reward function, r t ( s, a ) , for state s and action a is the following (Eq 1):
[Equation]
(1) ∆ geo ( g ) =
d geot ( x baset , g ) − d geot − 1 ( x baset − 1 , g ) , where d geot ( x baset , g ) is the geodesic distance between the current base position x baset and the 2D goal position g .
d geot ( g ) is short for d geot ( x baset , g ) .
∆ ang = d angt − d angt − 1 = (cid:107) θ t − θ ∗ (cid:107) 1 − (cid:107) θ t − 1 − θ ∗ (cid:107) 1 , where θ t is the current base orientation, and θ ∗ is the target orientation.
Note that the 2D goal on the ﬂoor is different from the 3D goal speciﬁcation for manipulation subtasks.
I [ d geot ≤ ˜ D ] is an indicator of whether the agent is close enough to the 2D goal, where ˜ D is a threshold.
I [ d geot ≤ D ∧ d angt ≤ Θ] is an indicator of navigation success, where D and Θ are thresholds for geodesic and angular distances.
r slack is a slack penalty.
λ ang , λ succ are hyper-parameters.
This reward has several drawbacks: 1) A single 2D goal needs to be assigned, which should be an initial base position of manipulation skills.
It is usually sampled with rejection, as explained in Sec 4.1.
It ignores the existence of multiple reasonable goals, introduces ambiguity to the reward (hindering training), and leads the skill to memorize (hurting generalization).
There is a hyperparameter ˜ D , which deﬁnes the region where the angular term ∆ ang is considered.
However, it can lead the agent to learn the undesirable behavior of entering the region with a large angular distance, e . g ., backing onto the target.
Region-Goal Navigation Reward.
To this end, we propose a region-goal navigation reward for training the navigation skill.
Inspired by object-goal navigation [30], we use the geodesic distancebetween the robot and a region of 2D goals on the ﬂoor instead of a single goal.
Thanks to the ﬂexibility of our mobile manipulation skills, we can simply reuse the candidates (Sec 4.1) for their initial base positions as the navigation goals.
However, these candidates are not all collision-free.
Thus, we add a collision penalty r col = λ col C t to the reward, where C t is the current collision force and λ col is a weight.
Besides, we simply remove the angular term, and ﬁnd that the success reward is sufﬁcient to encourage correct orientation.
Our region-goal navigation reward is the following (Eq 2):
[Equation]
Experiments 5.1 Experimental Setup
We use the ReplicaCAD dataset [1] and the Habitat 2.0 simulator [1] for our experiments.
The
ReplicaCAD dataset contains 5 macro variations, with 21 micro variations per macro variation 4 .
We hold out 1 macro variation to evaluate the generalization of unseen layouts.
For the rest of the 4 macro variations, we split 84 scenes into 64 scenes for training and 20 scenes to evaluate the generalization of unseen conﬁgurations (object and goal positions).
For each task, we generate 6400 episodes ( scenes) for training, 100 episodes (20 scenes) to evaluate cross-conﬁguration generalization, and another 100 episodes (the hold-out macro variation) to evaluate cross-layout generalization.
The robot is a Fetch [31] mobile manipulator with a 7-DoF arm and a parallel-jaw gripper.
Observation space: The observation space includes head and arm depth images ( 128 × 128 ), arm joint positions (7-dim), end-effector position (3-dim) in the base frame, goal positions (3-dim) in both base and end-effector frames, as well as a scalar to indicate whether an object is held.
The goal position, depending on subtasks, can be either the initial or desired position of the target object.
We assume a perfect GPS+Compass sensor and proprioceptive sensors as in [1], which are used to compute the relative goal positions.
For the navigation skill, only the head depth image and the goal position in the base frame are used.
Action space: The action space is a 10-dim continuous space, including 2-dim base action (linear forwarding and angular velocities), 7-dim arm action, and 1-dim gripper action.
Grasping is abstract as in [1–3].
If the gripper action is positive, the object closest to the end-effector within 15cm will be snapped to the gripper; if negative, the gripper will release any object held.
For the navigation skill, we use a discrete action space, including a stop action, as in [1, 32].
A discrete action will be converted to continuous velocities to move the robot, while arm and gripper actions are masked out.
Hyper-parameters: We train each skill by the PPO [33] algorithm.
The visual observations are encoded by a 3-layer CNN as in [1].
The visual features are concatenated with state observations and previous action, followed by a 1-layer GRU and linear layers to output action and value.
We use 64 parallel environments and train each skill for 100M steps with 16 CPU cores and 1 2080Ti GPU.
Each skill is trained with 3 different seeds.
See Appendix C.1 for more details.
Metrics: Each HAB task consists of a sequence of subtasks to accomplish, as illustrated in Sec 3.3.
The completion of a subtask is conditioned on the completion of its preceding subtask.
We report progressive completion rates of subtasks, and the completion rate of the last subtask is thus the success rate of the full task.
For each evaluation episode, the robot is initialized at a random base position and orientation without collision, and its arm is initialized at the resting position.
The completion rate is averaged over 9 different runs (3 seeds for RL training multiplied by 3 seeds for initial states).
5.2 Baselines
We denote our method by M3 , short for a multi-skill mobile manipulation pipeline where mobile manipulation skills ( M ) are chained by the navigation skill trained with our region-goal navigation reward ( R ).
We compare our method with several RL baselines.
All baselines follow the same experimental setup in Sec 5.1 unless speciﬁed. We refer readers to [1] for a task-and-motion-planning (TAMP) baseline, which is shown to be inferior to the skill chaining pipeline emphasized in this work. Stationary manipulation skills and point-goal navigation reward are denoted by S and P . Monolithic RL (mono) :
This baseline is an end-to-end RL policy trained with a combination of reward functions of individual skills. See Appendix D for more details. Stationary manipulation skills + point-goal navigation reward (S+P) : This baseline is TaskPlanning+SkillsRL (TP+SRL) introduced in [1], where stationary manipulation skills are chained by the navigation skill trained with the point-goal navigation reward.
Compared to the original implementation, we make several improvements, including better reward functions and training schemes. Mobile manipulation skills + point-goal navigation reward (M+P) : Compared to our M3 , this [Figure]
[Table]
ﬁ gu r a ti on
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
SetTable
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] baseline does not use the region-goal navigation reward. It demonstrates the effectiveness of proposed mobile manipulation skills. Note that the point-goal navigation reward is designed for the start of stationary manipulation skills.
Fig 3 shows the progressive completion rates of different methods on all tasks. Our method M achieves an average success rate of 71.2% in the cross-conﬁguration setting, and 55.0% in the cross-layout setting, over all 3 tasks. It outperforms all the baselines in both settings, namely mono (1.8%/1.8%), S+P (57.4%/31.1%) and M+P (64.9%/36.2%).
First, all the modular approaches show much better performance than the monolithic baseline, which veriﬁes the effectiveness of modular approaches for long-horizon mobile manipulation tasks.
Mobile manipulation skills are in general superior to stationary ones ( M+P vs . S+P ).
Fig 4 provides an example where mobile manipulation skills can compensate for imperfect navigation. Furthermore, our region-goal navigation reward can reduce the ambiguity of navigation goals to facilitate training (see training curves in Appendix C). Since it does not require the policy to memorize ambiguous goals, the induced skill shows better generalizability, especially in the cross-layout setting (55.0% for M3 vs .36.2% for M+P ). 5.4 Ablation Studies
We conduct several ablation studies to show that mobile manipulation skills are more ﬂexible to formulate than stationary ones, and to understand the advantage of our navigation reward. Can initial states be trivially enlarged? We conduct experiments to understand to what extent we can enlarge the initial states of manipulation skills given the trade-off between achievability and composability . In the S(L)+P experiment, we simply replace the initial states of stationary manipulation skills with those of mobile ones.
The success rates of stationary manipulation skills on subtasks drop by a large margin, e .
g ., from 95% to 45% for Pick on TidyHouse .
Fig 5 shows that
S(L)+P (37.7%/18.1%) is inferior to both S+P (57.4%/31.1%) and M+P (64.9%/36.2%).
It indicates that stationary manipulation skills have a much smaller set of feasible initial states compared to mobile ones, and including infeasible initial states during training can hurt performance signiﬁcantly.
Besides, we also study the impact of initial state distribution on mobile manipulation skills in Appendix F.
Is the collision penalty important for the navigation skill?
Our region-goal navigation reward beneﬁts from unambiguous region goals and the collision penalty.
We add the collision penalty to the point-goal navigation reward (Eq 1) in S+P(C) and M+P(C) experiments.
Fig 5 shows that the collision penalty signiﬁcantly improves the success rate: S+P(C) (65.2%/44.6%) vs .
S+P
[Figure]
TidyHouse
[Figure]
PrepareGroceries
[Figure]
SetTable
[Figure]
C r o ss - l a you t [Figure] S u cc e ss r a t e S+PM+PM3(ours)S(L)+PS+P(C)
M+P(C) pick_0 place_0 pick_1 place_1 pick_2 place_2 0.
S u cc e ss r a t e
S+PM+PM3(ours)S(L)+PS+P(C)
M+P(C) open_0 pick_0 place_0 close_0 open_1 pick_1 place_1 close_1 0.
S u cc e ss r a t e
S+PM+PM3(ours)S(L)+PS+P(C)
M+P(C)
(57.4%/31.1%) and M+P(C) (67.9%/49.2%) vs .
M+P (64.9%/36.2%).
A collision-aware navigation skill can avoid disturbing the environment, e .
g ., accidentally closing the fridge before placing an object in it.
Besides, M+P(C) is still inferior to our M3 (71.2%/55.0%).
It implies that reducing the ambiguity of navigation goals helps learn more robust and generalizable navigation skills.
Conclusion and Limitations
In this work, we present a modular approach to tackle long-horizon mobile manipulation tasks in the Home Assistant Benchmark (HAB), featuring mobile manipulation skills and the region-goal navigation reward.
Given the superior performance, our approach can serve as a strong baseline for future study.
Besides, the proposed principles ( achievability, composability, reusability ) can serve as a guideline about how to formulate meaningful and reusable subtasks.
However, our work is still limited to abstract grasp and other potential simulation defects.
We leave fully dynamic simulation and real-world deployment to future work.
A Overview
Compared to the original implementation [1], our implementation beneﬁts from repaired assets (Sec B), improved reward functions and better training schemes (Sec C).
Other differences include observation and action spaces.
We introduce in observations the target positions in the base frame in addition to those in the end-effector frame.
The arm action is deﬁned in the joint conﬁguration space (7-dim) rather than the end-effector Euclidean space (3-dim with no orientation).
B Dataset and Episodes
[1] keeps updating the ReplicaCAD dataset.
The major ﬁx is “minor furniture layout modiﬁcations in order to better accommodate robot access to the full set of receptacles” 5 .
The agent radius is also decreased from 0.4m to 0.3m to generate navigation meshes with higher connectivity.
Besides, [1] also improves the episode generator 6 to ensure stable initialization of objects.
Those improvements eliminate most unachievable episodes in the initial version.
The episodes used in our experiments are generated with the ReplicaCAD v1.4 and the latest habitat-lab 7 .
For TidyHouse , each episode includes 20 clutter objects and 5 target objects along with their goal positions, located at 7 different receptacles (chair, 2 tables, tv stand, two kitchen counters, sofa).
For
PrepareGroceries , each episode includes 21 clutter objects located at 8 different receptacles (the 7 receptacles used in TidyHouse and the top shelf of the fridge) and 1 clutter object located at the middle shelf of the fridge.
2 target objects are located at the middle shelf, and each of their goal positions is located at one of two kitchen counters.
The third target object is located at one of two kitchen counters, and its goal position is at the middle shelf.
SetTable generates episodes similar to
PrepareGroceries , except that two target objects, bowl and apple, are initialized at one of 3 drawers and at the middle fridge shelf respectively.
Each of their goal positions is located at one of two tables.
C Skill Learning
Each skill is trained to accomplish a subtask and reset its end-effector at the resting position.
The robot arm is ﬁrst initialized with predeﬁned resting joint positions, such that the corresponding resting position of the end-effector is (0 . 5 , 1 . 0 , 0 . 0) in the base frame 8 .
The initial end-effector position is then perturbed by a Gaussian noise N (0 , 0 . 025) clipped at 0 .
05 m .
The base position is perturbed by a Gaussian noise N (0 , 0 . 1) truncated at 0 .
2 m .
The base orientation is perturbed by a Gaussian noise
N (0 , 0 . 25) truncated at 0.5 radian.
The maximum episode length is 200 steps for all the manipulation skills, and 500 steps for the navigation skill.
The episode terminates on success or failure.
We use the same reward function for both stationary and mobile manipulation skills, unless speciﬁed.
For all skills, d oee is the distance between the end-effector and the object, d ree is the distance between the end-effector and the resting position, d hee is the distance between the end-effector and a predeﬁned manipulation handle (a 3D position) of the articulated object, d ga is the distance between the joint position of the articulated object and the goal joint position.
∆ b a = d b a ( t − 1) − d b a ( t ) stands for the (negative) change in distance between a and b .
For example, ∆ oee is the change in distance between the end-effector and the object.
I holding indicates if the robot is holding an (correct) object or handle.
I succ indicates the task success.
C t refers to the current collision force, and C 1: t stands for the accumulated collision force.
The 7-dim arm action stands for the delta joint positions added to the current target joint positions of the PD controller.
The input arm action is assumed to be normalized to [ − 1 , 1] , and will be scaled by 0.025 (radian).
The 2-dim base action stands for linear and angular velocities.
The base movement in the Habitat 2.0 is implemented by kinematically setting the robot’s base transformation.
The collision between the robot base and navigation meshes is taken into consideration.
The input base action is assumed to be normalized to [ − 1 , 1] , and will be scaled by 3 (navigation skill) or 1.5 (manipulation skills).
For the navigation skill, we follow [1] to use a discrete action space and translate the discrete action into the continuous one.
Concretely, the (normalized) linear velocity from -0.5 to 1 is discretized into 4 choices ( {− 0 .
5 , 1 } ), and
the (normalized) angular velocity from -1 to 1 is discretized into 5 choices (( {− 1 , − 0 .
The stop action corresponds to the discrete action representing zero velocities.
[Table]
[Table]
• Objective: pick the object initialized at s• Initial base position (noise is applied in addition): – Stationary: the closest navigable position to s– Mobile: a randomly selected navigable position within 2m of s• Reward: I pick indicates whether the correct object is picked and I wrong indicates whether a wrong object is picked.
[Table]
• Success: The robot is holding the target object and the end-effector is within 5cm of the resting position.
I succ =
I holding ∧ d
[Table]
≤ 0 .
[Table]
• Failure: – I [ C 1: t > 5000] = 1 : The accumulated collision force is larger than 5000 N .
– I wrong = 1 : A wrong object is picked.
{+– I [ d oee > 0 . 
+}
] I holding = 1 : The held object slides off the gripper.
• Observation space: – Depth images from head and arm cameras.
– The current arm joint positions.
– The current end-effector position in the base frame.
– Whether the gripper is holding anything. – The starting position s 0 in both the base and end-effector frame. • Action space: The gripper is disabled to release. Place
s
[Table]
• Objective: place the held object at s ∗ • Initial base position (noise is applied in addition): – Stationary: the closest navigable position to s ∗ – Mobile: a randomly selected navigable position within 2m of s ∗ • Reward:
[Table]
• Success: The object is within 15cm of the goal position and the end-effector is within 5cm of the resting position.
I succ =
d s ∗ o ≤ 0 .
15 ∧ I !
holding ∧ d
[Table]
≤
[Table]
• Failure: – I [ C 1: t > 7500] = 1 : The accumulated collision force is larger than 7500 N .
– I drop = 1 : The object is released beyond 15cm of the goal position.
{+– I [ d oee > 0 . 
+}
] I holding = 1 : The held object slides off the gripper.
• Observation space: – Depth images from head and arm cameras.
– The current arm joint positions.
– The current end-effector position in the base frame.
– Whether the gripper is holding anything.
– The goal position s ∗ in both the base and end-effector frame.
• Action space: The gripper is disabled to grasp after releasing the object.
Open drawer( s )
• Objective: open the drawer containing the object initialized at s .
The goal joint position of the drawer is g = 0 .
45 m .
• Initial base position (noise is applied in addition): – Stationary: a navigable position randomly selected within a [0 . 80 , − 0 .
35] × [0 . 95 , 0 . 35] region in front of the drawer.
– Mobile: a navigable position randomly selected within a [0 . 3 , − 0 .
6] × [1 . 5 , 0 . 6] region in front of the drawer.
• Reward: I open =
d ga ≤ 0 .
05 indicates whether the drawer is open.
I release indicates whether the handle is released when the drawer is open.
I grasp indicates whether the correct handle is grasped.
a base is the (2-dim) base action.
[Table]
• Success:
The drawer is open, and the end-effector is within 15cm of the resting position.
I succ =
I open ∧ I !
holding ∧ d ree ≤ 0 .•
Failure: – I wrong = 1 : The wrong object or handle is picked.
– I [ d hee > 0 .
2] I holding = 1 : The grasped handle slides off the gripper. – I out = 1 : The robot moves out of a predeﬁned region (a 2 m × 3 m region in front of the drawer). – I I [ open ( t − 1) ∧ ! open ( t )] =
The drawer is not open after being opened.
– The gripper releases the handle when the drawer is not open ( I ! open = 1 ). –
∆ ga > = 0 .
1 : The drawer is opened too fast.
• Observation space: – Depth images from head and arm cameras.
– The current arm joint positions.
– The current end-effector position in the base frame.
– Whether the gripper is holding anything.
– The starting position s in both the base and end-effector frame.
Close drawer( s )
• Objective: close the drawer containing the object initialized at s .
The goal joint position is g = 0 m .
• Initial joint position: q a ∈ [0 . 4 , 0 . 5] , where q a is the joint position of the target drawer.
A random subset of other drawers are slightly open ( q (cid:48) a ≤ 0 . 1 ).
• Initial base position (noise is applied in addition): – Stationary: a navigable position randomly selected within a [0 . 3 , − 0 .
35] × [0 . 45 , 0 . 35] region in front of the drawer.
– Mobile: a navigable position randomly selected within a [0 . 3 , − 0 .
6] × [1 . 0 , 0 . 6] region in front of the drawer.
• Reward:
It is almost the same as Open drawer by replacing open with close .
I close =
d ga ≤ 0 .
1 . • Success: The drawer is closed, and the end-effector is within 15cm of the resting position.
• Failure: It is almost the same as Open drawer by replacing open with close , except that the last constraint ∆ ga > = 0 .
1 is not included.
Open fridge( s )
• Objective: open the fridge containing the object initialized at s .
The goal joint position is g = π 2 .
• Initial base position (noise is applied in addition): a navigable position randomly selected within a [0 . 933 , − 1 . 5] × [1 . 833 , 1 . 5] region in front of the fridge.
• Reward: I open =
g − q a > 0 . 15 , where q a is the joint position (radian) of the fridge.
To avoid the robot from penetrating the fridge due to simulation defects, we add a collision penalty but excludes collision between the end-effector and the fridge.
[Table]
• Success: The fridge is open, and the end-effector is within 15cm of the resting position.
I succ =
I
[Table]
• Failure: – I wrong = 1 : The wrong object or handle is picked.
– I [ d hee > 0 .
2] I holding = 1 : The grasped handle slides off the gripper.
– I out = 1 : The robot moves out of a predeﬁned region (a 2 m × 3 . 2 m region in front of the fridge).
– I I [ open ( t − 1) ∧ !
open ( t )] =
1 : The fridge is not open after being opened.
– The gripper releases the handle when the fridge is not open ( I ! open = 1 ).
• Observation space: – Depth images from head and arm cameras.
– The current arm joint positions.
– The current end-effector position in the base frame.
– Whether the gripper is holding anything.
– The starting position s in both the base and end-effector frame.
Close fridge( s )
• Objective: close the fridge containing the object initialized at s .
The goal joint position is g = 0 .
• Initial joint position: q a ∈ [ π 2 − 0 .
15 , 2 . 356] , where q a is the joint position of the target fridge.
• Initial base position (noise is applied in addition): a navigable position randomly selected within a [0 . 933 , − 1 . 5] × [1 . 833 , 1 . 5] region in front of the fridge.
• Reward:
It is almost the same as Close fridge by replacing open with close .
I close =
d ga ≤ 0 .
• Success: The fridge is close, and the end-effector is within 15cm of the resting position.
[Table]
• Objective: navigate to the start of other skills speciﬁed by s • Reward: refer to Eq 1.
r slack = 0 .
002 , ˜ D
= 0 .
9 , λ ang
= 0 .
25 , λ succ = 2 .•
Success: The robot is within 0.3 meter of the goal, 0.5 radian of the target orientation, and has called the stop action at the current time step.
• Observation space: – Depth images from the head camera.
– The goal position s ∗ in the base frame.
[Table]
• Objective: navigate to the start of other skills speciﬁed by s • Reward: refer to Eq 2.
r slack = 0 .
002 , r col = min(0 .
001 C t , 0 . 2) , λ succ = 2 .•
Success: The robot is within 0.1 meter of any goal in the region, 0.25 radian of the target orientation at the current position, and has called the stop action at the current time step.
• Observation space: – Depth images from the head camera. – The goal position s ∗ in the base frame. 100 Environment steps (x10 6 ) 0. S u cc e ss r a t e
TidyHousePrepareGroceriesSetTable(drawer)
(a
[Figure]
Pick(stationary)     100 Environment steps (x10 6 ) 0.
S u cc e ss r a t e
TidyHousePrepareGroceriesSetTable(drawer) (
[Figure]
(mobile)     100 Environment steps (x10 6 ) 0.
S u cc e ss r a t e
TidyHousePrepareGroceries (c) Place(stationary)     100 Environment steps (x10 6 ) 0.
S u cc e ss r a t e
TidyHousePrepareGroceries (d) Place(mobile)     100 Environment steps (x10 6 ) 0.
S u cc e ss r a t e
SetTable
[Figure]
[Figure]
(stationary)     100 Environment steps (x10 6 ) 0.
S u cc e ss r a t e
SetTable (
[Figure]
Open drawer(mobile)
[Figure]
steps (x10 6 ) 0.
S u cc e ss r a t e
SetTable (
[Figure]
drawer(stationary)
100 Environment steps (x10 6 ) 0.
S u cc e ss r a t e
SetTable (
[Figure]
Close drawer(mobile)
[Figure]
[Figure]
S u cc e ss r a t e
SetTable (i) Open fridge     100 Environment
[Figure]
(x10 6 ) 0.
S u cc e ss r a t e
SetTable (j
[Figure]
[Figure]
(x10 6 ) 0.
[Figure]
[Figure]
[Figure]
[Figure]
S u cc e ss r a t e
TidyHousePrepareGroceriesSetTable
[Figure] [Figure] [Figure] S u cc e ss r a t e
TidyHousePrepareGroceriesSetTable
[Figure]
C.1 PPO Hyper-parameters
Our PPO implementation is based on the habitat-lab.
The visual encoder is a simple CNN 9 .
The coefﬁcients of value and entropy losses are 0.5 and 0 respectively.
We use 64 parallel environments and collect 128 transitions per environment to update the networks.
We use 2 mini-batches, 2 epochs per update, and a clipping parameter of 0.2 for both policy and value.
The gradient norm is clipped at 0.5.
We use the Adam optimizer with a learning rate of 0.0003.
The linear learning rate decay is enabled.
The mean of the Gaussian action predicted by the policy network is activated by tanh .
The (log) standard deviation of the Gaussian action, which is an input-independent parameter, is initialized as − 1 . 0 .
Fig 6 shows training curves of skills.
C.2 Other Implementation Details
The PPO algorithm implemented by the habitat-lab does not distinguish the termination of the environment (MDP) and the truncation due to time limit. We ﬁx this issue in our implementation. Furthermore, we separately train all the skills for each HAB task to avoid potential ambiguity. For example, the starting position of an object in the drawer is computed when the drawer is closed at the beginning of an episode.
However, the skill Pick needs to pick this object up when the drawer is open and the actual position of the object is different from the starting position. It is inconsistent with other cases when the object is in an open receptacle or the fridge. We observe such ambiguity can hurt performance. See Fig 6 for all task-speciﬁc variants of skills.
For the monolithic baseline, a monolithic RL policy is trained for each HAB task. During training, the policy only handles one randomly selected target object, e . g ., picking and placing one object in TidyHouse .
During inference, the policy is applied to each target object. We use the same observation space, action space and training scheme as those for our mobile manipulation skills. The main challenge is how to formulate a reward function for those complicated long-horizon HAB tasks that usually require multiple stages. We follow [1] to composite reward functions for individual skills, given the sequence of subtasks.
Concretely, at each time step during training, we infer the current subtask given perfect knowledge of the environment, and use the reward function of the corresponding skill. To ease training, we remove the collision penalty and do not terminate the episode due to collision. Besides, we use the region-goal navigation reward for the navigation subtask. Thanks to our improved reward functions and better training scheme, our monolithic RL baseline is much better than the original implementation in [1].
However, although able to move the object to its goal position, the policy never learns to release the object to complete the subtask Place during training.
It might be due to exploration difﬁculty since Place is the last subtask in a long sequence and previous subtasks all require the robot not to release.
To boost its performance, we force the gripper to release anything held at the end of execution during evaluation.
E Evaluation E.1 Sequential Skill Chaining
For evaluation, skills are sequentially executed in the order of their corresponding subtasks, as described in Sec 3.3. The main challenge is how to terminate a skill without privileged information. Basically, each skill will be terminated if its execution time exceeds its max episode length (200 steps for manipulation skills and 500 steps for the navigation skill). The termination condition of Pick is that an object is held and the end-effector is within 15cm of the resting position, which can be computed based on proprioception only.
The gripper is disabled to release for Pick .
The termination condition of Place is that the gripper holds nothing and the end-effector is within 15cm of the resting position.
The gripper is disabled to grasp for Place .
Besides, anything held will be released when
Place terminates. For Open and Close , we use a heuristic from [1]: the skill will terminate if the end-effector is within 15cm of the resting position and it has moved at least 30cm away from the resting position during execution. Navigate terminates when it calls the stop action. Furthermore, since the manipulation skills only learn to reset its end-effector, we apply an additional operation to reset the whole arm after each skill.
This reset operation is achieved by setting predeﬁned joint positions as the target of the robot’s PD controller.
E.2 Progressive Completion Rate
In this section, we describe how progressive completion rates are computed.
The evaluation protocol is the same as [1] (see its Appendix F), and here we phrase it in a way more friendly to readers with little knowledge of task planning and Planning Domain Deﬁnition Language (PDDL).
To partially evaluate a HAB task, we divide a full task into a sequence of stages (subgoals). For example, TidyHouse can be considered to consist of pick_0 , place_0 , pick_1 , etc .Each stage can correspond to multiple subtasks. For example, the stage pick_i includes Navigate( s i 0 ) and Pick( s i 0 ) .
Thus, to be precise, the completion rate is computed based on stages instead of subtasks. We deﬁne a set of predicates to measure whether the goal of a stage is completed. A stage goal is completed if all the predicates associated with it are satisﬁed. The predicates are listed as follows:
• holding(target_obj|i) : The robot is holding the i-th object. • at(target_obj_pos|i,target_goal_pos|i) : The i-th object is within 15cm of its goal position.
• opened_drawer(target_marker|i) : The target drawer is open (the joint position is larger than 0.4m). • closed_drawer(target_marker|i) : The target drawer is close (the joint position is smaller than 0.1m).
• opened_fridge(target_marker|i) : The target fridge is open (the joint position is larger than π 2 radian). • closed_fridge(target_marker|i) : The target fridge is close (the joint position is smaller than 0.15 radian).
During evaluation, we evaluate whether the current stage goal is completed at each time step. If the current stage goal is completed, we progress to the next stage.
Hence, the completion rate
[Table]
on ﬁ gu
[Table]
a ti on
[Table]
[Table]
SetTable
[Table] S u cc e ss r a t e
M3(ours)M(S)+RM(L1)+RM(L2)+R
M(L3)+R
open_0 pick_0 place_0 close_0 open_1 pick_1 place_1 close_1 0.
S u cc e ss r a t e
M3(ours)M(S)+RM(L1)+RM(L2)+R
M(L3)+R monotonically decreases.
Listings 1, 2, 3 present the stages deﬁned for each HAB task and the predicates associated with each stage.
Note that the stage goal place_i only indicates that the object has been released at its goal position, but the placement can be unstable ( e .
g .
, the object falls down the table), which can lead to the failure of the next stage.
Besides, due to abstract grasp, it is difﬁcult to place the object stably since the pose of the grasped object can not be fully controlled.
Therefore, we modify the objective of SetTable to make the task achievable given abstract grasp.
Concretely, instead of placing the fruit in the bowl, the robot only needs to place the fruit picked from the fridge at a goal position on the table.
In this section, we study the impact of different initial state distributions on mobile manipulation skills.
We enlarge initial states by changing the distributions of the initial base position (the radius around the target) and orientation.
For reference, the maximum radius around the target is set to 2m in the main experiments (Sec 5).
Several experiments are conducted: M(S)+R , M(L1)+R , M(L2)+R , M(L3)+R .
M(S)+R , M(L1)+R and M(L2)+R stand for the experiments where the maximum radii around the target are set to 1.5m, 2.5m and 4m respectively.
M(L3)+R keeps the radius as 2m, but samples the initial base orientation from [ − π, π ] , instead of using the direction facing towards the target.
Figshows the quantitative results.
Enlarging the initial states in general leads to performance degradation.
Compared to M3 (71.2%/55.0%), M(L1)+R (67.4%/49.7%) and M(L3)+R (67.5%/46.4%) show moderate performance drop.
M(L2)+R (55.2%/38.9%) shows the largest performance drop, which indicates that mobile manipulation skills are not able to handle long-range navigation yet.
Moreover,
M(S)+R (69.5%/52.1%) performs on par with M3 .
It implies that there usually exists a “sweet spot” of the initial state distribution for mobile manipulation skills as a trade-off between achievability and composability .
pick_0: - "holding(target_obj|0)" place_0: - "not_holding()" - "at(target_obj_pos|0,target_goal_pos|0)" pick_1: - "holding(target_obj|1)" - "at(target_obj_pos|0,target_goal_pos|0)" place_1: - "not_holding()" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" pick_2: - "holding(target_obj|2)" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" place_2: - "not_holding()" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" - "at(target_obj_pos|2,target_goal_pos|2)" pick_3: - "holding(target_obj|3)" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" - "at(target_obj_pos|2,target_goal_pos|2)" place_3: - "not_holding()" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" - "at(target_obj_pos|2,target_goal_pos|2)" - "at(target_obj_pos|3,target_goal_pos|3)
" pick_4: - "holding(target_obj|4)" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" - "at(target_obj_pos|2,target_goal_pos|2)" - "at(target_obj_pos|3,target_goal_pos|3)" place_4: - "not_holding()" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" - "at(target_obj_pos|2,target_goal_pos|2)" - "at(target_obj_pos|3,target_goal_pos|3)" - "at(target_obj_pos|4,target_goal_pos|4)"
pick_0: - "holding(target_obj|0)" place_0: - "not_holding()" - "at(target_obj_pos|0,target_goal_pos|0)" pick_1: - "holding(target_obj|1)" - "at(target_obj_pos|0,target_goal_pos|0)" place_1: - "not_holding()" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" pick_2: - "holding(target_obj|2)" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" place_2: - "not_holding()" - "at(target_obj_pos|0,target_goal_pos|0)" - "at(target_obj_pos|1,target_goal_pos|1)" - "at(target_obj_pos|2,target_goal_pos|2)"
open_0: - "opened_drawer(target_marker|0)" pick_0: - "holding(target_obj|0)" place_0: - "not_holding()" - "at(target_obj_pos|0,target_goal_pos|0)" close_0: - "closed_drawer(target_marker|0)" - "at(target_obj_pos|0,target_goal_pos|0)" open_1: - "closed_drawer(target_marker|0)" - "at(target_obj_pos|0,target_goal_pos|0)" - "opened_fridge(target_marker|1)" pick_1: - "closed_drawer(target_marker|0)" - "at(target_obj_pos|0,target_goal_pos|0)" - "opened_fridge(target_marker|1)" - "holding(target_obj|1)" place_1: - "closed_drawer(target_marker|0)" - "at(target_obj_pos|0,target_goal_pos|0)" - "not_holding()" - "at(target_obj_pos|1,target_goal_pos|1)"
close_1: - "closed_drawer(target_marker|0)" - "at(target_obj_pos|0,target_goal_pos|0)" - "closed_fridge(target_marker|1)" - "at(target_obj_pos|1,target_goal_pos|1)"
Listing 3: and are in the order for progressive evaluation.