Title: POMDP Planning for Object Search in Partially Unknown Environment
Venue: NeurIPS 2023
Accepted: yes
Keywords: robotics

Partially Observable Markov Decision Process (POMDP)

object search
Abstract: Efficiently searching for target objects in complex environments that contain various types of furniture, such as shelves, tables, and beds, is crucial for mobile robots, but it poses significant challenges due to various factors such as localization errors, limited field of view, and visual occlusion. To address this problem, we propose a Partially Observable Markov Decision Process (POMDP) formulation with a growing state space for object search in a 3D region. We solve this POMDP by carefully designing a perception module and developing a planning algorithm, called Growing Partially Observable Monte-Carlo Planning (GPOMCP), based on online Monte-Carlo tree search and belief tree reuse with a novel upper confidence bound. We have demonstrated that belief tree reuse is reasonable and achieves good performance when the belief differences are limited. Additionally, we introduce a guessed target object with an updating grid world to guide the search in the information-less and reward-less cases, like the absence of any detected objects. We tested our approach using Gazebo simulations on four scenarios of target finding in a realistic indoor living environment with the Fetch robot simulator. Compared to the baseline approaches, which are based on POMCP, our results indicate that our approach enables the robot to find the target object with a higher success rate faster while using the same computational requirements.

POMDPPlanning for Object Search in Partially Unknown Environment

Abstract

E ffi ciently searching for target objects in complex environments that contain various types of furniture, such as shelves, tables, and beds, is crucial for mobile robots, but it poses significant challenges due to various factors such as localization errors, limited field of view, and visual occlusion. To address this problem, we propose a Partially Observable Markov Decision Process (POMDP) formulation with a growing state space for object search in a 3D region. We solve this POMDP by carefully designing a perception module and developing a planning algorithm, called Growing Partially Observable Monte-Carlo Planning (GPOMCP), based on online MonteCarlo tree search and belief tree reuse with a novel upper confidence bound. We have demonstrated that belief tree reuse is reasonable and achieves good performance when the belief di ff erences are limited. Additionally, we introduce a guessed target object with an updating grid world to guide the search in the information-less and reward-less cases, like the absence of any detected objects. We tested our approach using Gazebo simulations on four scenarios of target finding in a realistic indoor living environment with the Fetch robot simulator. Compared to the baseline approaches, which are based on POMCP, our results indicate that our approach enables the robot to find the target object with a higher success rate faster while using the same computational requirements.

1 Introduction

Finding an object in a partially known and cluttered environment is critical for many robotics tasks. For instance, a robot operating in human environments, such as our homes may need to find mugs, glasses, or books that may be placed on top of various furniture like beds, tables, or chairs and partially occluded by other objects. While the positions of the furniture are commonly known, the positions of the smaller objects, such as the mugs and glasses, are not generally known and move very often. Humans can typically find those smaller objects by relying on prior knowledge, assuming the object they are searching for has been seen before[1] and removing occlusions as necessary. This kind of target objects-oriented problems is also called "Mechanical Search" problem [7].

This paper proposes a planning method based on a novel solver, called Growing Partially Observable Monte-Carlo Planning (GPOMCP), that enables mobile manipulators to do just that: Finding a target object that might be occluded by other objects in a partially unknown environment by moving to positions with better visibility or by removing occlusion when necessary.

We assume the robot is operating in a home environment that consists of multiple rooms and is provided with 3D point cloud and 2D occupancy grid maps of the environment and a set of the target object's photos. These maps contain the furniture but do not contain information about the objects on top of the furniture. GPOMCP computes an e ffi cient search policy that utilizes onboard sensors and its ability to remove objects within a designated search region to find the target object even if it is partially or completely occluded while accounting for localization and perception errors. To handle the partial observability nature of the problem, GPOMCP is based on the Partially Observable Markov Decision Process (POMDP). A unique feature of GPOMCP is it can e ffi ciently modify policies when the number of state variables increases. This feature allows GPOMCP to model the environment incrementally, starting from an environment map with no objects on top of the furniture to a more faithful map where all the objects on top of the furniture have been included in the map and everything in-between, as needed until the target object is found.

In addition to the above solver, we propose a POMDP model for the object-finding problem that exploits GPOMCP's ability to modify policies as the number of state variables increases. To this end, the model maintains a set of state variables that represents information about the robot's guess of the target object, which we refer to as a guessed target object. A belief over this set of state variables represents the robot's uncertainty over the characteristics of the target object, such as the pose, whether the object can be moved or not, etc. As new observations are perceived, the robot refines its belief about the guessed target object, and eventually, the belief has a high chance to converge to the correct target object positions based on suitable parameter settings. Experimental results on four di ff erent scenarios using a Fetch robot simulator in Gazebo indicate that the above model and GPOMCP significantly outperforms POMCP[20].

2 Background

APOMDPis a general mathematical formulation for a sequential decision-making problem where the state is not fully observable by the agent. Formally, it is defined as an 8-tuple < S , A , O , T , Z , R , b 0 , γ > , where the state space S denotes the set of all possible states of the robot and the environment; the action space A denotes the set of all actions the robot can perform; the observation space O means the set of all observations the robot can perceive. At each time step, a POMDP agent is at a state s ∈ S , but this state is generally not known exactly by the agent and therefore the POMDP agent uses a belief b -adistribution over the state space- to represent this state uncertainty. The set of all beliefs is referred to as the belief space and denoted as B . At each time step, the agent performs an action a ∈ A , which may result in the agent arriving at a new state. However, the e ff ect of actions is uncertain and is represented by a conditional probability function, namely the transition function T ( s , a , s ′ ) = Pr ( s ′ | s , a ). Once the agent arrives at a new state (which may not be fully known), the agent perceives perceives an observation o ∈ O , governed by the observation function Z ( s ′ , a , o ) = Pr ( o | s ′ , a ), which is a conditional probability function that represents the observation the robot may perceive after performing action a ∈ A in state s ′ ∈ S . Together with perceiving an observation, the agent also receives an immediate reward, governed by the reward function R ( s , a , s ′ ) ∈ R , which can be parameterized by a state, an action, a state-action pair, or a tuple of state, action, and subsequent state. The estimate on the initial state is called the initial belief and denoted as b 0. Last but not least, γ is a discount factor following 0 < γ < 1 [13]. The goal of solving a POMDP problem is to find an optimal policy Π ∗ ( b ) = argmax a ∈A Q ( b , a ) for belief b ∈ B , where the Q ( b , a )-value is the value of executing action a when the agent is at belief b and continuing optimally afterwards.

POMDP solvers can be broadly classified into two main classes based on their real-time performance: o ffl ine and online solvers. While o ffl ine solvers can achieve optimal solutions for small-scale problems, such as the point-based POMDP algorithm [19, 14], they require too much memory for real-world applications. Most online POMDP solvers commonly explore the sparse belief sampling with a collection of particles and perform a Monte-Carlo tree search (MCTS) to grow the belief tree within the limited planning time. State-of-the-art online methods include Partially Observable Monte Carlo Planning (POMCP) [20], Adaptive Belief

Tree (ABT) [15], Determinized Sparse Partially Observable Tree (DESPOT) [21], Partially Observable Monte Carlo Planning with Observation Widening (POMCPOW) [22], and Adaptive Discretization using Voronoi Trees (ADVT) [9] methods. Among them, POMCP and POMCPOW methods rely on the Monte Carlo backup, while others use the Bellman backup. Although most of these methods are designed to explore belief nodes more deeply in continuous or large action and observation spaces, continuous state, action, and observation spaces continue to pose a significant challenge.

Asubstantial category of mechanical search methodologies is closely intertwined with advancements in learning technologies, including deep reinforcement learning (RL) [12], deep Q-learning [25], receding horizon planning guided by neural network-based distribution [2], deep-geometric inference systems [7, 10]. For example, in [12], the authors use a deep RL procedure that combines teacher-aided exploration, a critic with privileged information, and mid-level representations to uncover a target object occluded by a heap of unknown objects. Except for the learning-based methods, some other methods exploit the classical technologies, like the adaptive horizon exploration algorithm [8] and mixed-integer program (MIP) [3]. A shared characteristic among these methods is their specialized application in specific environments, primarily centered around scenarios involving shelves and boxes.

Some other works on object search are solved using POMDP planning approaches, such as a recent MCTS-based mechanical search method that relies on a 1D occupancy distribution for the current camera observation in cluttered shelves. [11]. The Object-Oriented POMDP (OO-POMDP) [24], which is very popular, considers the state and observation spaces to be factored by a set of n objects, where each belongs to a class with a set of attributes, and the beliefs about di ff erent objects are independent for the small computational reason. Many previous works also limit the workspace to a 2D plane with simple transition and observation models. For example, Aydemir et al. proposed a 2D POMDP formulation for the object search in a room by calculating candidate viewpoints [1]; The object search task has been extended to 3D with an object size-friendly property using a multi-resolution planning algorithm [28]. We follow the same Object-Oriented idea because of the good applicability of the real-world application but go forward one step to consider the occlusion relationship for the belief update between di ff erent objects in a complicated 3D environment, including some challenging cases with partial and fully occluded objects.

3 POMDPformulation for object search

Our focused object search task is to find a movable target object { objtar } in several clustered workspaces that contain an unknown number n > 1 of obstacle objects { objobs } with unknown locations. To facilitate robot navigation, we have access to point cloud and occupancy grid maps representing the planning environment, including furniture. However, we do not possess the point clouds of the obstacle objects { objobs } or the target object objtar . Therefore, during the execution process, we rely on on-board sensors to detect and identify these objects ( objobs ∪ objtar ). Since we gather information about the objects using real-world sensors, they are not mutually independent in our scenario and may influence each other. This di ff ers from the assumption made in OO-POMDP.

State space S . Astate s = { s r , s o 0 , s o 1 , · · · , s o n } ∈ S is represented as two parts including robot configuration and object-oriented factors, where s r is the robot state and s o i denote the object state of the i -th object. A robot state is written as s r = ( p , l h , l p , l t ), where p is the 6D robot base pose, l h is the lift height, l p and l t are the pan and tilt angles of the robot head. At the beginning stage, the robot may not get into the workspace, so no detected object is available. In order to guide the robot toward meaningful actions, we introduce a guessed target object s o 0 = ( p o 0 , s x o 0 , s y o 0 , s z o 0 , g o 0 , mo 0 , uo 0 ) ∈ R 1 × 20 , where p o 0 = ( p p o 0 , p o o 0 ) is the 6D object pose (3D position p p o 0 and 3D rotations represented by quaternion p o o 0 ); s x o 0 , s y o 0 , and s z o 0 mean the sizes along the principal 3D axis of the object; g o 0 ∈ R 1 × 8 denotes 8 grid odds values to identify whether this object is a target or obstacle object, which will be updated using the scores obtained from image matching. mo 0 ∈ R is the odds value to evaluate the move-ability of the guessed target object; uo 0 ∈ {-2 , -1 , 0 , 1 } is the int front value to claim whether this object is updating or not. Specifically, -2 means this object is

W

ViewingArea

FOV

Gf

Robot

Unknown grid update

Oriented

bounding box

Free grid update

with 8 grids

Object grid update

Cut, based on (s,sau)

Setp 1

Setp 2 and 3

v’(b(s)

Belief b(s)

v’(b(s)

G2

G2

01

02

v’(b(s))

00

Belief b(s)

Belief b(s',Sas): s'→Sad

Setp 4

Directly perform

MCTS from no

particles with

new tree

(lad

SA

for additional

particles

New action for

new object

new object

add/

Oadf?

01

020

v"(b(s,sas): sSas/)

V’(b(s,Sas): s'sai)

Main differences: The belief of the old state vector.

removed after performing the removing action; -1 denotes that the object is still updating without performing declaring and removing actions, 0 and 1 means this object has been declared as an obstacle object and a target object. The belief of the position p p o 0 of the guessed target object is saved in a grid world G f generated by all workspaces W using many odds values Odd ( G f ) with a given resolution. Inside each grid, the position is sampled uniformly and its probability is obtained by its odds value. The odds values are updated based on the field of view (FOV) of the camera using real-world measurement in the excursion process, which is similar to the update of the occupancy grid map in Simultaneous Localization and Mapping (SLAM) [4], following:

where P ( G f | z 1: t ) and P ( ¬G f | z 1: t ) means the probability of the object belonging to and not belonging to this grid based on multiple observations z 1: t ; Odd ( G f | z 1: t ) is the corresponding odd value. In the belief tree search, we do not update the odds values Odd ( G f ) and it will be updated after the real-world excursion. In the planning stage, this grid world is just used to sample the potential position of the guessed target object in the root node. The guessed target object is special with a constant orientation (set as (0, 0, 0, 1)), size (0.1, 0.1, 0.1), and move-ability value (set as 100, movable). The grid values g o 0 and the declared value uo 0 are update-able in the belief tree search but need to be reinitialized as the given value after each excursion. Fig.1 shows the scenario about the grid world for the guessed target object. In fact, the guessed target object variable within the state vector saves the belief about the position of the target object. It not only encodes the probability information derived from the grid world with odds values but also facilitates e ffi cient planning within Monte Carlo tree search with low computational complexity, acceptable memory cost, and coding friendliness. More explanations to the guessed target object are shown in the supplementary materials. The other object s o i , i , 0 ∈ R 1 × 20 follows the same format and all parameters should be updated both in belief tree search and real-world excursion. It is noted that the pose of the other objects s o i is estimated based on the point cloud which is independent of the grid world G f . The newly detected objects after each real-world excursion will be appended to the state vector and form a growing state space ( s , s add ) ∈ S × S add .

Action space A . Our method is task-level planning with many primitive actions instead of control-level planning. The key purpose of setting these actions is to change the FOV of the robot camera to cover the workspace and further operate objects. The action space consists of 5 types of primitives: Move ( p t , ω m ) means to move the robot base to a given target pose p t ; Head ( l p , l t ) denotes the actions to change the angles of the robot head pan (left / right) and tilt (up / down) to l p for the pan and l t for the tilt; Li f t ( l h ) represents to move the robot height to l h ; Declaring ( uo i ) means to declare the updating object ( uo i = -1) to be the target object ( uo i = 1) or the obstacle object ( uo i = 0); Removing ( i , ω r ) is the action to remove i -th object using robot manipulator. All actions are assumed to have a 100% success rate for simplicity 2 .

Observation space O . The main focused observation is a FOV viewing frustum V projected from the robot head camera. Because we view each object by an oriented box with 8 grids, for each observation, we assume to update at most the 4-nearest grids with a noise negative log-odds value -co + η , a noise positive log-odds value co + η , a noise near-zero log-odds value η , where η follows the Gaussian distribution. Following the blind spot caused by the other objects, sometimes, the robot camera can not detect all 4-nearest grids. We set the observation space to be discrete. By fusing all no detected objects, we can get the observation to be {{ i , · · · , j } , { o ( i ) , · · · , o ( j ) }} , |{ i , · · · , j }| = no , where { i , · · · , j } records the observed object with at least one updated grid and { o ( i ) , · · · , o ( j ) } are their updated log-odds value.

Transition function T . If the actions Move ( p t , ω m ), Head ( l p , l t ), and Li f t ( l h ) are applied to the robot, the camera location and orientation will change with some noises. The camera motion follows the rigid transformation and the static structure information of the applied robot is read from tf message. The declaring action Declaring ( uo i ) will change the robot status uo i based on the comparison between the mean value of the minimal 2 log-odds of 8 grid values and the thresholds C o d for the obstacle object and C o t for the target object. If no removing action is performed to the i -th object, the i -th object is static and rigid. The removing action Removing ( i , ω r ) on i -th object will move the i -th object outside the workspace and set the status uo i to -2, which means they will not block the observation of the other objects.

Observation function Z . The coordinates of 8 grids of the i -th object can be computed using center point p p o i and the orientation p o o i of the object pose and its size s x o 0 , s y o 0 , s z o 0 estimated by point cloud. After performing one action, the visible grids of di ff erent objects belonging to the 4 nearest grids will be updated. The updated value will be decided based on the mean log-odds value of all grids. If the original mean log-odds value is larger than a positive threshold ν p > 0, the updated value will be set to be positive c 0 + η . If it is smaller than a negative threshold ν n < 0, the updated value will be set to be negative -c 0 + η . Otherwise, the value between the positive and negative thresholds will generate a near-zero η . So as to encourage the exploration of the robot, the initial grid values g o 0 of the guessed target object are reinitialized to be larger than the positive threshold ν p > 0 after each real-world excursion.

Reward function R . Our expected sequential actions involve the robot moving to the workspace for observation, continuously changing its viewpoints to gather more measurements, updating odds values in 8 grids, then declaring the updating object to be the obstacle or target object, and finally removing the obstacle objects for the more visible area or directly the target object to complete the whole task. Hence, if the whole task is completed successfully with the removal of the target object, the robot will receive a large positive reward Rmax ≫ 0. As the prerequisite action, correctly identifying the obstacle and target objects yields relatively large rewards ( Rco and Rct , respectively), where Rmax ≫ Rct > Rco ≫ 0. All actions incur a negative step cost ( Rmin < 0). The cost of the removal action Removing ( i , ω r ) is set as 2 Rmin due to its complexity. Illegal actions incur a large negative penalty ( Rill ≪ 0), including collisions with occupied grids in the occupancy grid map, attempting to remove immovable objects, declaring an object that doesn't meet the condition, removing updating objects without declaration, and trying to remove objects that have already been removed.

4 Action execution and perception with on-board sensors

Our perception part is carefully explored with many useful outputs, like estimated object pose, estimated object size, object move-ability, and object detector, fusing both point cloud data and the image data without using manual quick response (QR) codes. Our framework is suitable for all mobile robots with 2D Lidar and RGBD cameras, but specifically, we mainly consider the Fetch robot here. Move base actions are implemented using a ROS interface move_base and the interaction with the AMCL-based navigation stack [18]. The robot lift height and head joints are controlled by following the joint-space trajectories on a group of joints based on a ROS interface ros_control [5]. Removing action is to pick up the object and place it in some given areas outside the workspace using the ROS tool moveit [6].

To simplify the scenario and reduce the navigation e ff ort, we assume the availability of a pre-built point cloud map M of the robot's environment, including furniture and known

objects, which can be created beforehand through environment mapping and reused during planning. Then, the 3D point cloud F j in j -th frame from the depth camera is matched with the map M by iterative closest point (ICP) [26]. The 3D ICP is initialized by matching 2D Lidar points with the 2D occupancy grid map. As the frame number increases, the point clouds of objects and environment P j = M S F 0 S · · · S F j are becoming more and more complete. After removing the original map M′ j = P j / M , the point clouds for n newly detected objects { o 0 , · · · , on -1 } ∈ M ′ j are extracted by point cloud segmentation using the Euclidean cluster extraction method. For estimating object pose and size, we obtain the minimum oriented bounding box for each object using principal component analysis. The point cloud segmentation is applied followed by data association based on the Mahalanobis distance of centroid points and point-wise mean distance. Additional details can be found in the supplementary materials.

So as to complete the given object detection task using several given images and some semantic words (optional), based on the previous point cloud segmentation, we perform the point cloud re-projection, sub-image fusion, YOLO detection, and SIFT matching [17] to compute the probability of object detection. We also would like to manipulate the objects in the workspace to free some FOV, so we estimate the move-ability rmove of each object by the learning-based Grasp Pose Detection (GPD) toolbox [23], the k-means clustering algorithm, and the ROS moveit toolbox. The details of the implementation are presented in supplementary materials. In the planning stage, for each particle, we will randomly sample a value for this object and compare it with the move-ability rmove to identify its move-ability.

5 Belief tree reuse for online POMDP solver with growing state space

Here, we present a POMDP solver, GPOMCP, reusing the previous belief tree when the state space of the problem is growing. In this kind of POMDP problem, the observation obtained by real measurement causes the unpredicted new state, which means that the whole belief tree should be cut and a totally new belief tree should be computed by sampling. It is time-consuming. As shown in Fig. 2, we solve the problem by following steps:

Step 1 : When state space grows from s ∈ S to ( s , s add ) ∈ S × S add , in the old belief tree T , we can pick out the branches corresponding to old sub-state s and set it as the root of new tree T ′ ( s ).

Step 2 : This new tree T ′ ( s ) has many particles with history h . For each particle, we can update the future state s ′ by the added state vector s ′ add corresponding to new objects based on the transition function T ( s add , a , s ′ add ) and observation function Z ( s ′ add , a , o add ) 3 . It is noted that, in the state updating, we should consider the e ff ect s ′ → s ′ add from the old state vector s corresponding to the old object. So we can get the updated histories about some or all particles within the given time limitation.

Step 3 : Then, starting from the root, we can grow the belief tree T ′ ( s ) using these updated particles 4 . It is noted that, in building the new belief tree T ′ ( s ), the new observation nodes with newly detected objects may be introduced, which means that the generated belief tree T ′ ( s ) may be not a sub-tree of the old belief tree T .

Step 4 : Then, we perform the Monte Carlo tree search (MCTS) for all branches including the new actions and new observations. The action selection strategy follows the upper confidence bound (UCB) and the belief distance Cb P o ∈ O Z ( s ′ new , a , o ′ ) ∥ b (( s ′ , s ′ add ) : s ′ → s ′ add ) -b (( s ′ , s ′ add ) : s ′ ⇄ s ′ add ) ∥ 1 + V ∗ ( b , a ) + CN q logN ( b ) N ( b , a ) , where Cb and CN are coe ffi cients set by users.

We know that the current optimal action should be selected following equation Eq. (2).

where s new = ( s , ∅ ) and s ′ new = ( s ′ , s ′ add ). Considering two ways (GPOMCP and the direct resampling way), in the sub-tree of the new belief tree, the only di ff erence is from the last terms V ∗ ( b ( s ′ new : s ′ ⇄ s ′ add )) (correct by building tree) and V ∗ ( b ( s ′ new : s ′ → s ′ add )) (GPOMCP, without considering partial e ff ect). Then, we have the following Theorems:

Theorem 1. We are considering the unweighted particles for approximating the extended full-correct belief b ( s ′ new : s ′ ⇄ s ′ add ) = 1 N P N i = 1 I ( s ′ new = s i new ′ : s ′ ⇄ s ′ add ) and the reused approximated belief b ( s ′ new : s ′ → s ′ add ) = 1 N P N i = 1 I ( s ′ new = s i new ′ : s ′ → s ′ add ) . Assuming the reward function is Borel measurable and bounded, denoted Rmax = ∥ R ∥ ∞ , the belief L1 distance is limited in ∥ b ( s ′ new : s ′ ⇄ s ′ add ) -b ( s ′ new : s ′ → s ′ add ) ∥ 1 ≤ δ , and the optimal action selected by building the whole tree is same as the one selected by approximated value max a V ( b ( s new ) , a ) = max a ˆ V ( b ( s new ) , a ) = a ∗ , the optimal value function of POMDP problem using our method V ∗ ( b ( s new )) and the optimal value function using the direct resampling way ˆ V ∗ ( b ( s new )) will satisfy the following bounding equation:

Proof. Please refer to the supplementary material.

□

This theorem indicates that the reuse technology works well with a performance bound when the reused belief tree shows good approximations to the exact accurate belief tree. To enhance the estimation of the upper bound, we introduce a term Cb P o ∈ O Z ( s ′ new , a , o ′ ) ∥ b ( s ′ new : s ′ → s ′ add ) -b ( s ′ new : s ′ ⇄ s ′ add ) ∥ 1 to the UCB strategy when reusing the previous belief tree.

It is noted that Theorem 1 is correct only under the assumption max a V ( b ( s new ) , a ) = max a ˆ V ( b ( s new ) , a ) = a ∗ , which means that there are some bounds for the performance guarantee shown in Theorem 1 using the reuse approximation.

Discussion 1. In fact, our belief tree reuse can be considered as the wrong estimation △ of the action and measurement node values in the correct belief tree. When the distance between the value function for the optimal action V ( b ( s new ) , a ∗ ) and the value function for any sub-optimal action V ( b ( s new ) , a ∗ sub ) is relatively large satisfying V ( b ( s new ) , a ∗ ) -V ( b ( s new ) , a ∗ sub ) >> △ , which means that the estimation error △ will not a ff ect the final solution and the convergence of our solver follows the method directly rebuilding the whole tree.

Following Discussion 1, we would like to get the exact performance bound to ensure that the obtained solution remains the same as the optimal solution using the fully-rebuilt belief tree. Firstly, we have the following corollary:

Corollary 1. With the same assumption as Theorem 1, given the action a , the optimal action-value function of POMDP problem using our method V ∗ ( b ( s new ) , a ) and the optimal value function using the direct resampling way ˆ V ∗ ( b ( s new ) , a ) will satisfy the following bounding equation:

Proof. Just follows the similar proof process shown in Theorem 1, so we ignore it here. □

Corollary 2. With the same assumption as Theorem 1, if the distance between the value function for the optimal action V ( b ( s new ) , a ∗ ) and the value function for any sub-optimal action V ( b ( s new ) , a ∗ sub ) is larger than 2 γ Rmax 1 -γ δ , the optimal action obtained by the tree reuse approximation way will be same as the optimal action using the direct resampling way max a V ( b ( s new ) , a ) = max a ˆ V ( b ( s new ) , a ) = a ∗ .

Proof. Please refer to the supplementary material.

□

The pseudocode Algorithms of the GPOMCP solver are shown in supplementary materials. Our proposed method follows the common procedure with four alternating stages, planning, execution, obtaining observation, and filtering. The optimal action for the current belief

is selected based on GPOMCP. Inside the POMDP solver, the Monte Carlo update 5 and a revised UCB strategy considering the belief error proposed in Section 5 are applied.

6 Simulations

In this section, we showcase our method's e ff ectiveness via simulations conducted using C ++ and Python codes within the Fetch robot simulator. The project operates on multiple Ubuntu 18.04 desktops equipped with an Intel Core i7-13700k processor, running exclusively on CPU. We evaluate our method using various object settings in di ff erent scenarios.

Problem settings We implement our approach in a realistic gazebo living environment (Fig. 3a). The 3D point cloud map and 2D occupancy grid map, generated using [16], of this environment with furniture are available before the object searching task (Fig. 3b). There are multiple workspaces on the surfaces of the large furniture, like tables, beds, and desks, in this planning environment. Some unknown objects including a target object are set inside the di ff erent workspaces sometimes with an occlusion relationship. Four photos of the target object from di ff erent viewing directions are available for the object detector.

We manually set several candidate poses that can observe partial workspace for the move base action Move ( p t , ω m ). The candidate robot head motion ( l p , l t ) are set within l p ∈ {-1 12 π, 0 , 1 12 π } , l t ∈ {-0 . 5 , 0 , 0 . 5 } , and the lift motion l h is limited in l h ∈ { 0 . 0 , 0 . 2 , 0 . 4 } . The FOV of the Fetch camera has a 60-degree horizontal view angle with a 480 / 600 height-width ratio. In the transition and observation function, using these actions, we add the nearest (0.5 meters to the camera center) and farthest planes (1.7 meters to the camera center) to the FOV and form a hexahedron. This hexahedron-shaped FOV will move based on the robot's motion and configuration change. As mentioned in Section 3, each object is represented as 8 grids based on its optimal oriented box, and the occlusion is computed using perspective projection of the cube vertices. The grid size of the grid world is set as 2 cm. A discussion about di ff erent grid sizes is shown in the supplementary materials. The log-odds value of these grids is initially set as unknown satisfying log 0 . 5 1 -0 . 5 = 0. These odds values will be updated after each real-world observation and then used to sample the potential position of the guessed target object. The planning time is limited to 60 s / step, the discounted factor γ = 0 . 9, the threshold to the object declaration C t d = log(8 / 2) and C o d = log(2 / 8), the thresholds for grid updating ν p = 0 . 1 and ν n = -0 . 1, the re-initialized grid values of the guessed target object are set as 0.2, the reward values Rmax = 10 5 , Rct = 5 × 10 4 , Rco = 10 4 , Rmin = -1, and Rill = -10 3 .

Comparison results Because a lot of existing object search approaches are based on the pure POMCP method with di ff erent problem settings, eg. [27, 24], we compare our approach using Bellman (Ours_b) or Monte Carlo update (Ours_m) with the POMCP method without using / using the guessed target object (denoted as POMCP, POMCP_f). The comparison is implemented based on 4 di ff erent scenarios with di ff erent object numbers, including LOOSE 1 (4 objects in 1 workspace), LOOSE 2 (6 objects in 2 workspaces), Hidden 1 (7 objects

in 1 workspace), and Covered 1 (7 objects in 1 workspace), as shown in Fig. 4. The target object is the blue snack box shown in the blue circle. Only the black headset box and the dish rack are not movable, and the others are all movable. Green arrows show the candidate robot poses. "Tick" means the target objects can be observed easily without changing the robot configuration. "Half tick" means the target can be observed if we change the robot configuration. "Cross" means the target object is fully hidden by the obstacle objects and cannot be seen from that pose even changing the robot configuration, including the lift of the robot, if the obstacle objects are not removed before. For each scenario, 20 trials are conducted and statistical results are reported. More resuls with di ff erent problem settings are shown in the supplementary materials.

(a)

LOOSE

1

(b)

LOOSE

2

(c)

Hidden

1

(d)

Covered

1

opacr

The obtained 95% confidence interval of the discounted cumulative reward, the steps, and the success rate are reported in Table 1. For clarity, we present one sequence of POMDP-generated actions (Fig. 5), of which the discounted cumulative reward is 57691.7, to complete the Covered 1 scenario by moving its pose, changing its configuration, declaring, and removing the target object within 14 steps. The robot removes the obstacles (objects 3 and 4) to free the visual occlusion and finally declares and removes the target objects. The sub-images in the red boxes show the probability of the grid world for the guessed target objects, which is updated using Eq. 1, and the ones in yellow boxes show the detected camera images. We can find that, with the growth of the robot's actions, the probability of most grids far away from the target object decreases quickly and the ones near the target object increase a lot, which guides the robot to make more e ff ort to observe the area with the target object.

Movebasetopose 2

Movebasetopose0

Moveheadto(π/12,0)

Declareobject4(obstacle)

Reward:-1

Reward:-1

Reward:-1

Reward:9999

Removeobject4

Reward:-2

Movehead to (0,0)

Remove object3

Declare object3(obstacle)Declare object5(obstacle)

Reward:-1

Reward:-2

Reward:9999

Reward:9999

Movebase topose 2

Reward:-1

Declare object7(target)

Move head to(-π/12,0.5)Declare object 7

Removeobject7

Reward:-1

Reward:49999

Reward:99998

Reward:-1001

Our method, utilizing Monte Carlo update, demonstrates superior performance in complex object search scenarios such as cases like Hidden 1 and Covered 1, where predicting camera observations is challenging. In simpler cases like Loose 1, our method performs comparably to the POMCP method when using our introduced guessed target object. The guessed target object proves to be beneficial in activating the robot in situations with limited information. By incorporating our approximated belief tree reuse, we retain more useful branches after receiving observations, leading to further improvement in overall performance. The concept of the guessed target object aligns with our natural tendency to assume the presence of certain objects in specific locations and adjust our estimations based on visual observations.

Limitation Our main limitations mainly stem from the errors and failures from perception and navigation components, not our focused planning part. The first limitation is the

pre-existing map M which may not be available for a real robot but can be solved by introducing advanced sementic SLAM method. Achieving 100% primitive action success in reality is impractical, a ff ecting our method's success rate. Another limitation is that our framework is based on point cloud segmentation to identify di ff erent objects. It may generate incorrect bounding boxes for objects with extensive contact areas. This misinterpretation can result in erroneous data association, impacting belief updates and removing actions. Furthermore, YOLO and SIFT object detection techniques may struggle in scenarios with limited SIFT features and YOLOv5 toolbox coverage, especially in low-light environments. However, we believe that addressing these limitations is possible by incorporating more advanced perception and navigation methods.

7 Conclusion and future work

We propose a novel object-level POMDP formulation with growing state space, a guessed target object, and its grid world for object search in 3D with real-world sensors and solve it using a novel GPOMCP method based on Monte-Carlo tree search and belief tree reuse. We demonstrate the e ff ectiveness of our approximation in cases with similar beliefs. Our method achieves object search with only maps, photos, and onboard sensors without relying on additional object information, like geometric models, making it highly practical for realworld robot applications. By utilizing the guessed target object, our method can successfully locate fully hidden objects even without prior information, by removing obstacles-a challenge for existing approaches. Our future directions involve dropping prior information, such as pre-built point cloud map M , and exploring continuous action domains. Through our experiments, we showcase the e ffi ciency of our method, and we anticipate that our results will inspire further advancements in this field.

References

Alper Aydemir, Andrzej Pronobis, Moritz Göbelbecker, and Patric Jensfelt. Active visual object search in unknown environments using uncertain semantics. IEEE Transactions on Robotics , 29(4):986-1002, 2013.

Wissam Bejjani, Wisdom C Agboh, Mehmet R Dogar, and Matteo Leonetti. Occlusion-aware search for object retrieval in clutter. In 2021 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 4678-4685. IEEE, 2021.

Lawrence Yunliang Chen, Huang Huang, Michael Danielczuk, Je ff rey Ichnowski, and Ken Goldberg. Optimal shelf arrangement to minimize robot retrieval time. In 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) , pages 993-1000. IEEE, 2022.

Yongbo Chen, Shoudong Huang, and Robert Fitch. Active slam for mobile robots with area coverage and obstacle avoidance. IEEE / ASME Transactions on Mechatronics , 25(3):1182-1192, 2020.

Sachin Chitta, Eitan Marder-Eppstein, Wim Meeussen, Vijay Pradeep, Adolfo Rodríguez Tsouroukdissian, Jonathan Bohren, David Coleman, Bence Magyar, Gennaro Raiola, Mathias Lüdtke, and Enrique Fernández Perdomo. ros_control: A generic and simple control framework for ros. The Journal of Open Source Software , 2017.

David Coleman, Ioan Sucan, Sachin Chitta, and Nikolaus Correll. Reducing the barrier to entry of complex robotic software: a moveit! case study. arXiv preprint arXiv:1404.3785 , 2014.

Michael Danielczuk, Andrey Kurenkov, Ashwin Balakrishna, Matthew Matl, David Wang, Roberto Martín-Martín, Animesh Garg, Silvio Savarese, and Ken Goldberg. Mechanical search: Multi-step retrieval of a target object occluded by clutter. In 2019 International Conference on Robotics and Automation (ICRA) , pages 1614-1621. IEEE, 2019.

Megha Gupta, Thomas Rühr, Michael Beetz, and Gaurav S Sukhatme. Interactive environment exploration in clutter. In 2013 IEEE / RSJ International Conference on Intelligent Robots and Systems , pages 5265-5272. IEEE, 2013.

Marcus Hoerger, Hanna Kurniawati, Dirk Kroese, and Nan Ye. Adaptive discretization using voronoi trees for continuous-action pomdps. In Algorithmic Foundations of Robotics XV: Proceedings of the Fifteenth Workshop on the Algorithmic Foundations of Robotics , pages 170-187. Springer, 2022.

Huang Huang, Marcus Dominguez-Kuhne, Vishal Satish, Michael Danielczuk, Kate Sanders, Je ff rey Ichnowski, Andrew Lee, Anelia Angelova, Vincent Vanhoucke, and Ken Goldberg. Mechanical search on shelves using lateral access x-ray. In 2021 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 2045-2052. IEEE, 2021.

Huang Huang, Letian Fu, Michael Danielczuk, Chung Min Kim, Zachary Tam, Je ff rey Ichnowski, Anelia Angelova, Brian Ichter, and Ken Goldberg. Mechanical search on shelves with e ffi cient stacking and destacking of objects. In The International Symposium of Robotics Research , pages 205-221. Springer, 2022.

Andrey Kurenkov, Joseph Taglic, Rohun Kulkarni, Marcus Dominguez-Kuhne, Animesh Garg, Roberto Martín-Martín, and Silvio Savarese. Visuomotor mechanical search: Learning to retrieve target objects in clutter. In 2020 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 8408-8414. IEEE, 2020.

Hanna Kurniawati. Partially observable markov decision processes and robotics. Annual Review of Control, Robotics, and Autonomous Systems , 5:253-277, 2022.

Hanna Kurniawati, David Hsu, and Wee Sun Lee. Sarsop: E ffi cient point-based pomdp planning by approximating optimally reachable belief spaces. In Robotics: Science and systems , volume 2008. Citeseer, 2008.

Hanna Kurniawati and Vinay Yadav. An online pomdp solver for uncertainty planning in dynamic environment. In Robotics Research: The 16th International Symposium ISRR , pages 611-629. Springer, 2016.

Mathieu Labbé and François Michaud. Rtab-map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation. Journal of Field Robotics , 36(2):416-446, 2019.

David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision , 60:91-110, 2004.

Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Eric Berger, Rob Wheeler, and Andrew Ng. ROS Navigation stack. http: // wiki.ros.org / navigation, 2009. Accessed: May 4, 2023.

GuyShani, Joelle Pineau, and Robert Kaplow. A survey of point-based pomdp solvers. Autonomous Agents and Multi-Agent Systems , 27:1-51, 2013.

DavidSilver and Joel Veness. Monte-carlo planning in large pomdps. Advances in neural information processing systems , 23, 2010.

Adhiraj Somani, Nan Ye, David Hsu, and Wee Sun Lee. Despot: Online pomdp planning with regularization. Advances in neural information processing systems , 26, 2013.

Zachary Sunberg and Mykel Kochenderfer. Online algorithms for pomdps with continuous state, action, and observation spaces. In Proceedings of the International Conference on Automated Planning and Scheduling , volume 28, pages 259-263, 2018.

Andreas Ten Pas, Marcus Gualtieri, Kate Saenko, and Robert Platt. Grasp pose detection in point clouds. The International Journal of Robotics Research , 36(13-14):1455-1473, 2017.

Arthur Wandzel, Yoonseon Oh, Michael Fishman, Nishanth Kumar, Lawson LS Wong, and Stefanie Tellex. Multi-object search using object-oriented pomdps. In 2019 International Conference on Robotics and Automation (ICRA) , pages 7194-7200. IEEE, 2019.

Yang Yang, Hengyue Liang, and Changhyun Choi. A deep learning approach to grasping the invisible. IEEE Robotics and Automation Letters , 5(2):2232-2239, 2020.

Juyong Zhang, Yuxin Yao, and Bailin Deng. Fast and robust iterative closest point. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(7):3450-3466, 2021.

Kaiyu Zheng, Anirudha Paul, and Stefanie Tellex. A system for generalized 3d multi-object search. arXiv preprint arXiv:2303.03178 , 2023.

Kaiyu Zheng, Yoonchang Sung, George Konidaris, and Stefanie Tellex. Multi-resolution pomdp planning for multi-object search in 3d. In 2021 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 2022-2029. IEEE, 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of finding efficient policies for a robot to search for a known target object in a physical environment.  The robot can find the target object by either moving around to change its field of view or by moving objects that occlude the target object to reveal the target object. While the robot has access to 3D point clouds and occupancy grid maps of the environment and included furniture, objects on the furniture are not included in the occupancy grid maps.   The authors frame this as a POMDP as the state of the environment is partially observed.  The authors propose a novel POMDP solver – Growing Partially Observable Monte-Carol Planner (GPOMCP) – which reuses the previous belief tree under very specific conditions as defined in Theorem 1, Corollary 1,2 – where the optimal action obtained by reusing the tree will be the same as direct sampling.  Overall this is an important practical problem for robots and a new technique for addressing the infeasibility of exactly solving the resulting POMDP is welcome.\n\nI appreciate the responses of the authors during the rebuttal phase and have raised my rating to Accept.\n\nSTRENGTHS:\nThe authors clearly explain their approach for action execution using ROS interfaces move-base, ros_control and moveit, perception for object detection using point cloud segmentation, point cloud re-projection, sub-image fusion, YOLO detection and SIFT matching, and an estimation of each objects move-ability using the Grasp Pose Detection toolbox, k-means clustering and moveit. The authors explain their planning method, GPOMCP, including their use of the belief tree, and provide appropriate steps, equations, theorems, and corollaries. The authors present simulation results using Gazebo across four environment scenarios based on whether the target object is loose, hidden, or covered and compare their proposed method with the Bellman update (method 1) and the Monte Carlo update (method 2) with the primary POMCP method without the fake object (method 3) and with the fake object (method 4). \n\nThe authors also introduce the concept of a “fake object” (last paragraph of section 1), which represents a fully occluded object to facilitate creating a distribution for the pose of this object.  The authors present results from a Gazebo simulation system comparing their algorithm against prior baselines in 4 levels of difficulty based on the visibility of the target object.  Table 1 indicates that the GPOMCP Planner more successfully locates the target object and with fewer steps.\n\nWEAKNESSES:\nThis paper addresses an interesting problem but the Prior Work section misses important precedents.  In line 105, the authors state that when the target object is fully occluded, this presents a new challenge to the best of their knowledge.  However, this problem – where the target object is fully occluded – is well-known in the robotics literature where it is often referred to as “Mechanical Search”.   The problem has also been posed as a POMDP and addressed with Monte-Carlo Search Tree methods (see references below, 1 uses MC Trees).\n\nThe authors did not include a Limitations section, this must be fixed if the paper is to be accepted.\n\nThe paper would be improved with more carefully proofreading to make the English clearer, for example the term “fake object” seems inappropriate.  A better term might be “hidden object” as there is nothing “fake” about this, and a more detailed description of how the distribution evolves in the experiments under the “Covered” condition.\n\nPotentially Relevant Prior Work:\n\nHuang, H. et al. (2023). Mechanical Search on Shelves with Efficient Stacking and Destacking of Objects. In: Billard, A., Asfour, T., Khatib, O. (eds) Robotics Research. ISRR 2022. Springer Proceedings in Advanced Robotics, vol 27. Springer, Cham. https://doi.org/10.1007/978-3-031-25555-7_14\nDanielczuk, M., et al.: Mechanical search: multi-step retrieval of a target object occluded by clutter. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 1614–1621. IEEE (2019)\nKurenkov, A., et al.: Visuomotor mechanical search: learning to retrieve target objects in clutter. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8408–8414. IEEE (2020)\nYang, Y., Liang, H., Choi, C.: A deep learning approach to grasping the invisible. IEEE Robot. Autom. Lett. 5(2), 2232–2239 (2020)\nBejjani, W., Agboh, W.C., Dogar, M.R., Leonetti, M.: Occlusion-aware search for object retrieval in clutter. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4678–4685. IEEE (2021)\nGupta, M., R¨uhr, T., Beetz, M., Sukhatme, G.S.: Interactive environment exploration in clutter. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5265–5272. IEEE (2013)\nHuang, H., et al.: Mechanical search on shelves using lateral access x-ray. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2045–2052. IEEE (2021)\nChen, L.Y., Huang, H., Danielczuk, M., Ichnowski, J., Goldberg, K.: Optimal shelf arrangement to minimize robot retrieval time. In: 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), pp. 993–1000. IEEE (2022)\nNakhimovich, D., Miao, Y. and Bekris, K.E., 2023. Resolution Complete In-Place Object Retrieval given Known Object Models. arXiv preprint arXiv:2303.14562.\n\nQUESTIONS:\nIn lines 53-54, the authors claim that the method will always converge to a delta distribution at the correct target pose.  It would be helpful to justify with a proof, or consider conditions where the method may not converge or may converge to the incorrect object pose.\n\nThe paper would also be improved with more details on the specific experiments as Figure 4 is difficult to follow (is the blue box the target in all 4 examples? This is never stated).\n\nOn a minor note, the paper has some typos or style choices that need correction, such as:\nLine 68: ‘an action’ not ‘a action.’ \nLine 145: ‘shows’ not ‘to show .’\nLine 147: ‘and’ not ‘and but .’\nLine 290: ‘rebuilt’ not ‘rebuild’ \n\nThe authors did not include a Limitations section, this must be fixed if the paper is to be accepted.\n\nLIMITATIONS:\nThe authors did not include a Limitations section.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

comments: SUMMARY:
This paper presents a POMDP based approach for finding objects in partially unknown environments.  They present a planning algorithm that reuses the belief tree and uses a fake object to guide the search.  The system is tested in four simulated scenes in Gazebo and outperforms a POMCP baseline.

STRENGTHS:
- The proposed system achieves better performance than the baseline in several experiments.

WEAKNESSES:
- The main experiment of the paper is a set of comparisons to POMCP [13], which was published in 2010. The related works section of the proposed work mentions several other more modern approaches.  It is not clear why these works were not also compared to.

QUESTIONS:
- Are the object positions the same in the 20 trials for each scenario or are they randomized?  If they are the same, only testing on four different configurations of objects is too few, especially when the experiments are done in simulation.


- I am confused about how the fake object is used and what the intuition behind it is.

LIMITATIONS:
Limitations are only very briefly discussed in the conclusion.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-06

comments: SUMMARY:
This paper proposes a method for mobile robots to efficiently search for objects in complex environments. They use a Partially Observable Markov Decision Process (POMDP) formulation and a planning algorithm called Growing Partially Observable Monte-Carlo Planning (GPOMCP) to improve the robot's success rate and speed in finding target objects.

STRENGTHS:
* A Monte-Carlo planning method to perform object search in the indoor environment based on the grid-type 3D representation.

* A systematic framework and experiments demonstrate the effectiveness of the whole sense-plan-act pipeline.

* A novel design for growing state space using a belief tree.

WEAKNESSES:
* The whole framework seems complex and required carefully mannual design of parameters (e.g. $R_{max}$ and $R_{min}$ ). It is unclear how the sim-to-real gap will impact these parameters. 

* It seems the baseline methods are POMCP and a variance of the proposed approach with replaced modules. It would be good if the authors can discuss or compare some other related object searching methods.

QUESTIONS:
* L123: the object pose is usually regarded as 6D or SE(3) although using quaternion.

* A concern regarding using growing state space and tree search is that the problem complexity when the object number increase. What is the computational time for the planner with a varying number of objects?

* How does the resolution of the grid representation impact the performance?

LIMITATIONS:
The authors have discussed the limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a full object search pipeline based on a MCTS solver for POMDPs. The object are represented by a octo-grid containing the odds of the object being is the target or an obstacle. A 3D point cloud map of the environment *without any object* is given to the algorithm, allowing for easier object detection and segmentation. The authors propose a more efficient MCTS algorithm for their problem to allow tree re-use when a new state variable is added (i.e. a new object is detected). They test their method in gazebo environments using a simulated fetch robot, and their method performs better than POMCP with nearly always 100% success rate.\n\nSTRENGTHS:\nThis is a very strong paper, systems-wise. The amount of work to create a full object search system such as this, even if just tested in simulation, is significant. The scientific contributions, for me, are 1: the minor (but still valid and significant) modification to POMCP to allow for a growing state-space. 2: the POMDP modeling of the object search problem, which was a very nice textbook-like example of how to model a challenging problem as a POMDP and actually solve it.\n\nWEAKNESSES:\nThe are not many other baselines. There exists many other POMDP solvers than POMCP, and probably other non-MCTS approaches to object search.\n\nI had trouble understanding the terminology \"fake object\", it's just the belief about the target object, which can be wildly inaccurate at first of course.\n\nThe existence of the point cloud map without any object \"is assumed to be available before planning, which is a reasonable and\n216 realistic assumption achieved by mapping the environment prior to planning.\". I disagree, it is not a realistic assumption to have an object-free map prior to doing the task. Imagine deploying this is a home, a cluttered home at that. You would need to clear the whole house before mapping it! Please discuss alternatives for future work.\n\nQUESTIONS:\nWhat limitations do you think would pop-up when deploying in the real-world?\n\nYou touch on this subject, but how would you handle unsuccessful execution of primitives like move or lift? These will happen in the real-world.\n\nLIMITATIONS:\nThere's no dedicated limitations paragraph or section, and it's hard to grasp from the paper. I think the object-free pre-existing map is a significant limitation, but I'd like the authors to do the mental exercise of writing down what would need to be done for this to be deployed in a home on a real-robot, and infer the limitations of their approach.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-10"}