POMDP Planning for Object Search in Partially Unknown Environment
Authors: Yongbo Chen
Hanna Kurniawati
Venue: NeurIPS 2023 poster (2023)
Keywords: robotics
Partially Observable Markov Decision Process (POMDP)
object search
TL;DR: This paper proposes a novel POMDP formulation and solver with a growing state space for object search in a 3D region using mobile robot.

Abstract: Efficiently searching for target objects in complex environments that contain various types of furniture, such as shelves, tables, and beds, is crucial for mobile robots, but it poses significant challenges due to various factors such as localization errors, limited field of view, and visual occlusion. To address this problem, we propose a Partially Observable Markov Decision Process (POMDP) formulation with a growing state space for object search in a 3D region. We solve this POMDP by carefully designing a perception module and developing a planning algorithm, called Growing Partially Observable Monte-Carlo Planning (GPOMCP), based on online Monte-Carlo tree search and belief tree reuse with a novel upper confidence bound. We have demonstrated that belief tree reuse is reasonable and achieves good performance when the belief differences are limited. Additionally, we introduce a guessed target object with an updating grid world to guide the search in the information-less and reward-less cases, like the absence of any detected objects. We tested our approach using Gazebo simulations on four scenarios of target finding in a realistic indoor living environment with the Fetch robot simulator. Compared to the baseline approaches, which are based on POMCP, our results indicate that our approach enables the robot to find the target object with a higher success rate faster while using the same computational requirements.

POMDP Planning for Object Search in Partially Unknown Environment
Yongbo Chen' School of Computing, Australian National University Canberra, ACT, 2601, Australia Yongbo. Chen@anu. edu au
Hanna Kurniawati
School of Computing,
Australian National University Canberra, ACT, 2601, Australia Hanna Hanna.Kurni www.ieanu......... edu, au
Abstract
Efficiently searching for target objects in complex environments that contain various types of furniture, such as shelves, tables, and beds, is crucial for mobile robots, but it poses significant challenges due to various factors such as localization errors, limited field of view, and visual occlusion. To address this problem, we propose a Partially Observable Markov Decision Process (POMDP) formulation with growing state space for object search in a 3D region. We solve this POMDP by carefully designing a perception module and developing a planning algorithm, called Growing Partially Observable Monte-Carlo Planning (GPOMCP), based on online Monte- Carlo tree search and belief tree reuse with novel upper confidence bound. We have demonstrated that belief tree reuse is reasonable and achieves good performance when the belief differences are limited. Additionally, we introduce a guessed target object with an updating grid world to guide the search in the information-less and reward-less cases, like the absence of any detected objects. We tested our approach using Gazebo simulations on four scenarios of target finding in realistic indoor living environment with the Fetch robot simulator. Compared to the baseline approaches, which are based on POMCP, our results indicate that our approach enables the robot to find the target object with a higher success rate faster while using the same computational requirements.
1 Introduction
Finding an object in a partially known and cluttered environment is critical for many robotics tasks. For instance, robot operating in human environments, such as our homes may need to find mugs, glasses, or books that may be placed on top of various furniture like beds, tables, or chairs and partially occluded by other objects. While the positions of the furniture are commonly known, the positions of the smaller objects, such as the mugs and glasses, are not generally known and move very often. Humans can typically find those smaller objects by relying on prior knowledge, assuming the object they are searching for has been seen before [ and removing occlusions as necessary. This kind of target objects-oriented problems is also called "Mechanical Search" problem 7
This paper proposes a planning method based on a novel solver, called Growing Partially Observable Monte-Carlo Planning (GPOMCP), that enables mobile manipulators to do just that: Finding target object that might be occluded by other objects in partially unknown environment by moving to positions with better visibility or by removing occlusion when necessary.
"Corresponding author
37th Conference Neural Information Processing Systems (NeurIPS 2023).

---

We assume the robot is operating in home environment that consists of multiple rooms and is provided with 3D point cloud and 2D occupancy grid maps of the environment and a set of the target object's s photos. These maps contain the furniture but do not contain information about the objects on top of the furniture. GPOMCP computes an efficient search policy that utilizes onboard sensors and its ability to remove objects within designated search region to find the target object even if it is partially or completely occluded while accounting for localization and perception errors. To handle the partial observability nature of the problem, GPOMCP is based on the Partially Observable Markov Decision Process (POMDP). A unique feature of GPOMCP is it can efficiently modify policies when the number of state variables increases. This feature allows GPOMCP to model the environment incrementally, starting from environment map with no objects top of the furniture to more faithful map where all the objects on top of the furniture have been included in the map and everything in-between, as needed until the target object is found.
In addition to the above solver, we propose POMDP model for the abject-finding problem that exploits GPOMCP's ability to modify policies as the number of state variables increases. To this end, the model maintains a set of state variables that represents information about the robot's i guess of the target object, which we refer to as guessed target object. A belief over this set of state variables represents the robot's uncertainty over the characteristics of the target object, such as the pose, whether the object can be moved or not, etc. As new observations are perceived, the robot refines its belief about the guessed target object, and eventually, the belief has high chance to converge to the correct target object positions based on suitable parameter settings. Experimental results on four different scenarios using Fetch robot simulator in Gazebo indicate that the above model and GPOMCP significantly outperforms POMCP 20]
Background
A POMDP is a general mathematical formulation for sequential decision-making problem where the state is not fully observable by the agent. Formally, it is defined as an 8-tuple < S, A, O, T, Z, R, bo, y>, where the state space S denotes the set of all possible states of the robot and the environment; the action space A denotes the set of all actions the robot can perform; the observation space O means the set of all observations the robot can perceive. At each time step, a POMDP agent is at a state s E S, but this state is generally not known exactly by the agent and therefore the POMDP agent uses belief b ba distribution over the state space- to represent this state uncertainty. The set of all beliefs is referred to as the belief space and denoted as B. At each time step, the agent performs an action E A, which may result in the agent arriving at a new state. However, the effect of actions is uncertain and is represented by a conditional probability function, namely the transition function T(s, a, s') = Pr(s'Is, a). Once the agent arrives at a new state (which may not be fully known), the agent perceives perceives an observation observation function Z(s', a, o) Pr(ols', a), which is a conditional probability function that represents the observation the robot may perceive after performing action a E A in state s' E S. Together with perceiving an observation, the agent also receives an immediate reward, governed by the reward function R(s, a, s') E R, which can be parameterized by state, an action, a state-action pair, or a tuple of state, action, and subsequent state. The estimate on the initial state is called the initial belief and denoted as by. Last but not least, y is a discount factor following 0 < y < 1 113. The goal of solving a POMDP problem is to find an optimal policy II'(b)= = argmax,c91(0 a) for belief b E B, where the Q(b, va-uul is the value of executing action when the agent is at belief b and continuing optimally afterwards.
O, governed by the
POMDP solvers can be broadly classified into two main classes based on their real-time performance: offline and online solvers. While offline solvers can achieve optimal solutions for small-scale problems, such as the boint-based POMDP algorithm 19 14, they require too much memory for real-world applications. Most online POMDP solvers commonly explore the sparse belief sampling with a collection of particles and perform a Monte-Card Carlo tree search (MCTS) to grow the belief tree within the limited planning time. State-of-the online methods include Partially Observable Monte Carlo Planning (POMCP) 20), Adaptive Belief
2

---

Tree (ABT) 15 Determinized Sparse Partially Observable Tree (DESPOT) [21], Partially Observable Monte Carlo Planning with Observation Widening (POMCPOW) [22], and Adaptive Discretization using Voronoi Trees (ADVT) [9] methods. Among them, POMCP and POMCPOW methods rely on the Monte Carlo backup, while others use the Bellman backup. Although most of these methods are designed to explore belief nodes more deeply in continuous or large action and observation spaces, continuous state, action, and observation spaces continue to pose significant challenge.
A substantial category of mechanical search methodologies is closely intertwined with ad- vancements in learning technologies, including deep reinforcement learning (RL) [12,, deep Q-leamnng (25], receding horizon planning guided by neural network-based distribution (2], deep-geometric inference systems [0]. . For example, in [12], the authors use a deep RL procedure that combines teacher-aides exploration, a critic with privileged information, and mid-level representations to uncover target object occluded by a heap of unknown objects. Except for the learning-basee methods, some other methods exploit the classical technologies, like the adaptive horizon exploration algorithm g and mixed-integer program (MIP) 3 A shared characteristic among these methods is their specialized application in specific environments, primarily centered around scenarios involving shelves and boxes.
Some other works on object search are solved using POMDP planning approaches, such as a recent MCTS-based mechanical search method that relies on a 1D occupancy distribution for the current camera observation in cluttered shelves. [ . The Object-Oriented POMDP (O0-POMDP) [24], which is very popular, considers the state and observation spaces to be factored by set of и objects, where each belongs to class with a set of attributes, and the beliefs about different objects are independent for the small computational reason. Many previous works also limit the workspace to a 2D plane with simple transition and observation models. For example, Aydemir et al. proposed 2D POMDP formulation for the object search in a room by calculating candidate viewpoints Mi The object search task has been extended to 3D with an object size-friende
property using a multi-resolution planning algorithm 288). We follow the same Object-Oriented idea because of the good applicability of the real-world application but go forward one step to consider the occlusion relationship for the belief update between different objects in a complicated 3D environment, including some challenging cases with partial and fully occluded objects.
3 POMDP formulation for object search
Our focused object search task is to find movable target object lobjiarl in several clustered workspaces that contain an unknown number n > 1 of obstacle objects (objas] with unknown locations. To facilitate robot navigation, we have access to point cloud and occupancy grid maps representing the planning environment, including furniture. However, we do not possess the point clouds of the obstacle objects (objoesl or the target object jar. Therefore, during the execution process, we rely on on-board sensors to detect and identify these objects (objare U jurt.). Since we gather information about the objects using real-world sensors, they are not mutually independent in our scenario and may influence each other. This differs from the assumption made in O0-POMDP.
State space S. A state s = (s,, 8 Bpl,..., , ) E S is represented as two parts including robot configuration and object-oriented factors, where 8, is the robot state and sy denote the object state of the i-th object. A robot state is written as 8, = (p, h, Ip, 4)), where p is the 6D robot base pose, 1, is the lift height, 1p and I, are the pan and tilt angles of the robot head. At the beginning stage, the robot may not get into the workspace, so no detected object is available. In order to guide the robot toward meaningful actions, we introduce a guessed target object s (p,s, Sir sy S 9e9, Mæ, 11,)) E RIx20 where P = (p),, P()) is the 6D object pose (3D position ph and 3D rotations represented by quaternion p²)); % sy and s mean the sizes along the principal 3D axis of the object; 9r- E R denotes grid odds values to identify whether this object is a target or obstacle object, which will be updated using the scores obtained from image matching. my E IR is the odds value to evaluate the move-ability of the guessed target object; u É (-2, - 1, 0, 1) is the int front value to claim whether this object is updating or not. Specifically, -2 means this object is
3

---

beod ont
Sep
Poom
belddirr
Vieving Are
Supg
6
MCTS the to sa ax
bo beriiiruna and
FOV
Robot
Mia ep Nor fo
eltorration
Unknowe usdae
1000 x 2
Orianed
hrazsing hes.
with gish
Obiect Obsect updiee
grid sydee
ViNicaaa chosal
Flhesoaal simal
Figure 1: Grid update in one frame
Figure 2: Main steps for GPOMCP
removed after performing the removing action; -1 denotes that the object is still updating without performing declaring and removing actions, 0 and 1 means this object has been declared as an obstacle object and target object. The belief of the position p", of the guessed target object is saved in grid world G; generated by all workspaces ' using many odds values Odd(G,) with given resolution. Inside each grid, the position is sampled uniformly and its probability is obtained by its odds value. The odds values are updated based on the field of view (FOV) of the camera using real-world measurement in the excursion process, which is similar to the update of the occupancy grid map in Simultaneous Localization and Mapping (SLAM) 4 following:
log (ld()))z1a) log (k.))) + log Odd((GlZzx=)) Odd(G)/Z1a)) P(G/
(1)
where P(G (lz::) and P(-G means the probability of the object belonging to and not belonging to this grid based on multiple observations Zιz; Odd(G is the corresponding odd value. In the belief tree search, we do not update the odds values Odd(Gy) and it will be updated after the real-world excursion. In the planning stage, this grid world is just used to sample the potential position of the guessed target object in the root node. The guessed target object is special with a constant orientation (set as (0, O, 0, 1)), size (0.1, 0.1, 0.1), and move-ability value (set 100, movable). The grid values g and the declared value u, are update-able in the belief tree search but need to be reinitialized as the given value after each excursion. Fig T shows the scenario about the grid world for the guessed target object. In fact, the guessed target object variable within the state vector saves the belief about the position of the target object. It not only encodes the probability information derived from the grid world with odds values but also facilitates efficient planning within Monte Carlo tree search with low computational complexity, acceptable memory cost, and coding friendliness. More explanations to the guessed target object are shown in the supplementary materials.
The other object Spi, i # 0 E R follows the same format and all parameters should be updated both in belief tree search and real-world excursion. is noted that the pose of the other objects soi is estimated based on the point cloud which is independent of the grid world Gy. The newly detected objects after each real-world excursion will be appended to the state vector and form growing state space (s, 8asa) E S x Sait.
Action space A. Our method is task-level planning with many primitive actions instead of control-level planning. The key purpose of setting these actions is to change the FOV of the robot camera to cover the workspace and further operate objects. The action space consists of 5 types of primitives: Move(p', (m) means to move the robot base to given target pose p"; 44) denotes the actions to change the angles of the robot head pan (left/right) and tilt (up/down) to I, for the pan and I, for the tilt; Lift(l,) represents to move the robot height to l;; Declaring(u,)) means to declare the updating object (u, = -1) to be the target object (n, = 1) or the obstacle object (u, 0); Removing(i, w.) is the action to remove i-th object using robot manipulator. All actions are assumed to have 100% success rate for simplicity
'Even in the gazebo Fetch simulator, it is hard ensure that each action is successfully performed, especially for removing actions. In order to guarantee the 100% success rate, after the robot's common control level operation, we call the gazebo server "set_ seaet_odee_ssteee to physically set the model state with some Gaussian noises, which will be improved in future work.

---

Observation space O. The main focused observation is a FOV viewing frustum V projected from the robot head camera. Because we view each object by an oriented box with 8 grids, for each observation, we assume to update at most the 4-nearest grids with noise negative log-odds value -C, + 17, a noise positive log-odds value c, + n, a noise near-zero log-odds value 77, where 77 follows the Gaussian distribution. Following the blind spot caused by the other objects, sometimes, the robot camera can not detect all 4-nearest grids. We set the observation space to be discrete. By fusing all n, detected objects, we can get the observation to be ([i,..., f, (o(i), ... oi)), Ili,...., .il = Hor where [i,... ... j] records the observed object with at least one updated grid and (o(i), ,(i)) are their updated log-odds value.
Transition function T. If the actions Movelp', wm), Head((p, 4), and Lift(h,) are applied to the robot, the camera location and orientation will change with some noises. The camera motion follows the rigid transformation and the static structure information of the applied robot is read from tf message. The declaring action Declaring(u,) will change the robot status u based on the comparison between the mean value of the minimal 2 log-odds of 8 grid values and the thresholds C2 for the obstacle object and C₁ for the target object. If no removing action is performed to the i-th object, the i-th object is static and rigid. The removing action Removing(i, as,) on i-th object will move the i-th object outside the workspace and set the status My to -2, which means they will not block the observation of the other objects.
Observation function Z. The coordinates of 8 grids of the i-th object can be computed using center point p', and the orientation p", of the object pose and its size s s Sy estimated by point cloud. After performing one action, the visible grids of different objects belonging to the 4 nearest grids will be updated. The updated value will be decided based on the mean log-odds value of all grids. If the original mean log-odds value is larger than a positive threshold Vp > 0, the updated value will be set to be positive co + 17 If is smaller than negative threshold v, < O, the updated value will be set to be negative -cg + n. Otherwise, the value between the positive and negative thresholds will generate near-zero 17. So as to encourage the exploration of the robot, the initial grid values ge" of the guessed target object are reinitialized to be larger than the positive threshold vp > 0 after each real-world excursion.
Reward function R. Our expected sequential actions involve the robot moving to the workspace for observation, continuously changing its viewpoints gather more measure- ments, updating odds values in 8 grids, then declaring the updating object to be the obstacle or target object, and finally removing the obstacle objects for the more visible area or directly the target object to complete the whole task. Hence, if the whole task is completed success- fully with the removal of the target object, the robot will receive large positive reward Rmar > 0. As the prerequisite action, correctly identifying the obstacle and target objects yields relatively large rewards (R,, and R., respectively), where Rmar > RR > R. > All actions incur negative step cost (Rmin < <0). The cost of the removal action Removing(i wr) is set as 2Rmin due to its complexity. Illegal actions incur a large negative penalty (Rar < 0), including collisions with occupied grids in the occupancy grid map, attempting to remove immovable objects, declaring an object that doesn't meet the condition, removing updating objects without declaration, and trying to remove objects that have already been removed.
4 Action execution and perception with on-board sensors
Our perception part is carefully explored with many useful outputs, like estimated object pose, estimated object size, object move-ability, and object detector, fusing both point cloud data and the image data without using manual quick response (QR) codes. Our framework is suitable for all mobile robots with 2D Lidar and RGBD cameras, but specifically, we mainly consider the Fetch robot here. Move base actions are implemented using a ROS interface move base and the interaction with the AMCL-based navigation stack [18]. The robot lift height and head joints are controlled by following the joint-space trajectories on group of joints based on a ROS interface ros ros_ccoosie control 5 Removing action is to pick up the object and place it in some given areas outside the workspace using the ROS tool moveit [6].
To simplify the scenario and reduce the navigation effort, we assume the availability of pre-b point cloud map M of the robot's 'ss environment, including furniture and known

---

objects, which can be created beforehand through environment mapping and reused during planning. Then, the 3D point cloud F, in j-th frame from the depth camera is matched with the map M by iterative closest point (ICP) [26]. The 3D ICP is initialized by matching 2D Lidar points with the 2D occupancy grid map. As the frame number increases, the point clouds of objects and environment P; M fu UF.U... ..) are becoming more and more complete. After removing the original map M; P/M, the point clouds for n newly
detected objects (oo.... 0,-1l E M" are extracted by point cloud segmentation using the Euclidean cluster extraction method. For estimating object pose and size, we obtain the minimum oriented bounding box for each object using principal component analysis. The point cloud segmentation is applied followed by data association based on the Mahalanobis distance of centroid points and point-wise mean distance. Additional details can be found in the supplementary materials.
So as to complete the given object detection task using several given images and some semantic words (optional), based on the previous point cloud segmentation, we perform the point cloud re-projection, sub-image fusion, YOLO detection, and SIFT matching [17] to compute the probability of object detection. We also would like to manipulate the objects in the workspace to free some FOV, so we estimate the move-ability mome of each object by the learning-based Grasp Pose Detection (GPD) toolbox 223,, the k-means clustering algorithm, and the ROS moveit toolbox. The details of the implementation are presented in supplementary materials. In the planning stage, for each particle, we will randomly sample value for this object and compare it with the move-ability Tmove to identify its move-ability.
5 Belief tree reuse for online POMDP solver with growing state space
Here, we present POMDP solver, GPOMCP, reusing the previous belief tree when the state space of the problem is growing. In this kind of POMDP problem, the observation obtained by real measurement causes the unpredicted new state, which means that the whole belief tree should be cut and totally new belief tree should be computed by sampling. It is time-consuming. As shown in Fig. 2 we solve the problem by following steps:
Step 1: When state space grows from s E to (s, aada) E S x Sadd, in the old belief tree T, we can pick out the branches corresponding to old sub-state s and set it as the root of new tree Tss).
This new tree (s) has many particles with history h. For each particle, we can update the future state s' by the added state vector & corresponding to new objects based on the transition function T(smid, a, s, a'a)) and observation function
Step 2:
7 mid a, Onti)) 3 It is noted that, in the state updating, we should consider the effect s' - 8'Sx from the old state vector s corresponding to the old object. So we can get the updated histories about some or all particles within the given time limitation.
Step 3: Then, starting from the root, we can grow the belief tree (s(s) using these updated particles P is noted that, in building the new belief tree (s),,, the new observation nodes with newly detected objects may be introduced, which means that the generated belief tree (s))s may be not sub-tree of the old belief tree T.
Step 4: Then, we perform the Monte Carlo tree search (MCTS) for all branches including the new actions and new observations. The action selec- tion strategy follows the upper confidence bound (UCB) and the belief distance Co Eoso Z(s a, o' (')sl'
s s'aa) :s' - (' b((s', s'ass) s' 2² s' slan V* a) +
TogN(h Níb,jj where Cs and CN are coefficients set by users.
CN
We know that the current optimal action should be selected following equation Eq. 2 maxaes (Eenes R(amau a, s'gnn ((@new)) + Loso Exi.es es Z(s a, a, (((mavv a, V アो 22 s((m)))
(2)
'It is noted that because our considered actions a and a₂ just perform on the old objects, the reward function does not need to be updated. Our reward function follows R((s, amit), as (s', 8 ((mu))
R(s, a2 a,,s),, because the old action aj and ag just operate one object each step. *Here, the belief of the old state does not consider the effect from the new object s''s + s'.

---

where Snew = (s, 2) and nnm = (s', s ) Considering two ways (GPOMCP and the direct resampling way), in the sub-tree of the new belief tree, the only difference is from the last terms V'(b( s' 2 s 'sa)) (correct by building tree) and V*(6($ : s' ( (GPOMCP,
without considering partial effect). Then, we have the following Theorems:
are considering the umweighted particles for approximating the extended full-correct belief b(s' :' 2 (aaa)) 3 I (ss'ew new ss 2 s'ais) and the reused approximated belief b(8 8' -8800 pds - Li=1 [1 s s'' - 8 mas) Assuming the reward function is Borel measurable and bounded, denoted Rmax = [| the belief L1 distance is limited in [ib(s'eew s 2 - s saaa)l I 6, and the optimal action selected by building the whole tree is same as the one selected by approximated value max, V(b(snew), a) = maxa ((b(s a) = a', the optimal value function of POMDP problem using our method (b(((mmew) and the optimal value function using the direct resampling way Vb((((new) will satisfy the following bounding equation:
Theorem 1. are
Y.mux
(V"((((new)) ("((( M1h S
1 y 8
(3)
Proof. Please refer to the supplementary material.
This theorem indicates that the reuse technology works well with a performance bound when the reused belief tree shows good approximations to the exact accurate belief tree. To enhance the estimation of the upper bound, we introduce a term C2 Eoco Z(s a, o ())((((s
s' + s (aaa)
b(s
new s' 2 s' Si to the UCB strategy when reusing the previous belief tree.
It is noted that Theorem is correct only under the assumption maxa V(b(Snew), a) max, (b(spr))) a) = a', which means that there are some bounds for the performance guarantee shown in Theorem D using the reuse approximation.
Discussion 1. In fact, our belief tree reuse
be considered as the wrong estimation A of the action and measurement node values in the correct belief tree. When the distance between the value function for the optimal action V(b(swew), a') and the value function for any sub-optimal action V(b(Snew), ( is relatively large satisfying V(b(Swaw), a') ) V(b(Bwe)), aaab) >> b, which means that the estimation error & will not affect the final solution and the convergence of our solver follows the method directly rebuilding the whole tree.
Following Discussion 1 we would like to get the exact performance bound to ensure that the obtained solution remains the same as the optimal solution using the fully-rebuilt belief tree. Firstly, we have the following corollary:
Corollary 1. With the same assumption as Theorem 7 given the action a, the optimal action-value function of POMDP problem using our method V* (wew), a) and the optimal value function using the direct resampling way ('(((new), a) will satisfy the following bounding equation:
/Rmmx 8
IIV (b(snew), a) (b(Ssee))) a)lh S 1-y
(4)
Proof. Just follows the similar proof process shown in Theorem so we ignore it here.
Corollary 2. With the same assumption Theorem I if the distance between the value function for the optimal action V(b(sneu) a²) and the value function for any sub-optimal action V(b(snew) a (uub is larger than 2yRaus 5, the optimal action obtained by the tree reuse approximation way will be same as 1-y
the optimal action using the direct resampling way max, V(b(snew), a) = .max. (b((sae)), a) = a".
Proof. Please refer to the supplementary material.
The pseudocode Algorithms of the GPOMCP solver are shown in supplementary materials.
Our proposed method follows the common procedure with four alternating stages, planning, execution, obtaining observation, and filtering. The optimal action for the current belief
7

---

is selected based on GPOMCP, Inside the POMDP solver, the Monte Carlo update and revised UCB strategy considering the belief error proposed in Section 5 are applied.
8
Simulations
In this section, we showcase our method's effectiveness via simulations conducted using C++ and Python codes within the Fetch robot simulator. The project operates on multiple Ubuntu 18.04 desktops equipped with an Intel Core 17-13700k processor, running exclusively on CPU. We evaluate our method using various object settings in different scenarios.
Problem settings We implement our approach in a realistic gazebo living environment (Fig. 3a). The 3D point cloud map and 2D occupancy grid map, generated using [16], of this environment with furniture are available before the object searching task (Fig. 3b). There are multiple workspaces on the surfaces of the large furniture, like tables, beds, and desks, in this planning environment. Some unknown objects including a target object are set inside the different workspaces sometimes with an occlusion relationship. Four photos of the target object from different viewing directions are available for the object detector.
(a) Gazebo environment
(b) Corresponding point cloud and occupancy grid maps Figure 3: The planning environment
We manually set several candidate poses that can observe partial workspace for the move base action Move(p', wm). The candidate robot head motion (p, [,) are set within
I, E -- 0, 1T, 4 E 1-0.5, 0, 0.5), and the lift motion I is limited in 4, |0.0, 0.4). The FOV of the Fetch camera has a 60-degree horizontal view angle with a 480/600 height-width ratio. In the transition and observation function, using these actions, we add the nearest (0.5 meters to the camera center) and farthest planes (1.7 meters to the camera center) to the FOV and form hexahedron. This hexahedron-shape motion and configuration change. As mentioned in Section 3 each object is represented as 8 grids based on its optimal oriented box, and the occlusion computed using perspective projection of the cube vertices. The grid size of the grid world is set as 2 cm. A discussion about different grid sizes is shown in the supplementary materials. The log-odds value of these grids is initially set as unknown satisfying log 1-0,5 0. These odds values will be updated after each real-world observation and then used to sample the potential position of the guessed target object. The planning time is limited to 60 s/step, the discounted factor y 0.9, the threshold to the object declaration C/ log(8/2) and C. log(2/8), the thresholds for grid updating Vp 0.1 and 001, the re-initialized grid values of the guessed target object are set 0.2, the reward values R = 10², R 5 10 R. =0*, Rmin 11, and Ra = -103.
FOV will move based on the robot's
Comparison results Because a lot of existing object search approaches are based on the pure POMCP method with different problem settings, eg. [27 24,, we compare our approach using Bellman (Ours_b or Monte Carlo update (Ours_m m) with the POMCP method without using/using the guessed target object (denoted as POMCP, POMCP _f) The comparison is implemented based on 4 different scenarios with different object numbers, including LOOSE, (4 objects in workspace), LOOSE, (6 objects in workspaces), Hidden, (7 objects
'Here, we use the Monte Carlo update because the visual measurement model in object search not well predictable. The robot will frequently receive unexpected measurements, which are not in the belief tree. It breaks the advantage of the Bellman update (explore deeper) of Adaptive Belief Tree (ABT) solver 15). We think that, in other applications, the Bellman update should be better.
8

---

in 1 workspace), and Covered, (7 objects in 1 workspace), as shown in Fig. 4 The target object is the blue snack box shown in the blue circle. Only the black headset box and the dish rack are not movable, and the others are all movable. Green arrows show the candidate robot poses. "Tick" means the target objects can be observed easily without changing the robot configuration. "Half tick" means the target can be observed if we change the robot configuration. "Cross" means the target object is fully hidden by the obstacle objects and cannot be seen from that pose even changing the robot configuration, including the lift of the robot, the obstacle objects are not removed before. For each scenario, 20 trials are conducted and statistical results are reported. More resuls with different problem settings are shown in the supplementary materials.
(a) LOOSE,
(b) LOOSE,
(c) Hidden;
(d) Covered
Figure 4: The planning environment
The obtained 95% confidence interval of the discounted cumulative reward, the steps, and the success rate are reported in Table [1] For clarity, we present one sequence of COMDP-generaleee actions (Fig- 5 which to complete the Covered, scenario by moving its pose, changing its configuration, declaring, and removing the target object within 14 steps. The robot removes the obstacles (objects 3 and 4) to free the visual occlusion and finally declares and removes the target objects. The sub-images in the red boxes show the probability of the grid world for the guessed target objects, which is updated using Eq. [1 and the ones in yellow boxes show the detected camera images. We can find that, with the growth of the robot's actions, the probability of most grids far away from the target object decreases quickly and the ones near the target object increase a lot, which guides the robot to make more effort to observe the area with the target object.
discounted cumulative reward 57691.77
Move base to Reward 1
Move base to pase
Move head to T 6)
Declare object 4 (ubstucle) Reward: 9999
Remove object
Move head
Remove object Reward:
(obstucle) Declare object (obstacle)
Reward: -1
Reward: 9999
Reward: 9999
Move base pose
Reward:
Doclare object (target)
Move hend to (-x/12, Declare object Reward: 49999
Remove object
Reward: -101
Reward:-1 -1
Reward: 9999g
Figure 5: Actions complete the Covered, scenario with 7 objects in steps.
Our method, utilizing Monte Carlo update, demonstrates superior performance in complex object search scenarios such cases like Hidden; and Covered, where predicting camera observations challenging. In simpler cases like Loose,, our method performs comparably to the POMCP method when using our introduced guessed target object. The guessed ta target object proves to be beneficial in activating the robot in situations with limited information. By incorporating our approximated belief reuse, we retain more useful branches after receiving observations, leading to further improvement in overall performance, The concept of the guessed target object aligns with our natural tendency to assume the presence of certain objects in specific locations and adjust our estimations based on visual observations.
Limitation Our main limitations mainly stem from the errors and failures from perception and navigation components, not our focused planning part. The first limitation is the

---

Table 1: Discounted cumulative reward, steps, and successful rate (within 50 steps)
Scenarios
Loose;
Loosez
POMCP
66003.7 + 16269.5 13.5
6.8|
90%
26175.8 8220.3 29.2 7.7 65%
78629.6 8472.1 7.2 1 0.7|
100%
38462.6 11221.2 23.2 6.6 995
Oursp
84701.5 5546.8 6.9 + 0.6 100%
33265.5 = 10085.3 22.2 = 4.2 100%
Oursm
4777
100%
51574.1 17930.0
17930.9
17.4 +: 2.9 100%
Scenarios
Hidden,
Covered
POMCP
40958.7 + 8070.9 20.1 +: 2.7| 100%
17700.2 3616.6 38.3 :: 5.1 65%
POMCP,
45815.2 + 7260.9
2 100%
31506.9 = 6249.7 32.3 # 6.0 80%
Ours,
45038.6 1 7631.0 18.2 +: 2.7| 100%
26130.6 3368.7 30.2 :: 4.6 | 90%
Ours
55574.8 + 6225.3
15.3 + 2.0 100%
34397.8 + 7381.6 26.7 = 4.8 95%
pre-existing map M which may not be available for a real robot but can be solved by introducing advanced sementic SLAM method. Achieving 100% primitive action success in reality is impractical, affecting our method's success rate. Another limitation is that our framework is based on point cloud segmentation to identify different objects. It may generate incorrect bounding boxes for objects with extensive contact areas. This misinterpretation can result in erroneous data association, impacting belief updates and removing actions. Furthermore, YOLO and SIFT object detection techniques may struggle in scenarios with limited SIFT features and YOLOv5 toolbox coverage, especially in low-light environments. However, we believe that addressing these limitations is possible by incorporating more advanced perception and navigation methods.
7 Conclusion and future work
We propose a novel object-level POMDP formulation with growing state space, a guessed target object, and its grid world for object search in 3D with real-world sensors and solve it using a novel GPOMCP method based on Monte-Carlo tree search and belief tree reuse. We demonstrate the effectiveness of our approximation in cases with similar beliefs. Our method achieves object search with only maps, photos, and onboard sensors without relying on additional object information, like geometric models, making it highly practical for real- world robot applications. By utilizing the guessed target object, our method can successfully locate fully hidden objects even without prior information, by removing obstacles-a challenge for existing approaches. Our future directions involve dropping prior information, such as pre-built point cloud map M, and exploring continuous action domains. Through our experiments, we showcase the efficiency of our method, and we anticipate that our results will inspire further advancements in this field.
Acknowledgements
This research is supported by the ANU Futures Scheme.
References
[1] Alper Aydemir, Andrzej Pronobis, Moritz Göbelbecker, and Patric Jensfelt. Active visual object search in unknown environments using uncertain semantics. IEEE Transactions on Robotics, 1944)998610002, 2013.
[2] Wissam Bejjani, Wisdom C Agboh, Mehmet R Dogar, and Matteo Leonetti. Occlusion-aware search for object retrieval in clutter. In 2021 TEEF/RS] International Conference Intelligent Robots and Systems (IROS), pages 6678-4685. IEEE, 2021.
[3] Lawrence Yunliang Chen, Huang Huang, Michael Danielczuk, Jeffrey Ichnowski, and Ken Goldberg. Optimal shelf arrangement to minimize robot retrieval time. In 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), pages 993-1000. IEEE, 2022.
[4] Yongbo Chen, Shoudong Huang, and Robert Fitch. Active slam for mobile robots with area coverage and obstacle avoidance. IEEE/ASME Transactions on Mechatronics, 25(3):1182-1192 2020.
10

---

[5] Sachin Chitta, Eitan Marder-Eppstein, Wim Meeussen, Vijay Pradeep, Adolfo Ro- driguez Isouroukdissinn, Jonathan Bohren, David Coleman, Bence Magyar, Gennaro Raiola, Mathias Lüdtke, and Enrique Fernández Perdomo. ros control: A generic and simple control framework for ros. The Journal of Open Source Software, 2017.
[6] David Coleman, Ioan Sucan, Sachin Chitta, and Nikolaus Correll. Reducing the barrier to entry of complex robotic software: a moveit! case study. arxiv preprint drXio::104..785, 2014.
[7] Michael Danielczuk, Andrey Kurenkov, Ashwin Balakrishna, Matthew Matl, David Wang, Roberto Martin-Martin, Animesh Garg, Silvio Savarese, and Ken Goldberg. Mechanical search: Multi-step retrieval of target object occluded by clutter. In 2019 International Conference on Robotics and Automation (ICRA), pages 1614-162) IEEE, 2019.
[8] Megha Gupta, Thomas Rühr, Michael Beetz, and Gaurav S Sukhatme. Interactive environment exploration in clutter. In 2013 [EEF/RS] International Conference on Intelligent Robots and Systems, pages 5265-5272. IEEE, 2013.
[9] Marcus Hoerger, Hanna Kurniawati, Dirk Kroese, and Nan Ye. Adaptive discretization using voronoi trees for continuous- cuttiuuouss-aaiiinnn pomdps. In Algorithmicc Foundations of Robolics XV: Proceedings of the Fifteenth Workshop on the Algorithmic Foundations of Robotics, pages 170-187. Springer, 2022.
[10] Huang Huang, Marcus Dominguez-Kuhnt Vishal Satish, Michael Danielczuk, Kate Sanders, Jeffrey Ichnowski, Andrew Lee, Anelia Angelova, Vincent Vanhoucke, and Ken Goldberg. Mechanical search shelves using lateral access x-ray. In 2021 IEEF/RS] International Conference Intelligent Robots and Systems (IROS), pages 1045-2056 IEEE, 2021.
[11] Huang Huang, Letian Fu, Michael Danielczuk, Chung Min Kim, Zachary Tam, Jeffrey Ichnowski, Anelia Angelova, Brian Ichter, and Ken Goldberg. Mechanical search on shelves with efficient stacking and destacking of objects. In The International Symposium of Robotics Research, pages 205-221. Springer, 2022.
[12] Andrey Kurenkov, Joseph Taglic, Rohun Kulkarni, Marcus Dominguez- Animesh Garg, Roberto Martin-Martin and Silvio Savarese. Visuomotor mechanical search: Learning to retrieve target objects in clutter. In 2020 [EEE/RS] Infernational Conference on Intelligent Robots and Systems (IROS), pages 8408-8414. IEEE, 2020.
[13] Hanna Kurniawati. Partially observable markov decision processes and robotics. Annual Review of Control, Rohotics, and Autonomous Systems, 5:253-277, 2022.
[14] Hanna Kurniawati, David Hsu, and Wee Sun Lee. Sarsop: Efficient point-based pomdp planning by approximating optimally reachable belief spaces. In Robotics: Science and systems, volume 2008. Citeseer, 2008
[15] Hanna Kurniawati and Vinay Yadav. An online pomdp solver for uncertainty planning in dynamic environment. In Robotics Research: The 16th International Symposium ISRR, pages 611-629. Springer, 2016.
[16] Mathieu Labbé and François Michaud. Rtab-map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term terg-termm online operation. Journal of Field Robotics, 36(2):416-446, 2019.
[17] David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60:91-110, 2004.
[18] Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Foote,Jrrmmy Leibs, Eric Berger, Rob Wheeler, and Andrew Ng. ROS Navigation stack. http://wii..roo.nnnnnviigtttt 2009. Accessed: May 4, 2023.
[19] Guy Shani, Joelle Pineau, and Robert Kaplow. A survey of point-based pomdp solvers. Autonoonus Agents and Multi-Agen Ageett Systems, 27:1-51, 2013.
[20] David Silver and Joel Veness. Monte-carlo planning in large pomdps. Advances in neural information processing systems, 23, 2010.
[21] Adhiraj Somani, Nan Ye, David Hsu, and Wee Sun Lee. Despot: Online pomdp planning with regularization. Advances in neural information processing systems, 26, 2013.
11

---

[22] Zachary Sunberg and Mykel Kochenderfer. Online algorithms for pomdps with continuous state, action, and observation spaces. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 28, pages 259-263. 2018.
[23] Andreas Ten Pas, Marcus Gualtieri, Kate Saenko, and Robert Platt. Grasp pose detection in point clouds. The International Journal of Robolics Research, 36(15-14):11555147, 2017.
[24] Arthur Wandzel, Yoonscon Oh, Michael Fishman, Nishanth Kumar, Lawson LS Wong, and Stefanie Tellex. Multi-object search using object-oriented pomdps. 2019 International Conference on Robotics and Automation (ICRA), pages 7194-7200. JEEE, 2019.
[25] Yang Yang, Hengyue Liang, and Changhyun Choi. A deep learning approach to grasping the invisible. IEEE Robotics and Automation Letters, 5(2):2232 2239, 2020.
[26] Juyong Zhang, Yuxin Yao, and Bailin Deng. Fast and robust iterative closest point. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3450-3466 2021.
[27] Kaiyu Zheng, Anirudha Paul, and Stefanie Tellex. system for generalized 3d multi-object search. arXio preprint arXiv:2303.03178, 2023.
[28] Kaiyu Zheng, Yoonchang Sung, George Konidaris, and Stefanie Tellex. Multi-resolution pomdp planning for multi-object search in 3d. In 2021 IEEE/RS] International Conference on Intelligent Robots and Systems (IROS), pages 2022-2029, IEEE, 2021.
12


Reviews:
[{"review_id": "LyB9mlHTZm", "reviewer": "Reviewer_uTWr", "summary": "The paper presents a full object search pipeline based on a MCTS solver for POMDPs. The object are represented by a octo-grid containing the odds of the object being is the target or an obstacle. A 3D point cloud map of the environment *without any object* is given to the algorithm, allowing for easier object detection and segmentation. The authors propose a more efficient MCTS algorithm for their problem to allow tree re-use when a new state variable is added (i.e. a new object is detected). They test their method in gazebo environments using a simulated fetch robot, and their method performs better than POMCP with nearly always 100% success rate.", "questions": "What limitations do you think would pop-up when deploying in the real-world?\n\nYou touch on this subject, but how would you handle unsuccessful execution of primitives like move or lift? These will happen in the real-world.", "limitations": "There's no dedicated limitations paragraph or section, and it's hard to grasp from the paper. I think the object-free pre-existing map is a significant limitation, but I'd like the authors to do the mental exercise of writing down what would need to be done for this to be deployed in a home on a real-robot, and infer the limitations of their approach.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "This is a very strong paper, systems-wise. The amount of work to create a full object search system such as this, even if just tested in simulation, is significant. The scientific contributions, for me, are 1: the minor (but still valid and significant) modification to POMCP to allow for a growing state-space. 2: the POMDP modeling of the object search problem, which was a very nice textbook-like example of how to model a challenging problem as a POMDP and actually solve it.", "weaknesses": "The are not many other baselines. There exists many other POMDP solvers than POMCP, and probably other non-MCTS approaches to object search.\n\nI had trouble understanding the terminology \"fake object\", it's just the belief about the target object, which can be wildly inaccurate at first of course.\n\nThe existence of the point cloud map without any object \"is assumed to be available before planning, which is a reasonable and\n216 realistic assumption achieved by mapping the environment prior to planning.\". I disagree, it is not a realistic assumption to have an object-free map prior to doing the task. Imagine deploying this is a home, a cluttered home at that. You would need to clear the whole house before mapping it! Please discuss alternatives for future work. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "$\\textbf{Response about weakness}$:\n\nW1. The are not many other baselines......\n\n$\\textbf{Answer}$: Thank you for your valuable comments. We appreciate the suggestion to consider more baselines for comparison. Please refer to the “Baseline methods” section of the global comments file for more details.\n\nW2. I had trouble understanding the terminology......\n\n$\\textbf{Answer}$: Thanks for pointing this out. Please refer to the “Fake object” section of the global comments file for more details.\n\nW3. The existence of the point cloud map without any object......\n\n$\\textbf{Answer}$: Thank you for pointing out the reasonability of the assumption. We partially agree with this point. A more complicated and realistic scenario is that the map of the environment is missing, and the navigation and perception errors will make the object search problem even more challenging. However, even with the given maps, the object search within a partially observable environment is still very challenging, which is our considered problem. In this paper, we focus more on the planning component of the system, while the perception is based on existing methods. If we consider the other planning-focused object search papers, like these papers ([1], [2]), a classical assumption is to offer the given maps and even the given object information, like object size, position, and orientation. The point cloud of the surrounding environment can be corrected once and our method can repeatedly use the map, which is not very expensive commonly.\n\nOf course, it is better to have stronger navigation in a full-unknown environment. In the future, we will apply the online semantic SLAM methods [3] to build the navigation map online, identify the workspaces, and complete the task without manual settings, like the autonomous continuous action domain instead of the discrete action domain. We have revised the statement about the reasonability of the assumption and state it in future work.\n\nRevise “In our framework, the point cloud map ${M}$ of the robot environment with some furniture and known objects is assumed to be available before planning, which is a reasonable and realistic assumption achieved by mapping the environment prior to planning.” into “To simplify the scenario and reduce the navigation effort, we assume the availability of a pre-built point cloud map (${M}$) of the robot's environment, including furniture and known objects, which can be created beforehand through environment mapping and reused during planning.”\n\nAdd \"Our future directions involve reducing manual settings, such as discrete pose candidates, dropping prior information, such as pre-built point cloud map  $\\mathcal{M}$, and exploring continuous action domains.\"\n\n[1] A. Wandzel, Y. Oh, M. Fishman, N. Kumar, L. L. S. Wong and S. Tellex, \"Multi-Object Search using Object-Oriented POMDPs\".\n[2] Zheng, K., Sung, Y., Konidaris, G. and Tellex, S., 2021, September. Multi-resolution POMDP planning for multi-object search.\n[3] Rosinol, Antoni, Andrew Violette, Marcus Abate, Nathan Hughes, Yun Chang, Jingnan Shi, Arjun Gupta, and Luca Carlone. \"Kimera: From SLAM to spatial perception with 3D dynamic scene graphs.\" \n\n$\\textbf{Response to questions}$: \n\nQ1. What limitations...\n\n$\\textbf{Answer}$: Thank you for the constructive question. Please refer to the “Limitation” section of the global comments file for more details.\n\nQ2. You touch on this subject...\n\n$\\textbf{Answer}$: This is a very good question for further improving the practice of the whole framework. Unsuccessful execution may cause totally different challenges to our framework. Some of them are fine to solve based on our current framework. For example, the robot may fail to pick up a detected object and instead push it down, which will cause the data association to stop updating the standing object and introduce a newly detected object based on our framework. This may increase the number of steps in finding the target objects, but commonly fine in completing the final tasks. However, some failures in primitive actions may affect the final performance. There are several ideas about dealing with poor primitives. (1) Based on prior experiments, we can identify the failure rate of different actions in the given scenario and consider the obtained state transition probability in the POMDP transition function, which means considering the failure in planning and reducing their effect; (2) Another way is to improve the successful rate of the primitives using the following possible ways:\n\nAction Execution Monitoring: Implement a robust action execution monitoring system that can detect and handle failures during primitive action execution. This could involve incorporating sensors or feedback mechanisms to validate the outcome of each action. If a failure is detected, the system can trigger a reattempt of the action or take corrective measures to ensure successful execution.\n\nBetter Action Implementation Methods: Enhance action implementation capabilities using more robust and advanced methods to mitigate the impact of uncertainties on primitive execution. Like the move_base and move_it toolboxes may not be robust enough in the complicated environment based on the default setting. For example, we are testing the locomotor, which is an extensible path planning coordination engine that replaces move_base and we are adjusting the used configurations in move_it to increase the removing successful rate. \n\nRobust Perception and Sensing: Enhance perception and sensing capabilities to mitigate the impact of uncertainties on primitive execution. By using more robust and diverse sensory inputs, the system can better adapt to changes in the environment and make informed decisions even in the presence of imperfect or incomplete information.\n\n$\\textbf{Response to Limitation}$: \n\n$\\textbf{Answer}$: Thanks for your comments. Please refer to W3 and the “Limitation” section of the global comments file for more details."}, {"review_id": "G0an0OqflU", "reviewer": "Reviewer_UoQp", "summary": "This paper proposes a method for mobile robots to efficiently search for objects in complex environments. They use a Partially Observable Markov Decision Process (POMDP) formulation and a planning algorithm called Growing Partially Observable Monte-Carlo Planning (GPOMCP) to improve the robot's success rate and speed in finding target objects.", "questions": "* L123: the object pose is usually regarded as 6D or SE(3) although using quaternion.\n\n* A concern regarding using growing state space and tree search is that the problem complexity when the object number increase. What is the computational time for the planner with a varying number of objects?\n\n* How does the resolution of the grid representation impact the performance?", "limitations": "The authors have discussed the limitations.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* A Monte-Carlo planning method to perform object search in the indoor environment based on the grid-type 3D representation.\n\n* A systematic framework and experiments demonstrate the effectiveness of the whole sense-plan-act pipeline.\n\n* A novel design for growing state space using a belief tree.", "weaknesses": "* The whole framework seems complex and required carefully mannual design of parameters (e.g. $R_{max}$ and $R_{min}$ ). It is unclear how the sim-to-real gap will impact these parameters. \n\n* It seems the baseline methods are POMCP and a variance of the proposed approach with replaced modules. It would be good if the authors can discuss or compare some other related object searching methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "$\\textbf{Response about weakness}$:\n\nW1. The whole...\n\n$\\textbf{Answer}$: Thank you for providing valuable comments. While some parameters require manual configuration, it's worth noting that the majority of these parameters do not significantly impact the final performance of the framework. Most of them are not sensitive to the final performance. As an example, the performance does not change too much, if orders of magnitude between $R_{max}$ and $R_{min}$ satisfy $R_{max}>>R_{min}$. For instance, varying $R_{min}$ from -1 to -20 does not exert a substantial influence on performance outcomes. To elucidate this phenomenon, we present a comparison between the results using $R_{min}=-1$ (the case in our paper) and $R_{min}=-20$ for the $Hidden_1$ case in Table 1 of the attached pdf file. \n\nFew parameters may affect the performance a lot. For example, before successfully declaring, we need to update the belief of the log-odds of 8 grid values and then complete the declaring action by comparing the mean value of the minimal $n_{odds} = 2$ log-odds with the thresholds $C_d^o$ for the obstacle object and $C_t^o$ for the target object. The smaller $n_{odds}$ means that this condition is hard to be satisfied and we need to observe more directions of the objects. As $n_{odds}$ decreases, the removing and declaration actions become more dependable, owing to the augmented array of diverse observations derived from different orientations. Our method retains more useful branches after receiving observations and it will show obvious advantages when the target belief is harder to reach. With this goal, we increase $n_{odds}$ to 4 and 6 and show the statistical results of the scenario with 6 objects (Fig. 1) in Table~2 with 20 trials setting.\n\nIn our opinion, the primary challenge of deploying our system to real-world robots lies in addressing errors and failures in the perception and navigation parts rather than our focused planning part, which constitutes our main contribution. If the considered parameters work fine in simulation and the perception and navigation methods work well, they should be fine to apply in real-world cases directly. This relatively direct sim-to-real parameter “transfer” is common in the applications of non-learning-based POMDP solvers to physical robots.\n\nW2. It seems...\n\n$\\textbf{Answer}$: We appreciate the suggestion to consider more baselines for comparison. Please refer to the “Baseline methods” section of the global comments.\n\n$\\textbf{Response to questions}$:\n\nQ1. L123: the...\n\n$\\textbf{Answer}$: Sorry and thanks for pointing it out. 7D in our original formulation means 7 dimensions instead of degree of freedom. We have revised it into a 6D (Degree of Freedom) pose.\n\nQ2. A concern...\n\n$\\textbf{Answer}$: Thanks for your insightful comments. Indeed, the computational complexity of our framework will mainly increase with the number of updating objects. These objects play a crucial role in both transition and visual observation functions, causing fewer sampled particles. Consequently, the computational expenditure associated with these declared objects becomes relatively economical, as it no longer necessitates exhaustive testing of occlusion relationships among all other objects. The good news is that commonly the number of the updating objects is limited by the declaring action. To facilitate a clearer understanding of how our method performs under varying object counts, we have conducted an ablation study. This study includes a comprehensive set of experiments, encompassing an increasing number of objects ranging from 2 to 10, with increments of 2. The scenarios, as depicted in Fig.1 of the attached pdf file, comprise 20 trials each, all adhering to a consistent planning time limit of 60 seconds per step. The results are meticulously documented in Fig.2. Our method will perform obviously better, if the object number is relatively large (blue box areas), because we reuse the useful belief tree and avoid branch-cutting caused by the newly detected objects, which is more common in scenarios with more objects.\n\nQ3. How does...\n\n$\\textbf{Answer}$: This question presents a fascinating insight. The grid world's finer resolution offers a unique advantage in enhancing the reliability of probability updating within the grid. This advantage becomes particularly pronounced when dealing with smaller target objects. It's important to acknowledge that opting for this finer resolution does entail a slightly higher computational complexity (odds updating is very cheap for large datasets). Upon conducting tests, it becomes evident that the good grid resolution should be smaller than the minimum dimensions of the target object along both x and y axes. We are here to correct a typo in our previous version. All the experiments are implemented in 2 cm resolution instead of 5 cm. For comparison, we present statistical results for the $Covered_1$ scenario using 10 cm resolution: 24803.5±3792.3|39.9±6.9|60%. The result is poorer than the POMCP method without using fake object in success rate. To dissect this phenomenon, we manually executed a designated sequence of actions: “move_head_5-move_base_0-move_base_1-move_base_2-move_base_3-move_lift_2-move_base_0-move_base_1-move_base_2-move_base_3-remove_object_4-move_head_8-move_base_0-move_base_1-move_base_2-move_base_3” for our method with 2 cm and 10 cm resolution, and the resulting changes in probability distributions across the grid world are visually depicted in Fig.3. The salient observation is that when the resolution is relatively large, grid probability updates lose precision, particularly when the target object is situated close to an obstacle. The use of a larger resolution not only fails to enhance our method's performance but also introduces potential distortions in the information of the fake object. In short, we recommend the used resolution had better be smaller than the object sizes to improve the accuracy of updating odds values."}, {"review_id": "Q3HAwpYbPx", "reviewer": "Reviewer_pj6p", "summary": "This paper presents a POMDP based approach for finding objects in partially unknown environments.  They present a planning algorithm that reuses the belief tree and uses a fake object to guide the search.  The system is tested in four simulated scenes in Gazebo and outperforms a POMCP baseline.", "questions": "- Are the object positions the same in the 20 trials for each scenario or are they randomized?  If they are the same, only testing on four different configurations of objects is too few, especially when the experiments are done in simulation.\n\n\n- I am confused about how the fake object is used and what the intuition behind it is.", "limitations": "Limitations are only very briefly discussed in the conclusion.", "rating": 4, "confidence": 1, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": " - The proposed system achieves better performance than the baseline in several experiments.", "weaknesses": "- The main experiment of the paper is a set of comparisons to POMCP [13], which was published in 2010. The related works section of the proposed work mentions several other more modern approaches.  It is not clear why these works were not also compared to.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "$\\textbf{Response about weakness}$:\n\nW1. The main experiment of the paper is a set of comparisons to POMCP [13], which was published in 2010. The related works section of the proposed work mentions several other more modern approaches. It is not clear why these works were not also compared to.\n\n$\\textbf{Answer}$: Thank you very much for this comment. Please refer to the “Baseline methods” section of the global comments file for more details.\n\n$\\textbf{Response to questions}$:\n\nQ1. Are the object positions the same in the 20 trials for each scenario or are they randomized? If they are the same, only testing on four different configurations of objects is too few, especially when the experiments are done in simulation.\n\n$\\textbf{Answer}$: Thanks for your comments. Yes, they are the same, not randomized. However, we like to point out that in the area of this paper’s main contribution (online POMDP solving and its application in robotics systems), such evaluation is common and acceptable. The main reason being the methods under this approach are less (if not minimally) reliance on data. Our method does fall under the category of the randomized algorithm, and for this purpose, we did evaluate each scenario 20x with different seeds for the random number generator and presented the results with its 95% confidence interval. \n\n Moreover, pur presented scenario settings are specifically defined to be more difficult/challenging than randomized, especially for the covered and hidden case. Randomized cases-often lead to simple occlusion relationships (easy to observe in one or multiple poses, similar to “loose” cases). Last but not least, we do recognize the importance of understanding the effects of the different components and parameters of the method on its overall performance. To this end, we have taken the initiative to introduce an ablation study that systematically spans an escalating range of object quantities, ranging from 2 to 10 at intervals of 2. The scenarios, as depicted in Figure 1 of the attached pdf file, comprise 20 trials each, all adhering to a consistent planning time limit of 60 seconds per step. The results of these trials are documented in Figure 2 of the attached pdf file, which we hope would provide a comprehensive statistical insight into the performance of our method. Our method indicates more benefit, if the object number is relatively large (blue box areas in Figure 2), because we reuse the useful belief tree and avoid branch-cutting caused by the newly detected objects, which is more common in scenarios with more objects.\n\nQ2. I am confused about how the fake object is used and what the intuition behind it is.\n\n$\\textbf{Answer}$: Thanks for pointing this out. Please refer to the “Fake object” section of the global comments file for more details.\n\n$\\textbf{Response to limitation}$:\n\nL1. Limitations are only very briefly discussed in the conclusion.\n\n$\\textbf{Answer}$: Thank you for the constructive question. Please refer to the “Limitation” section of the global comments file for more details.\n"}, {"review_id": "ToRB8QUUsK", "reviewer": "Reviewer_2WUu", "summary": "This paper addresses the problem of finding efficient policies for a robot to search for a known target object in a physical environment.  The robot can find the target object by either moving around to change its field of view or by moving objects that occlude the target object to reveal the target object. While the robot has access to 3D point clouds and occupancy grid maps of the environment and included furniture, objects on the furniture are not included in the occupancy grid maps.   The authors frame this as a POMDP as the state of the environment is partially observed.  The authors propose a novel POMDP solver – Growing Partially Observable Monte-Carol Planner (GPOMCP) – which reuses the previous belief tree under very specific conditions as defined in Theorem 1, Corollary 1,2 – where the optimal action obtained by reusing the tree will be the same as direct sampling.  Overall this is an important practical problem for robots and a new technique for addressing the infeasibility of exactly solving the resulting POMDP is welcome.\n\nI appreciate the responses of the authors during the rebuttal phase and have raised my rating to Accept.", "questions": "In lines 53-54, the authors claim that the method will always converge to a delta distribution at the correct target pose.  It would be helpful to justify with a proof, or consider conditions where the method may not converge or may converge to the incorrect object pose.\n\nThe paper would also be improved with more details on the specific experiments as Figure 4 is difficult to follow (is the blue box the target in all 4 examples? This is never stated).\n\nOn a minor note, the paper has some typos or style choices that need correction, such as:\nLine 68: ‘an action’ not ‘a action.’ \nLine 145: ‘shows’ not ‘to show .’\nLine 147: ‘and’ not ‘and but .’\nLine 290: ‘rebuilt’ not ‘rebuild’ \n\nThe authors did not include a Limitations section, this must be fixed if the paper is to be accepted.", "limitations": "The authors did not include a Limitations section.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors clearly explain their approach for action execution using ROS interfaces move-base, ros_control and moveit, perception for object detection using point cloud segmentation, point cloud re-projection, sub-image fusion, YOLO detection and SIFT matching, and an estimation of each objects move-ability using the Grasp Pose Detection toolbox, k-means clustering and moveit. The authors explain their planning method, GPOMCP, including their use of the belief tree, and provide appropriate steps, equations, theorems, and corollaries. The authors present simulation results using Gazebo across four environment scenarios based on whether the target object is loose, hidden, or covered and compare their proposed method with the Bellman update (method 1) and the Monte Carlo update (method 2) with the primary POMCP method without the fake object (method 3) and with the fake object (method 4). \n\nThe authors also introduce the concept of a “fake object” (last paragraph of section 1), which represents a fully occluded object to facilitate creating a distribution for the pose of this object.  The authors present results from a Gazebo simulation system comparing their algorithm against prior baselines in 4 levels of difficulty based on the visibility of the target object.  Table 1 indicates that the GPOMCP Planner more successfully locates the target object and with fewer steps.", "weaknesses": "This paper addresses an interesting problem but the Prior Work section misses important precedents.  In line 105, the authors state that when the target object is fully occluded, this presents a new challenge to the best of their knowledge.  However, this problem – where the target object is fully occluded – is well-known in the robotics literature where it is often referred to as “Mechanical Search”.   The problem has also been posed as a POMDP and addressed with Monte-Carlo Search Tree methods (see references below, 1 uses MC Trees).\n\nThe authors did not include a Limitations section, this must be fixed if the paper is to be accepted.\n\nThe paper would be improved with more carefully proofreading to make the English clearer, for example the term “fake object” seems inappropriate.  A better term might be “hidden object” as there is nothing “fake” about this, and a more detailed description of how the distribution evolves in the experiments under the “Covered” condition.\n\nPotentially Relevant Prior Work:\n\nHuang, H. et al. (2023). Mechanical Search on Shelves with Efficient Stacking and Destacking of Objects. In: Billard, A., Asfour, T., Khatib, O. (eds) Robotics Research. ISRR 2022. Springer Proceedings in Advanced Robotics, vol 27. Springer, Cham. https://doi.org/10.1007/978-3-031-25555-7_14\nDanielczuk, M., et al.: Mechanical search: multi-step retrieval of a target object occluded by clutter. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 1614–1621. IEEE (2019)\nKurenkov, A., et al.: Visuomotor mechanical search: learning to retrieve target objects in clutter. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8408–8414. IEEE (2020)\nYang, Y., Liang, H., Choi, C.: A deep learning approach to grasping the invisible. IEEE Robot. Autom. Lett. 5(2), 2232–2239 (2020)\nBejjani, W., Agboh, W.C., Dogar, M.R., Leonetti, M.: Occlusion-aware search for object retrieval in clutter. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4678–4685. IEEE (2021)\nGupta, M., R¨uhr, T., Beetz, M., Sukhatme, G.S.: Interactive environment exploration in clutter. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5265–5272. IEEE (2013)\nHuang, H., et al.: Mechanical search on shelves using lateral access x-ray. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2045–2052. IEEE (2021)\nChen, L.Y., Huang, H., Danielczuk, M., Ichnowski, J., Goldberg, K.: Optimal shelf arrangement to minimize robot retrieval time. In: 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), pp. 993–1000. IEEE (2022)\nNakhimovich, D., Miao, Y. and Bekris, K.E., 2023. Resolution Complete In-Place Object Retrieval given Known Object Models. arXiv preprint arXiv:2303.14562.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "$\\textbf{Response about weakness}$:\n\nW1. This paper...\n\n$\\textbf{Answer}$: We appreciate the reviewer's insightful feedback regarding the Prior Work section. Apologies for overlooking the concept of 'Mechanical Search' in our initial version. We have revised the paper and connected our framework with “Mechanical Search”. Some related papers have been added in the revised version, especially for the concept-related paper (Mechanical search: multi-step retrieval of a target object occluded by clutter) and the MCTS-based paper (Mechanical Search on Shelves with Efficient Stacking and Destacking of Objects).\n\nW2. The authors... and Q4. The authors...\n\n$\\textbf{Answer}$: Thanks. Please refer to the “Limitation” section of the global comments file for more details.\n\nW3. The paper...\n\n$\\textbf{Answer}$: Thanks. Please refer to the “Fake object” section.\n\nThe distribution evolution in the $Covered_1$ case follows a similar pattern to other scenarios. Initially, a uniform distribution spreads across the grid (The odds is 1 for each grid), followed by odds updates using Eq. 1 inspired by occupancy grid map updates adapting with field of view (FOV) changes:\n\n• In FOV where no objects are detected, the log-odds value is incremented by a negative value, indicating that the grid is unlikely to contain the target object.\n\n• In FOV where non-target objects are detected, the log-odds value is augmented by a negative value corresponding to the object detection confidence (<0.5), signifying that the grid is less likely to contain the target object.\n\n• In target object-detected FOV areas, log-odds rise positively with confidence (>0.5), signifying higher target likelihood.\n\nTo provide a clearer understanding of the odds updating process in the $Covered_1$ case, we manually executed a designated sequence of actions: “move_head_5 -move_base_0-move_base_1-move_base_2-move_base_3-move_lift_2-move_base_0-move_base_1-move_base_2-move_base_3-remove_object_4-move_head_8-move_base_0-move_base_1-move_base_2-move_base_3” for our method with 2 cm (We are here to correct a typo in our previous submission. All the experiments are implemented in 2 cm resolution instead of 5 cm.) and 10 cm resolution, and Fig. 3 shows resulting grid world probability distribution changes.\n\nIn the last two images, locate two green dotted circles. For the one with a 10 cm resolution, the updated probability corresponding to the target object is not predicted well, which is a near-0 value, after all actions. Our 2 cm method offers better prediction (~1), benefiting from finer object resolution. This detail aids in updating reliability, especially for small targets. This advantage becomes particularly pronounced when dealing with smaller target objects, but with slightly higher computation (but odds updating is very cheap commonly). The salient observation is that when the resolution is relatively large, grid probability updates lose precision, particularly when the target object is situated close to an obstacle. The use of a larger resolution not only fails to enhance our method's performance but also introduces potential distortions in the information of the fake object. As evidence, we present statistical results for the covered scenario using a 10 cm resolution: 24803.5±3792.3|39.9±6.9|60%. It is worse than the POMCP without using fake object in success rate. Recommendation: Resolution < object sizes for accurate odds updating.\n\n$\\textbf{Response to questions}$: \n\nQ1. In lines 53-54...\n\n$\\textbf{Answer}$: Thanks. Apologies for the seemingly overly strong claim, especially in the context of the delta distribution. For instance, when dealing with multiple target objects situated at different positions, the resulting distribution may not exhibit the characteristics of a delta distribution. I've adjusted the claim: “As new accurate observations are perceived, the robot refines its belief about the fake objects, and eventually the belief has a high chance to converge to the correct target object positions based on suitable parameter settings.” However, it is our intention to demonstrate that, in general, the probability tends to behave in a certain manner.\n\nAssuming all grids are updated at least $N$ times in the whole action sequence with $N_1>=N$ observations and, in each observation $z_t$, any $t\\in[0, N_1]$, the values $\\log Odd(g_i|z_t), i=1,2,3,4$ corresponding to different cases (a. the grid $g_1$ in FOV where no objects are detected, b. $g_2$ in the FOV area where non-target objects are detected, c. $g_3$ in FOV where the target object is detected, d. $g_4$ outside the FOV area) are accurate enough satisfying $\\log Odd(g_i|z_t)<-∆<0, i = 1,2$, $\\log Odd(g_i|z_t)>∆>0, i=3$, and $\\log Odd(g_i|z_t)=0, i=4$. We have: \n$\\log Odd(g_i|z_{1: N_1})<= \\log Odd(g_i|z_{1: N})<-N∆, i=1,2$ and $\\log Odd(g_i|z_{1:N_1}) >=\\log Odd(g_i|z_{1:N})>N∆, i=3$, so we have:\n$Odd(g_i|z_{1:N_1})<\\exp^{-N∆}, i=1,2$, and $Odd(g_i|z_{1:N_1})>\\exp^{N∆}, i=3$,\nIf $N→+∞$, $Odd(g_i|z_{1:N_1})→0$ and $P(g_i|z_{1: N_1})= Odd(g_i|z_{1:N_1})/(1+Odd(g_i|z_{1:N_1}))→0, i=1,2$ and  $Odd(g_i | z_{1:N_1})→+∞$ and $P(g_i|z_{1:N_1})→1, i=3$. Normalized probabilities (0-1 range) across the grid world lead to convergence towards actual target object grid cells. This progressive refinement of probabilities contributes to a more precise understanding.\n\nQ2. The paper...\n\n$\\textbf{Answer}$: Thanks. We've added sentences clarifying the target object and other specifics, including “The comparison is implemented based on 4 different scenarios with different object numbers, including LOOSE_1 (4 objects in 1 workspace), LOOSE_2 (6 objects in 2 workspaces), Hidden_1 (7 objects in 1 workspace), and Covered_1 (7 objects in 1 workspace),  as shown in Fig. 4. The target object is the blue snack box. Only the black headset box and the dish rack are not movable, and others are all movable.”\n\nQ3. On a...\n\n$\\textbf{Answer}$: Thanks for your thorough review. Typos have been corrected in the revision."}]

Author rebuttal:
$\textbf{Baseline methods}$

$\textbf{Answer}$: We appreciate the suggestion to consider more baselines for comparison. However, as a system-level application, it is generally fair because many object search approaches are based on the pure POMCP method with different problem settings, eg.[1] [2]. Our GPOMDP solver is designed specifically for problems with growing state space using belief tree reuse and a modified upper confidence bound for improved upper bound estimation. This method can be easily applied to other online POMDP solvers that use particle representations with belief trees, such as ABT[3] and DESPOT[4], offering a competitive advantage when dealing with problems involving a growing state space. Other approaches, which are mainly learning-based methods, eg.[5], often require substantial training datasets and may achieve a general performance level of 40%-60% success rate, which is lower than our results.

[1] Zheng, Kaiyu, et al. "A System for Generalized 3D Multi-Object Search." 
[2] Wandzel, Arthur, et al. "Multi-object search using object-oriented POMDPs"
[3] Kurniawati, Hanna, et al. "An online POMDP solver for uncertainty planning in dynamic environment." 
[4] Somani, Adhiraj, et al. "DESPOT: Online POMDP planning with regularization."
[5] Druon, Raphael, et al. "Visual object search by learning spatial context."

$\textbf{Fake object}$

$\textbf{Answer}$: Our fake object is essentially a state variable that represents a “guess” on the target object. Our fake object is not a real object or a future detected object. The real target object will be included in one of the later sub-state-vector $s_{o_1}$ to $s_{o_n}$. In our problem, initially, we do not know the objects in the environment, neither the number of objects nor their positions, which means the whereabouts of the target object are also initially unknown. To capture uncertainty about the target object compactly, we propose to “guess” the target object. It can be considered as an additional visible object, related to the target object, in transition and visual observation functions of the POMDP framework, but not detectable in the real world. It uses the same structure as the structure used to represent an object in the system, which includes an 8-cell occupancy grid map where each cell maintains a probability of whether the cell intersects with the target object. Under the POMDP framework, a belief over the fake object variable represents multiple guesses with different probabilities about the target object. Initially, the uncertainty about the target object can be quite large, but over time the belief in the position of the fake object tends to converge to a probability mass whose mode is at the true target object. The distribution of the fake object (grid world) saves the belief about the target object. But, you know, in POMDP problems, we need to predict the future probability of all girds in MCTS by going through the transition functions and observation functions, which will be time-consuming and memory-consuming. The application of the fake object becomes meaningful to involve the useful information (represent grid world), is very computational (fast for transition functions and observation functions) and memory (a low-dimension vector) cheap in exploration and rollout, and is friendly for coding (use the same transition and visual observation function as the other detected real objects). We have added some sentences for better explanation in the revision. Last but not least, we apologize for the seemingly odd “fake object” terminology and will revise the name to “guessed target object”.

The way to use the fake object is shown in Fig.4 of the attached file. Its grid world is updated based on the real-world observation from the camera and its position will be sampled and the other terms of the fake object will be re-initialized to use in the transition function and observation function of the Monte Carlo tree search. Then, this iterative process persists until the task is completed. We will add more explanations in the supplementary material.

$\textbf{Limitation}$

$\textbf{Answer}$: When deploying the system in the physical world, our main limitations mainly stem from the errors and failures from perception and navigation components, rather than our focused planning part. Our planning method should be fine to apply in real-world cases directly with reliable support from perception and navigation methods. Under the POMDP framework, for primitive actions, some of the errors and failures can be handled. However, large failures with multiple knock-on effects would be difficult to handle. An example is when the robot’s failed grasping resulted in the object falling under the table or falling and causing many other objects to fall too. Another limitation is that our framework is based on point cloud segmentation to identify different objects. It may generate incorrect bounding boxes for objects with extensive contact areas. This misinterpretation can result in erroneous data association, impacting belief updates and removing actions. Furthermore, YOLO and SIFT object detection techniques may struggle in scenarios with limited SIFT features and YOLOv5 toolbox coverage, especially in low-light environments. However, we believe that addressing these limitations is possible by incorporating more advanced perception and navigation methods. 

In terms of the algorithm itself, scalability to problems with too many unknown objects in the environment can be difficult. In such scenarios, one could cluster objects hierarchically, and apply the proposed method to clusters of objects, rather than single objects, in the higher level of the hierarchy. Application at the single object level can then be limited to a medium-sized area after the system has a better understanding of the target object. 

We have added a new paragraph to emphasize the limitations of our method in the revised paper by summarizing the above points.


Meta-review: The paper considers the problem of searching for a target object in a potentially cluttered environment using a mobile robot that is subject to a limited field-of-view, localization errors, and occlusions caused by objects in the environment. Given access to a 3D point cloud and occupancy grid of the environment, the paper formulates the search problem as a partially observable Markov decision process (POMDP) and proposes a Monte Carlo tree search (MCTS)-based solver (GPOMCP) that is able to reuse the previous belief tree under certain conditions when a new state (e.g., a newly observed object) is added. The system is evaluated on four simulated mobile manipulation environments and shown to outperform a POMCP baseline.

The paper was evaluated by four reviewers who largely agree on the paper's core strengths and weaknesses. Among them, the reviewers emphasize the significance of the object search system that the paper proposes---this is a strong systems paper that clearly describes a non-trivial framework and its implementation that addresses an important problem in robotics. Beyond the value of the systems-level integration and the merits of the underlying POMDP formulation, they also appreciate the modifications to the POMCP required to account for a growing state space. However, the reviewers are also in agreement about the weaknesses of the paper as originally submitted. Key among them is the lack of appropriate baselines both with regards to quantitative comparisons as part of the experimental evaluation as well as qualitative comparisons to related work that was not included in the original submission. The authors address this, stating in part that several existing search methods are based on the POMCP method to which they compare, while other methods require substantial training. The paper should be updated to better justify the choice of baselines and include a qualitative comparison to the methods that Reviewer 2WUu references. Additionally, several reviewers point out that the paper lacks an explicit section that identifies the limitations of the method or really any discussion of its limitations. The authors provide a detailed discussion of the limitations in their response and state that this discussion will be added to the paper. The reviewers and AC appreciate the authors' efforts to address the reviewers' initial questions and concerns, which make this a solid systems-level paper.
Decision: Accept (poster)