Stealthy Terrain-Aware Multi-Agent Active Search
Authors: Nikhil Angad Bakshi
Jeff Schneider
Venue: CoRL 2023 Poster (2023)
Keywords: Reconnaissance
Adversarial Search
Multi-robot
Active Learning

Abstract: Stealthy multi-agent active search is the problem of making efficient sequential data-collection decisions to identify an unknown number of sparsely located targets while adapting to new sensing information and concealing the search agents' location from the targets. This problem is applicable to reconnaissance tasks wherein the safety of the search agents can be compromised as the targets may be adversarial. Prior work usually focuses either on adversarial search, where the risk of revealing the agents' location to the targets is ignored or evasion strategies where efficient search is ignored. We present the Stealthy Terrain-Aware Reconnaissance (STAR) algorithm, a multi-objective parallelized Thompson sampling-based algorithm that relies on a strong topographical prior to reason over changing visibility risk over the course of the search. The STAR algorithm outperforms existing state-of-the-art multi-agent active search methods on both rate of recovery of targets as well as minimising risk even when subject to noisy observations, communication failures and an unknown number of targets.

Stealthy Terrain-Aware Multi-Agent Active Search
Nikhil Angad Bakshi Carnegie Mellon University nabakshi@cs, cmu. .uu
Jeff Schneider Carnegie Mellon University s schneide@cs. cmu. edu
Abstract: Stealthy multi-agent active search is the problem of making efficient sequential data-collection decisions to identify an unknown number of sparsely located targets while adapting to new sensing information and concealing the search agents' location from the targets. This problem is applicable to recon- naissance tasks wherein the safety of the search agents can be compromised the targets may be adversarial. Prior work usually focuses either on adversarial search, where the risk of revealing the agents' location to the targets is ignored or evasion strategies where efficient search is ignored. We present the Stealthy Terrain-Aware Reconnaissance (STAR) algorithm, multi-objective parallelized Thompson sampling-based algorithm that relies on a strong topographical prior reason over changing visibility risk over the course of the search. The STAR algorithm outperforms existing state-of-the-art multi-agent active search methods both rate of recovery of targets well as minimising risk even when subject noisy observations, communication failures and an unknown number of targets.
Keywords: Reconnaissance, Adversarial Search, Multi-robot, Active Learning
1 Introduction
Search and reconnaissance tasks are distinguished from each other only by the adversarial nature of the targets: they do not wish to be found and search agents must attempt to conceal their own locations from the them. Despite this, these two problems share many common elements. Histori- cally, both have been a largely human endeavour, but several factors may impede effective search by human teams. The search region could be too vast to mobilise enough human resources effectively the personal safety of human searchers could be risk. Multi-robot systems increasingly being deployed for search missions via tele-operation [1, 2. 4]. While human operators motely control a small number of robotic platforms, they cannot efficiently coordinate larger teams [5]. Consistent communication between the agents may not be possible either due to environmental factors or hardware failures. Finally, search operations are often time-critical, hence decentralized multi-robot teams capable of efficient asynchronous adversarial active search are crucial.
The problem of adversarial search has been theoretically studied the pursuer-evader problem [6, 7]. Several approaches seek to maximise the worst-case performance of the pursuer and imbibe the evader with extraordinary abilities like complete knowledge, infinite travel speed, and infinite compute. However, these solutions are often prohibitively expensive to compute [8] in real-time or too conservative to be applicable in the real-world settings [9].
We model the problem as one of stealthy target detection [10], with multiple pursuers and immobile evaders that may be placed on the map adversarially. This is in contrast target tracking [11, 12] where the targets can move. Target detection with static targets is a realistic choice as, in the reconnaissance task, it is not uncommon that the search is being carried out for well-concealed static objects like pieces of infrastructure or environmental features as this knowledge can be of strategic importance; and, in search and rescue missions, stranded people could be immobile or mobile, but until they are detected for the first time they can be treated immobile and the problem formulation remains the same.
The core idea of our approach is simple, during search and reconnaissance missions, strong prior on the number of targets or their locations is usually not available, however, satellite imagery and
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

(a) Desert mountainous landscape in (b) Top-down view of the terrain in (c) Traversal costmap of the Unity simulation environment with two (a) showing average visibility as terrain full-size ground vehicles also pictured. colour gradient. Y high; B- low. search zone (red polygon).
and (h) and the
Terrain can inform search and evasion strategies. The grid size in Fig. lb is 60m x 60m. This region's dimensions 2305.5m x 1837.5m representing total area of 4.266km In the costmap in Fig. Ic the area inside red polygon 2.48km² Magenta is con-traavrsable. Black to blue red indicate increasing ropography-awan visibility risk. The green cross is the goal location provided to the OMPL [13] or A-star [14] path planner and the green curve is the path computed to the goal location while minimizing the visibility risk.
Figure 1:
by extension topographical information of the search region may be available. If the agent were to search for sparsely located targets in the map shown in Fig. I, the intuition is that it would be beneficial to search in better hiding places.
We present the Stealthy Tware Reconnaissance (STAR)' algorithm, a multi-objectto opti- misation algorithm that follows the Myopic Posterior Sampling (MPS) framework [15, 16, 17, 18] which has been shown be near optimal for adaptive goal-oriented tasks [19] such in the multi- agent asynchronous active search problem we have here. In MPS, we select the optimal action by optimising a reward based on single sample (known as the Thompson Sample) taken from the cur- rent belief. This allows for a calculated randomness in the search decisions made by various agents even communication or hardware failures prevent coordination. As more observations are made, the agents learn to take better actions that bring them closer to their search/risk objective.
We ablate the performance of STAR against existing methods varying the map type, availability of communication, number of search agents and whether or not targets are placed adversarially in simulation. In all cases, STAR outperforms existing methods. We have designed STAR to be de- ployable on ground-based robotic platforms described in [18] in a search region that is 4km² in size. Our motivation for only presenting simulation results in this paper is to ablate and assess the performance of the STAR algorithm in our realistic simulator against the state of the from an al- gorithmic standpoint physical runs cannot usually be conducted in quantities that show statistical significance.
To the best of our knowledge, STAR is the state-of-the-art in search efficiency for (adversarial) multi-agent active search given a known terrain map that also robust to communication failures, operates without any human direction or explicit subdivision of the search region. Our contributions may be summarized follows:
We propose the Stealthy Terrain-Aware Reconnaissance (STAR) algorithm, a multi-objective search algorithm that combines the information-seed seeking reward term presented in Bakshi et al. [18] with novel stealth objective that uses a known terrain map to encourage concealment of the search agent while improving search efficiency by searching in locations with greater likelihood of recovery.
We ablate this superior performance in communication-tisableed scenarios with our proposed terrain-aware noisy observation model, varying number of agents, map types, and in adversar- ial and non-adversarial scenarios. each case STAR outperforms all other methods.
Finally, STAR has been deployed on our physical systems for search and/or reconnaissance mis- sions. Appendix A contains details of the physical systems.

---

2 Related Work
In pursuer- problems the primary objective of the search agent(s) is to trap or track the evader with theoretical guarantees for worst-case evader [20, 21, 22, 23). However, these approaches don't consider the inverse adversarial problem of minimizing risk of detection by the adversary (as it already has complete knowledge). Some approaches attempt use approximate algorithms and relax the requirement for guarantees [24], however none of these approaches can be extended to have an unknown number of evaders and that limits their practical applications.
Probabilistic search methods seek to improve expected or average case performance. Bayesian learn- ing provides an effective way to probabilistically model the world, inculcate prior information and adapt information over the course of the search [25, 26]. However, these approaches often rely on perfect observation models (no noise, no false positives) and their examination of optimal behaviour is usually confined to single pursuer single evader cases [27, 28). In similar vein, modelling the problem using Partially Observable Markov Decision Processes (POMDPs) [29] yields tractable optimal solutions only with a single pursuer. An excellent survey on adversarial search by Chung ct. al. [30] covers an overview of the field and open research questions.
Decentralised adversarial multi-agent active search that is robust to communication failure is an actively researched field. Though multi-robot teams may partition the search region for exploration efficiency [31], generating such partitioning is challenging with unreliable communication. Given the success of the POMDP formulation in the single-agent case, researchers have attempted to apply reinforcement learning to the problem [10, 32]; however, these approaches are extremely sample inefficient and prone to overfit to the environment they are trained on. In our formulation with known topography, it is not clear if these RL methods will generalize to different topographies and therefore they aren't I a good candidate for realistic search missions.
Terrain-awasd aware path planning with adversarial targets is well-studied in the context of military op- erations [33, 34, 35] and in the context of stealth-based video games [36, 37]. However, these approaches focus on path planning but not on a competing search objective, that is, they assume that the adversary locations are known and need to be avoided, or are unknown and need to be evaded encountered en route the goal.
Adversarial search has some implementations on real-hardware and there are approaches that at- tempt to validate their results in simulation [27, 38, 39]. However, these approaches are usu- ally single-agent [40, 41]. If they are multi-agent then they rely on strong coordination between agents [42, 43]. This motivates that an efficient solution to multi-agent reconnaissance problems that can be deployed on real systems remains an open question.
GUTS [18] is non-adversarial multi-agent active search algorithm that has been shown to out- perform state-of-the-art algorithms on recovery rate of targets and has been deployed on physical hardware. It is the state-of-the-art for robust multi-agent active search and it can handle intermittent communication and observation uncertainties, but it is not suitable for the reconnaissance task.
3 Problem Formulation
We model the search region as a grid with cell size of 60m x 60m. Fig 1c shows an overhead view of the costmap for one of the maps we test on. Due to the ubiquity of satellite imagery, it is reasonable to assume such approximate map information of the search region is available. Some parts of the map may be different during deployment on physical systems, but our on-robot sensing and mapping system can recognize changes and dynamically update the prior map. For our experiments, however, assume the map to be fixed. We model the stealthy active search problem follows:
We give the same search region to all robots, for example, see red polygon in Fig. 1c.
The targets are sparsely placed and static. They need to be recovered quickly with high certainty.
The robots must minimise their exposure to the targets which considered hostile.
Each robot must plan its next data collection action on-board, i.c., central planner exists.

---

(a)
(b)
(c)
(d)
Figure 2: A simple illustration of the viewahle
Figure 3: A realistic illustration of the viewable region of the target when (c) unbounded if it were located the red dot; (d) discretized (60m 60m cell size grid) and subject to vicwing limits (200m - 300m) superimposed on the topo- graphical map. Further details in Appendix A.4
regions of the (a) rohot given its coordinates and direction of facing, and (b) target given co- ordinates. Lighter the shade greater the noise. Black cells are obstructions.
The robots may communicate their locations and observations with each other; however, the al- gorithm's 's performance should improve with increasing number of search agents anywhere in the spectrum of total absence of communication to perfect communication.
Formally, we represent the locations of targets in our 2D) grid representation using sparse matrix R&M xM2, Let ß E RM be the flattened version of matrix B, where M M2 This is a sparse vector, with is corresponding to target locations and Os elsewhere. The objective is to recover the true B through search. We model the terrain-aware noisy observations through Eqn. 1 and Eqn. 2.
y clcip(x((BBbb..O
(1)
b, D;
(2) P(L)) = LEXL
(3)
where, X/ E R@MM describes the sensing matrix such that each row in X/ is a one-hot vector indi- cating one of the grid cells in view of the robot j at timestep 1, and Q is the total number of grid cells the robot can view. y/ E [R@x1 is the resultant observation including the additive terrain-aware noise vector b/ E R@x1, The observation is clipped to be within O and as the additive noise can cause the resultant quantity to exceed those bounds. Going forward, we assume that these quantities are defined on a per-robot basis and drop the superscript j for ease of notation. The topography-awes noise b, has two components; firstly, it encodes the intuition that observation uncertainty increases with distance the robots; secondly, it encodes the intuition that observation uncertainty increases with occlusions in the line of sight from the robot.
In Eqn. 2, denotes element wise division, n, ~ (0, with diagonal elements of the noise covariance matrix Σ, monotonically increasing with the square of the distance of the observed cell from the robot. v, Rex1 where each entry represents the square of the fractional visibility (ac- counting for occlusions) of each of the @ cells visible in X,. The noise is sampled from positive half-Gaussian distribution ./5 E.) and is added for cells without targets and subtracted for cells with targets. Similarly, we have observation model for the targets albeit with some relaxations, namely, only accounting for occlusions but no depth-aware noise. Fig. 2 shows simple examples of the modelled viewable regions of the robots and targets, and Fig. shows a realistic example for target viewable region.
Let the robot trajectory for robot j until timestep be denoted L/ R& such that each entry in L/ is the integer count of the number of times robot has visited that cell in the whole space M. In Eqn. 3, we define a penalty function IP(L)), which penalizes the robot for showing itself to any of the targets. Similar above, X ReO XM is the sensing matrix for the 4t target, note that is not dependent on the timestep single real number. The stealth penalty can be thought of as a scaled discretized representation of the time spent in the viewable region of the target(s).
targets static. The second summation is to reduce the value to
Let D/ be the set of observations available to robot j at timestep 1. D/ comprises of (X,,y:) pairs collected by robot as well as those communicated to robot j by other robots. Let the total number of sensing actions by all agents be T. Our main objective is to sequentially optimize the next sensing action X based on D/ at each timestep to recover the sparse signal ß with as few measurements

---

T as possible while minimizing the stealth penalty over all robots E,P(L/). Each robot optimizes this objective based on its own partial dataset D in a decentralized manner.
STAR: Stealthy Terrain-Aware Reconnaissance
This section presents the STAR algorithm. Each robot asynchronously estimates the posterior distribution over target locations based on its partial dataset D/. During the action selection stage, each robot generates a sample from this posterior and simultancously optimizes a reward function and stealth penalty for this sampled set of target locations. The reward function represents potential information gain for the sensing action in consideration, while the stealth penalty represents the potential information leakage based on the partial dataset D/ available.
4.1 Calculating Posterior
Following Bakshi et al. [18], each robot assumes zero-mean gaussian prior per entry of the vector B s.t. (Bm) = M (0, 7m)- The variances - diag (n /wl) are hidden variables which are estimated using data. Bakshi al. [18] follows Tipping [44] and Wipf and Rao [45] and uses a conjugate in- verse gamma prior (I...M. The salient feature of the inverse gamma distribution IG(.) is that selects a small number of variances Yo to be significantly greater than zero, while the rest will be nearly zero, this enforces sparsity. We estimate the posterior distribution on B given data D'' for robot us- ing Expectation Maximisation [46]. We can write analytic expressions for the E-step (estimating p(B D .)) N(u, V)) and M-step (computing maxr pD, B)r) respectively:
to enforce sparsity s.t. p(m)) IG bm) [(a)) Yin
bm
1 -- ------)
(bm -w- /m)) Vm E
V -
(4)
Yn = (V mm + [u] + 2bm) 2am)
(5)
where X and y are created by vertically stacking all measurements y.) in D/. and E is a diagonal matrix composed of their corresponding terrain-aware noise variances.
Each robot estimates (( D N(u, V) on-board using its partial dataset D/. We set the values of am = 0.1 and bm 1 as these were found to be effective in Ghods et al. [16]. Finally, agent j samples from the posterior B p(B)D/).
4.2 Choosing Next Sensing Action
Each robot chooses the next sensing action X by assuming that the sampled set of target locations is correct. Specifically, let B (D/ (X++1. be our expected estimate of the parameter using all available measurements D/ and the next candidate measurement (X++1,)+1). Then, following Bakshi al. [18] the reward function defined as:
2 (B. = - B (X.
(6)
Where, (-= 0.01) is hyperparameter that reduces the reward for a search location the estimated β does not have high likelihood entries in common with the sample at the current step B. Let & and & be the number of non-zero entries in and B. then the indicator function 7(.) is defined as:
0, if any matches between top entries in B and top entries in B
(B.B) =
1, otherwise
The reward function is stochastic due the sampling of B and this ensures that the search actions selected by the robots are diverse. The intuition behind this reward term is that those search de- cisions are preferred that can confirm the locations of suspected (but not confident) targets as per the sample. This reward function was shown to improve search recovery rate in robotic search and rescue missions over existing methods [18] which tend to be more explorative.
The reward in Eqn. must be balanced against the risk quantitified by the stealth penalty term (Eqn. 3). Since it is computationally infeasible to compute the risk all possible trajectories, calculate the penalty for every possible goal location I of each robot E I is one hot
5

---

X1.7
30
Sheelth Peraltty Freed
Dectaton Agent
25
CINERAAE
OUTS
GUTS TIA
57A0
Rii
15
10
2
6
2
Mumber of agents ())
Number of agents ()
(b)
(c)
(d)
Figure 4: Depth Elevation Map (DEM) of (a) the Figure 5: Experimental Results in the Grid Map (Fig. 4-b).
STAR (red) outperforms existing methods (c) narrowly on and (b) grid of perpendicular corridors. DEMs Search Efficiency, and (d) significantly Visibility Risk
natural mountainous search region from Fig. 1,
are heightmaps, when represented an image the brighter the region, greater elevation.
due novel multi-objective function. RST's efficiency remains flat as it is an information-gree method.
vector indicating the location of the robot corresponding to the potential measurement X/ in the reward defined in Eqn. 6. This will yield a risk landscape over the entire map that is used for action selection and path planning- Since we only have the posterior ß over target locations and ground truths, in Eqn. we use the folded normal distribution determine a separate posterior mean Aiis R&M for visibility risk that accounts for the mean and variance of the posterior B. The final risk objective is defined in Eqn. 8:
2V π
exp
F 2V
+ Hi
1 26
-HI VV
(7)
34
-
A X
(8)
where, in Eqn. 7 V and μl are the variance and mean of the posterior defined Eqn. 4, ø is the error function (z) / Jö e e and indexes into the M length vector. In Eqn. 8, f behaves weighting scalar for each location i the map where threat may located. When the posterior variance for location i close to zero then A will tend to the posterior mean μí', and when the variance is high but the mean is (as it is at the beginning of the run) the weighting factor will still be non-zert nenz it will be governed by the variance. The overall optimisation objective then be thought of as two competing objectives as follows:
X, 1
mag X.
(B COBB".D.... -7 from (6) and (8)
where, is hyperparameter controls the tradeoff between goal selection to satisfy the stealth penalty and the reward term. We found that the best value for y is combined with normalising both the reward and stealth penalty terms between and 1.
Experiments and Results
Our experiments demonstrate the superior search efficiency of our proposed algorithm STAR com- pared to existing search methods: GUTS, RSI, coverage-based search and random search. The GUTS algorithm [18] is parallelized Thompson sampling based algorithm that prioritises recovery rate in multi-agent active search missions with realistically modelled noise. It has been optimised to real robots and the best of our knowledge it is the state-of-the-ar in decentralised multi- agent active search methods. Region Sensing Index (RSI) [47] is active search algorithm that locates sparse targets while taking into account realistic sensing constraints based greedy max- imisation of information gain. The coverage baseline myopically chooses the next waypoint in an unvisited part of the search region while the random search policy randomly selects a cell to visit.
5.1 Testing Setup
Each search run is initialized by specifying the same search region for each robot (see Fig. Ic). All robots start the same location. We evaluate the various search algorithms under two target sampling paradigms: uniform and adversarial. In uniform sampling the targets placed uniformly

---

Non-Adversarial Placement of Targets
Adversarial placement of Targets
Dection 200 dgeer (td)
8 1 8
Dictioon stop par Aoeett TTll
8 8 8
K20
ANEOO
1.3
CATS
2
Full Communicatioo
1.
1.0
5
0.3
0.5
Number agents
Number
Number agents
Number agents
Communicatoo Broaldwwn
Decsin zteo pe Agere 110)
8 E 5 8
x10²
Dec san SEeO per Aaann 173
0 88
Sootth Ponaty ircumees
X10
CDV
sath
kM
E E
Natria of uganis
Number of opents 00
Number of agpani 02
Numbe: of agents 01
Table 1: Simplified Simulator Results. STAR (red) outperforms existing sfate-of teffthheatt art methods both metrics (lower is better) regardless of how targets are placed, and even with total communication breakdown.
at random in the search region. In adversarial sampling the targets are placed stealthily, i.c., they are placed in locations with lower average visibility within the search region. We control for the locations the targets using these two paradigms. We utilize two possible maps, a mountainous desert landscape (See Fig. la and Fig. 4a) and a grid of corridors (see Fig. 4b). There are always targets (K) but this is not known apriori. We run experiments varying number of search agents (J) which may may not be able to communicate with each other showcase the robustness of STAR.
5.2 Evaluation Metrics
Our primary evaluation metric the recovery rate, which defined the fraction of targets the search method has located against the number of search decisions made within the time budget. Our secondary metric is the stealth penalty incurred by the team of search agents.
We evaluate the algorithm on the basis of decision steps, i.e. one search decision is one time step. This is equivalent to assessing an algorithm's sample complexity while abstracting away environ- ment/hardware specific factors like terrain conditions or particular compute specifications that might affect wall clock time. That being said, when evaluating in the realistic simulator each algorithm is given the same runtime budget of hr and 15 min, this is short enough to be realistic duration for search operation and long enough that robots with max speed of 5 m/s may complete exploration.
The stealth penalty a scaled and discretized representation of the time spent in the viewable region of the target(s). Eqn. 3 describes the stealth penalty as the dot product between the viewable region of the targets and the path(s) taken by the robot(s). Given the cell size of the grid and speed of the robots, may calculate time spent by the agents in view of the targets using the stealth penalty. Similar to search efficiency, we choose to abstract away physical quantities like speed and report performance on the stealth penalty directly.
5.3 Simple Simulator Results
Our simple simulator accurately simulates the robots' sensing and trajectory planning while using very simple physics model for robot traversal in order to speed up simulation time. Fig. 5c-d shows the performance the results in the simple simulator on the grid of corridors (Fig. 4b) with adver- sarially placed targets. STAR wins out on both metrics, maximising recovery rate of targets and minimising risk. Table shows the results on the simple simulator for the mountainous desert land- scape (Fig. 4a) and ablates it against communication failures and non-adversarial target placement. Across the board, STAR (marked in red) outperforms existing methods. When communication fail- ures exist, coverage based planners suffer the most as their search efficiency relies coordination. RSI remains unaffected as it is an information greedy algorithm and more agents doesn't 't translate
7

---

I-Robot Realistic Simulator Runs
2-Robot Realistic Simulator Runs
1.0
1.0
XXC
Advemaraal Placement
1,00
0.00
1.00
5.75
0.75
0.4
Recov
d.13
0.50
0.25
0.00
0.00
Docs steps per agent (TU)
Dectaonn strps per rpent (T:)
Dectaion staze per agant (T))
Decedon steps perageett
Nor- Adt.rddaasaaall Planement
x1CC2
X100
0.8
1.5
Recowery pate
0.G
Recoverr Rabe
0.4
05
20M
0.0g
0.4 3.0
30
0.0
15
Docs
(1D)
Decillon
Dectatio
Table 2: Realistic Simulation Resulis. STAR (red) has greater search efficiency (largets found / total targets
K) and lower stealth penalty incurred regardless of target placement strategy. In the top-left, RSI (purple) is able to achieve a lower penalty than STAR as it fails to locate all targets in most runs. The end of each curve is the end of the runtime budget, hence the penalty at the end of each curve is the final penalty for that run.
greater efficiency either. GUTS, the current ofate-of-the-ar decentralised multi-agent active search, performs fairly well on recovery targets but clearly loses out on the stealth metric to STAR.
5.4 Realistic Simulator Results
We have designed STAR such that it runs in real-time the multi-robot team presented in [18]. We simulate field tests in desert mountainous environment with search area of ~ 2.5km² with realistic physics model and present those results here. We compare the search efficiency of STAR, with GUTS, RSI, random search and coverage-based search Table 2. We plot results varying team size and target placement strategy with five targets. Results are aggregated across 10-12 runs for each line on the graph. We can see that STAR outperforms our baselines: GUTS, RSI, coverage- based search and random search on the recovery rate (odd columns) as well in terms of the stealth penalty (even columns) regardless if the targets are placed adversarially uniformly.
5.5 Discussion
Despite optimising to reduce the stealth penalty, competing objective, STAR still outperforms the other algorithms on recovery rate, including GUTS, the the previous best algorithm in terms of search efficiency. This indicates that the terrein-aware stealth penalty term improves the discrimina- tory power of the reward function.
The results shown in this work demonstrate that fully autonomous robots can effectively search in complicated natural terrains in time efficient manner. We believe the algorithm presented here paves the way for more ubiquitous application of autonomous robotics in multi-agent search for dis- aster response and reconnaissance and will save human effort and human lives with greater adoption.
6 Limitations
This work tackles a gap current literature wherein, prior work in adversarial search ignores visi- bility risk when solving for efficient search ignores efficient search when designing stealth algo- rithms. We utilize Depth Elevation Maps since our primary use case is open outdoor environments. 3-D structures that breakdown the 2-D assumption like caves or cliff overhangs are failure cases. We assume a symmetric sensing model for the targets and the agents, while this is realistic as it requires no special knowledge, it can be improved by incorporating a directionality to the assumed target sensing model, possibly by incorporating a movement model since STAR currently assumes static targets.
8

---

Acknowledgments
This material is based upon work supported by the U.S. Army Research Office and the U.S. Army Futures Command under Contract No. W911NF-20-D-0002. Authors would like to acknowledge the contributions of Conor Igoe, Tejus Gupta and Arundhati Banerjee to the development of the STAR algorithm. Additionally the work on the physical platforms and true-to-life simulations were enabled thanks to Herman Herman, Jesse Holdaway, Prasanna Kannappan, Luis Ernesto Navarro- Serment, Maxfield Kassel and Trenton Tabor.
References
[1] C. Mouradian, J. Sahoo, R. H. Glitho, M. J. Morrow, and P. A. Polakos. A coalition forma- tion algorithm for multi-robot task allocation large-scale natural disasters. 2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC), pages 1909-1914, 2017.
[2] D. S. Drew. Multi-agent systems for search and rescue applications. Curr Robot Rep, 2(2): 189-200, 2021.
[3] K. Nagatani, Y. Okada, N. Tokunaga, S. Kiribayashi, K. Yoshida, K. Ohno, E. Takeuchi, S. Ta- dokoro, H. Akiyama, I. Noda, T. Yoshida, and E. Koyanagi. Multirobot exploration for search and rescue missions: A report on map building in RoboCupRescue 2009. J. robot., 28(3): 373-387, 2011.
[4] J. I.. Baxter, E. K. Burke, M. Garibaldi, and M. Norman. Multi-Robot Search and Rescue: A Potential Field Based Approach, pages 9-16. Springer Berlin Heidelberg, Berlin, Heidelberg, 2007. ISBN 978-3-540-73424-6.
[5] Greenwood, Nelson, and P. G. Greenough. Flying into the hurricane: case study of uav use in damage assessment during the 2017 hurricanes in texas and florida. PLoS one, 15 0)ee0227808-e0227777
2020. ISSN 1932-6203.
[6] R. Nowakowski and P. Winkler. Vertex-to-verre pursuit in a graph. Discrete Mathematics, 43 (2):235-239, 1983, ISSN 0012-365X.
[7] M. Aigner and M. Fromme. A game of cops and robbers. Discrete Applied Mathematics, 8(1): 1-12, 1984. ISSN 0166-218X.
[8] A. S. Goldstein and M. Reingold. The complexity of pursuit on graph. Theor. Comput. Sci., 143:93-12 112, 1995.
[9] V. Isler, S. Kannan, and S. Khanna. Randomized pursuit-evasion with local visibility. SIAM j. discrete math., 20(1):26-44) 2006.
[10] J. Buermann and J. Zhang. Multi-robot adversarial patrolling strategies via lattice paths. Arti- ficial Intelligence, 311:103769, 2022. ISSN 0004-3702.
[11] L. Zhou, V. Tzoumas, G. J. Pappas, and Tokekar. Resilient active target tracking with multiple robots. IEEE Robotics and Automation Letters, 4(1):129-136; 2019.
[12] L. Zhou and Kumar. Robust multi-robot active target tracking against sensing and commu- nication attacks. In 2022 American Control Conference (ACC), pages 4443-4450, 2022.
[13] I. A. Sucan, M. Moll, and L. E. Kavraki. The Open Motion Planning Library. IEEE Robotics & Automation Magazine, 1944)722-8, December 2012.
[14] E, Hart, N. Nilsson, and Raphael. formal basis for the heuristic determination of minimum cost paths. IEEE Transactions Systems Science and Cybernetics, 4(2):100-107, 1968.

---

[15] K. Kandasamy, A. Krishnamurthy, J. Schneider, and B. Poczos. Parallelised bayesian optimi- sation via thompson sampling. A. Storkey and F. Perez-Cruz, editors, Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, volume 84 of Proceedings of Machine Learning Research, pages 133-142. PMLR, 09-11 Apr 2018.
[16] R. Ghods, W. Durkin, and Schneider. Multi-Agent active search using realistic Depth- Aware noise model. CoRR, abs/201 1.0825, 2020.
[17] R. Ghods, A. Banerjee, and J. Schneider. Decentralized multi-agent active search for sparse signals. In C. de Campos and M. H. Maathuis, editors, Proceedings of the Thirty-Sevents Conference on Uncertainty in Artificial Intelligence, volume 161 of Proceedings of Machine Learning Research, pages 696-706. PMLR, 27-30 Jul 2021.
[18] N. A. Bakshi, T. Gupta, R. Ghods, and J. Schneider. Guts: Generalized uncertainty-awee thompson sampling for multi-agent active search. Robotics and Automation (ICRA), 2023. To appear.
In IEEE International Conference on
[19] K. Kandasamy, W. Neiswanger, R. Zhang, A. Krishnamurthy, Schneider, and B. Poczos. Myopic posterior sampling for adaptive goal oriented design of experiments. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97, pages 3222-3232, PMLR, 2019.
[20] G. Hollinger, Kehagias, and S. Singh. Gsst: Anytime guaranteed search. Auton, Robots, 29 (1):99-118, jul 2010. ISSN 0929-5593.
[21] G. Hollinger, S. Singh, and A. Kehagias. Improving the efficiency of clearing with multi-agent teams. The International Journal of Robotics Research, 29(8):1088-111 2010.
[22] A. Kehagias, G. Hollinger, and S. Singh. A graph search algorithm for indoor pursuit/evasion. Mathematical and Computer Modelling, 50(9):1305-13) 2009. ISSN 0895-7177.
[23] A. Kleiner and A. Kolling. Guaranteed search with large teams unmanned acrial vehicles. 2013 IEEE International Conference on Robotics and Automation, pages 2977-2983, 2013.
[24] R. J. Marcolte, A. Haggenmiller, G. Ferrer, and E. Olson. Probabilistic multi-robot search for adversarial target.
[25] D. Assaf and S. Zamir. Optimal Sequential Search: A Bayesian Approach. The Annals of Statistics, 13(3):1213 1221, 1985.
[26] F. Bourgault, A. Göktogan, T. Furukawa, and H. F. Whyre Coordinated search for lost target in a bayesian world. Advanced Robotics, 18(10):979-1000, 2004.
[27] E.-M. Wong, Bourgault, and T. Furukawa. Multi-vehicle bayesian search for multiple lost targets. Proceedings of the 2005 IEEE International Conference on Robotics and Automa- tion, pages 3169-3174, 2005.
[28] G. Kimeldorf and F. H. Smith. Binomial searching for random number of multinomially hidden objects. Management Science. 25(11):1115-11 11261, 1979.
[29] N. Roy, G. Gordon, and S. Thrun. Finding approximate pomdp solutions through belief com- pression. J. Artif. Int. Res., 23(1):1-40, jan 2005. ISSN 1076-9757.
[30] Chung, G. A. Hollinger, and V. Isler. Search and pursuit-evasion mobile roboties: A survey, Auton. Robots, 3144)2299-316 2011.
[31] D. W. Schuldt and J. A. Kurucar. Efficient partitioning of space for multiple UAS search in an unobstructed environment. Int. J. Intell. Robot. Appl., 2(1):98-109 109, 2018.
10

---

[32] B. Baker, I. Kanitscheider, T. Markov, Y. Wu, G. Powell, B. McGrew, and I. Mordatch. Emer- gent tool use from multi-agent autocurricula. In International Conference on Learning Repre- sentations, 2020.
[33] A. Teng, D. DeMenthon, and L. Davis. Stealth terrain navigation. Systems, Man and Cyber- netics, IEEE Transactions on, 23:96 110, 02 1993.
[34] A. Tews, G. Sukhatme, and M. Mataric. A mobti-robot approach to stealthy navigation in the presence of an observer. JEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004, volume pages 2379-2385 Vol.3, 2004.
[35] B. McCue and National Defense University Press. U-boats in the bay of Biscay: An essay in operations analysis. National Defense University Press, 1990.
[36] W. Al Enezi and C. Verbrugge. Skeleton-based multi-agent opponent search. In 2021 IEEE Conference on Games (CoG), pages 1-8. IEEE, 2021.
[37] D. Isla. Third eye crime: building a stealth game around occupancy maps. In Proceedings of the Ninth AAAI Conference Artificial Intelligence and Interactive Digital Entertainment, page 206. AAAI Press, 2013.
[38] G. Hollinger, S. Singh, J. Djugash, and A. Kehagias. Efficient multi-robot search for a moving target. Int. Rob. Res., 28(2):201-219, 2009.
[39] Buermann and J. Zhang. Multi-robot adversarial patrolling strategies via lattice paths. Arti- ficial Intelligence, 31 111:76,,,,, 2022. ISSN 0004-3702.
[40] R. Vidal, O. Shakernia, H. Kim, D. Shim, and S. Sastry. Probabilistic pursuit-evasion games: theory, implementation, and experimental evaluation. IEEE Transactions on Robotics and Automation, 18(5):662-669, 2002.
[41] J. Tisdale, Z. Kim, and J. K. Hedrick. Autonomous uav path planning and estimation. 1EEE Robotics & Automation Magazine, 16(2):35 4,.35-42,222 2009.
[42] B. P. Gerkey, S. Thrun, and G. Gordon. Parallel stochastic hill- climbing with small teams. In L. E. Parker, F. Schneider, and A. C. Schultz, editors, Multi-Robot Systems. From Swarms to Intelligent Automata Volume III, pages 65-77, Dordrecht, 2005. Springer Netherlands. ISBN 978- 778-1-4020-3388-6.
[43] M. Vieira, R. Govindan, and G. Sukhatme. Scalable and practical pursuit-evasion with net- worked robots. Intelligent Service Robotics, 2:247-263, 10 2009.
[44] M. Tipping. Sparse bayesian learning and the relevance vector machine. Mach. Learn. Res., 1:211-22 1244, sep 2001. ISSN 1532-4435.
[45] D. Wipf and B. Rao. Sparse bayesian learning for basis selection. IEEE Transactions on Signal Processing, 52(3):2153-2164 2004.
[46] Moon. The expectation 47 67, 1996.
popeetationmmaximizatiin maximization algorithm. IEEE Signal Processing Magazine, 13(6):
[47] Y. Ma, R. Garnett, and J. Schneider. Active search for sparse signals with region sensing. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1), Feb. 2017.
[48] Wang, G. Robinson, and K. White. Generating viewsheds without using sightlines. 66, 2000.
[49] S. D. Roth. Ray casting for modeling solids. Computer Graphics and Image Processing, 18
(2): 144,144, 1982. ISSN 0146-664X.

---

[50] M. F. Goodchild and J. Lee. Coverage problems and visibility regions on topographic surfaces. Annals of Operations Research, 18(1):175-186, Dec 1989. ISSN 1572-9338.
[51] M. R. Travis, G. H. Elsner, W. D. Iverson, and C. G. Johnson. VIEWIT: computation of seen areas, slope, and aspect for land-use planning. General Technical Report PSW-GTR-11, Pacific Southwest Research Station, Forest Service, U.S. Department of Agriculture, Berkeley, CA, 1975.
[52] Fisher. First experiments in viewshed uncertainty: The accuracy of the viewshed area. Photogrammetric Engineering & Remote Sensing, 57, 01 1991.
[53] Fisher. Extending the applicability of viewsheds in landscape planning. Photogrammetric Engineering and Remote Sensing. 52:1297-1302, 1996.
12

---

A Appendix
A.1
Glossary of Notations
Notation
Description
B E IR
Sparse matrix representing the locations of targets in a 2D grid Flattened version B, sparsely populated vector Total number of cells in the grid
B
RM
M M2
y/
Observation vector for robot at timestep
X
Sensing matrix for robot j at timestep Total number of grid cells visible to robot
Q
by
Terrain and depth-aware noise vector for robot j timestep
((L))
Penalty function penalizing robot j for showing itself to targets given a path of traversal One-hot vector indicating the location of robot j
(
D
The reward term in the optimization objective The stcalth penalty term in the optimization objective given the potential location of the robot Sensing matrix for the kdh target
P(
X
D/
Set of observations available to robot j at timestep Total number of sensing actions by all agents
T
D
Set of observations available robot j timestep
B
Sampled set of target locations
r Ym
Diagonal matrix of hidden variables, estimated using data
Variance for the m² entry of ß
am
Parameter of the inverse gamma prior on Yn
bm
Parameter of the inverse gamma prior on Yn
Mean of the posterior distribution p(B D r) Variance of the posterior distribution p(B D Matrix formed by stacking all sensing actions in D Diagonal matrix of terrain-aware noise variances Hyperparameter for the reward function
V
X
Σ
Expected estimate of B using all available measurements and the next candidate measurement Number of non-zero entries in ß Number of non-zero entries in B
&
&
(B.B B)
Indicator function for comparing entries in B and
Hyperparameter controlling the tradeoff between goal selection and the stealth penalty
a vir d (2))
Posterior mean for visibility risk
Error function
K
The number targets in a search run
J
The number of search agents participating in search The number of targets found during a run
F
A.2
Algorithm Pseudo-code
The algorithm has been summarized Alg.
A.3 Real Hardware Performance
The STAR algorithm has been tested on the physical systems shown in the demo vidco?. In the main paper we ablated and assessed the performance of the STAR algorithm against the state of the art from an algorithmic standpoint to show statistical significant superiority. Here, we provide some statistics from running the algorithm on physical systems.
hhtps://youtu.ber be//ttubbbbe//l//eeeeeee 11v456/5
13

---

Algorithm STAR Algorithm
Assume: Sensing model (I), sparse signal B. J agents Set: D/ < 0
Lg {x,,)}} V jE (1, Y Vm E M)
for = l...... do
Wait for agent to finish; for the free agent
Sample p(BID (H, V) from (4)
x, arg maxx.í (B, (/D - from (6) and (8)
Observe y, given action X;
Update D D U vx, (robot observations)
Update L +1 = L (L) (robot path)
Share (x) Estimate diag (angg(().... (I,...., Yvl) using (5)
end for
Computational Performance of STAR
Bust fit curvi
30-
10-
0.0
0.5
1.0
1.5
2.4
2.5
search Magioon AFIO (km
Figure 6: Planning time Search region Size
The Fig. 6 shows a plot of the planning time for one decision of the STAR algorithm against the size of the search space. The planning time in search regions under a sq km is around 10-15 secs. At 2.5 sq. km (search region size in the paper), rises over a minute. The compute on the robot is luvo-8108Ge with Intel Xeon E-2278GEL (Coffee Lake R) 2.0 GHz Processor. In practice the robot may start planning its next decision slightly before it expects to arrive its next goal location so this planning time doesn't 't impact search performance. To isolate such engineering optimisations from algorithmic assessment we evaluated the algorithms on their sample complexity rather than wall clock time.
A.4 Terrain Visibility Prior
Since our use case is outdoor spaces we use Depth Elevation Maps (DEMs) to represent the terrain (See Fig. 4) since it is more memory efficient than voxels.
To determine what portion of the map visible given location and direction of facing we use reference basee approach [48] which can compute the viewable region in the map given any point in constant time as opposed ray casting methods [49, 50, 51, 52, 53] which takes variable time. We assume that the topography remains unchanged over the course of the run; however, our physical systems are capable of dynamically updating the topography using point clouds generated by stereo cameras. Hence, having a constant time algorithm for viewshed computation allows for efficient onboard updating of the visibility map if there are differences between the terrain prior and the dynamic observations made by the robot on the ground.
Once we have the viewable region from given point the map discretise it and apply viewa- bility limits in accordance with physical system as shown in Fig. 3 and described ahead.
14

---

A.5 UGV Sensing Action model
We use array of 5MP RGB cameras with an effective lateral field-of-view (FOV) of 193" for the ground vehicles. This allows the pereeption system to pick up detections several hundred metres out.
We model the sensing action model in the grid representation as a trapezium of fifteen cells along the bearing of the UGV as shown in Fig. 2a. Its full extent is upto 210m - 300m in front of the robot subject to occlusions. The motivation behind this is that beyond certain distance even if the terrain line sight, it is not possible to make accurate detections of targets as they are just a few pixels the image.
A.6 Target Sensing Action Model
Fig. 2b shows representative example of the viewshed of the targets. Since don't have informa- tion the direction of facing of the targets, model the FOV such that targets in all directions subject to the topography and the 210m 300m viewing limit but without depth aware noise. Fig. shows an example of the viewshed computed at an example location in the map desert mountainous map (Fig. 1) assuming a 360 FOV.
A.7 Visibility Risk Aware Path Planning
Since our robot and target viewing models are symmetric, it implies that detecting a target is ac- companied by the target detecting the search agent, however being identified once does mean the task is over, there could be more targets to locate and known targets should be avoided for the remainder of the search. We expect to minimize the stealth penalty over the course of the run but don't expect it to be zero. As an aside, were we to employ asymmetric viewing models such that viewing targets without being viewed was possible, we might aim to have zero risk policies but we outline this for future work.
In order for the search agents to respect the visibility risk map when path planning (See Fig. 1c), use the OMPL planner [13] the physical system and for the realistic simulation and the A-star planner [14] for our simplified simulations. Both planners can plan paths within time constraints and subject to state costs, which in our case is the visibility risk map, and an occupancy map of obstacles.
15


Reviews:
[{"review_id": "tMdOdENFYX", "reviewer": "Reviewer_o4rp", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes an algorithm (STAR) that enables a group of agents to actively search in a specified area for targets without being spotted by these targets. The state space is defined as a grid world, where STAR controls the agents in a decentralized manner and is optimized with a multi-objective parallelized Thompson sampling-based algorithm. The paper presents simulated results to show that STAR is effective and better than baselines.", "summary_of_recommendation": "As is stated in the review section, due to\n1. Confusing setups and missing assumptions in the method section\n2. Insufficient experiments to prove the claimed generalization capability\n3. Limited interested to robot learning researchers\n\nI suggest rejecting this paper before all the concerns are addressed.\n", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "I have listed my questions as major concerns above.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I find there to be substantial room for improvement in the quality and clarity of this work.  \n\n**Major Concerns**\n1. Is the method correct? There are confusing setups and missing assumptions. For example, in Sec 3 $\\beta$ is defined to be vector of 0s and 1s (line 112), but then it is modeled with a zero-mean gaussian in Sec 4.1 (line 152). As another example, the paper assumes decentralized control, but Eq (4) contains stacked measurements from all agents in Sec 4.1 (line 162) without stating when and how those agents exchange information. Both examples (and there are others) lead me to doubt the correctness of the method.\n2. How generalizable is the method? In Sec 2, the paper argues that *\"... it is not clear if these RL methods will generalize to different topographies\"*. However, the paper does not give results on different terrains either. \n3. How useful is this paper to robot learning researchers? In general, this paper can be presented as proposing a multi-agent exploration task in a grid world, and the stealthy search is one of the applications. A CoRL reader may expect more robot and real-world related components which are currently missing in the reviewer's opinion. For example, depending on the terrain, a vehicle may not be able to visit all the area and may turn over so that it loses mobility, will this affect agent's planning behavior? Lastly, I find the statement in the introduction *\"... Our motivation for only presenting simulation results in this paper is to ablate and assess the performance of the STAR algorithm ...\"* not convincing.\n\n**Other Concerns**\n1. The paper uses a large number of notations and equations that are cluttered and redundant. E.g., Eq (3) and Eq (8) are the same thing.\n2. Figures need to be modified and rearranged. Most of the captions and legends are too small to be viewed clearly.", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "mUEa-NjjG0J", "reviewer": "Reviewer_ajuq", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper studies the problem of multi-agent active search. The problem is set to have multiple agents searching for sparsely placed targets, which can be adversarial. Hence, the setting motivates the search strategy to hide the agents from the targets by leveraging the prior information on the terrain. The authors propose a new algorithm named Stealthy Terrain-Aware Reconnaissance (STAR), which is a multi-objective optimization algorithm following the myopic posterior sampling framework that has been studied since 2018. The algorithm helps the agent to search with better decisions with the Thompson sampling technique, which is robust to communication or hardware failures. An ablation study of the STAR algorithm's performance is conducted in simulations by varying several factors, in which STAR shows better performance than existing methods. Experiments using ground robots were also conducted to demonstrate the performance.", "summary_of_recommendation": "Again, I'm not an expert in this domain. My recommendation is based on my limited understanding of this paper and the existing work.\n\nupdated on Aug. 18: The statement of contributions has been clearly made now. But since I'm not an expert in this domain, I choose weak accept to let the other reviewers' opinions weigh in.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "What is the contribution of this paper compared to existing work?\n\nupdated on Aug. 18: the questions have been clarified by the authors. Scores are updated.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "well I'm not an expert in the domain of multi-agent search. To me, it seems the ablation study should be conducted to see how much the prior helps the proposed problem. The reported ablation study over different map types, reliability of communications, number of search agents, and whether the targets are placed adversarially seems more like evaluating the proposed algorithm over other baselines in different settings. Please use appropriate words to describe the results. \n\nAlso, it is not clear to me what this paper's contribution is. If using the prior terrain information is novel, then has previous research used this information in other papers? If so, how is the proposed algorithm compared to those? If not, then the authors should state that this is the first time that terrain information is used for a stealthy target search algorithm.", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "LaLMcnjR-2G", "reviewer": "Reviewer_AFR3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper studies how multiple mobile robots can stealthily detect immobile targets.  A decentralized algorithm is proposed that relies on a prior of where the targets are, and on a given topological map that quantifies the risk for the robots to be visible from a target.  The number of targets is assumed to be unknown.  Extensive simulations evaluate the performance of the algorithm.", "summary_of_recommendation": "The paper proposes an excellent problem and a method that appears promising.  In my opinion, the current version of the paper would benefit from discussing the practicality of the assumptions and from elaborating on the rigorous performance of the proposed algorithm given the design choices. \n\nUPDATE: Given the rebuttal, I am updating my score to Weak Accept.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "Please see all my comments in the weakness section above, repeated here:\n\n\n-  I would discuss in more detail the necessary assumptions and their impact on the practicality of the algorithm:\n\n(i)  Although the number of targets is unknown a priori, it seems that the algorithm needs (i-a) a prior on the targets' location, and (i-b) knowledge of their sensing matrix.  I would discuss in the paper how (i-a) and (i-b) can be obtained. \n\nAlso, I would discuss these assumptions in the introduction and in the problem formulation to help guide the reader on what the problem and its limitations are.\n\n(ii)  To my understanding, the algorithm also needs a topological map that quantifies the risk for the robots to be visible from a target.  Similarly to the previous comment, I would explain how such a map can be obtained when the targets are unknown a priori.\n\n-  I would define some used terms/notions:\n\n(i)  I would elaborate on what \"information leakage\" means.  \n\n(ii)  In eq. (1), I would define what the clip operator does immediately after eq. (1).\n\n(iii)  I would define what a \"one-hot vector\" is.\n\n(iv)  I would define what \"recall\" means in Figure 6. \n\n-  I would elaborate on design choices, to help the reader understand the practicality and potential impact of the algorithm:\n\n(i)  I would explain why the observation model is clipped to be between 0 and 1.\n\n(ii)  I would discuss what is the intuition behind the reward in eq. (6). \n\n(iii)  I would discuss under what assumptions there are performance guarantees.\n\n-  I would discuss in more detail the differences of this work from existing works, in particular [27, 28, 29].  In lines 87-89, the comparison is through the statement \"However, these approaches focus on path planning but not on a competing search objective.\"  I understand from this that the difference between [27, 28, 29] and this paper is that this paper uses a different objective function.  I would present the comparison from a more qualitative perspective:  Will the algorithms in [27, 28, 29] not perform well in the proposed setting in this paper, and why? In what scenarios they will not be able to be applied but the proposed algorithm will be?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths\n\n-  The problem is well-motivated and of relevance to the CoRL community.\n-  The simulations are extensive, based on a realistic simulator environment.\n\nWeaknesses\n\n-  I would discuss in more detail the necessary assumptions and their impact on the practicality of the algorithm:\n\n(i)  Although the number of targets is unknown a priori, it seems that the algorithm needs (i-a) a prior on the targets' location, and (i-b) knowledge of their sensing matrix.  I would discuss in the paper how (i-a) and (i-b) can be obtained. \n\nAlso, I would discuss these assumptions in the introduction and in the problem formulation to help guide the reader on what the problem and its limitations are.\n\n(ii)  To my understanding, the algorithm also needs a topological map that quantifies the risk for the robots to be visible from a target.  Similarly to the previous comment, I would explain how such a map can be obtained when the targets are unknown a priori.\n\n-  I would define some used terms/notions:\n\n(i)  I would elaborate on what \"information leakage\" means.  \n\n(ii)  In eq. (1), I would define what the clip operator does immediately after eq. (1).\n\n(iii)  I would define what a \"one-hot vector\" is.\n\n(iv)  I would define what \"recall\" means in Figure 6. \n\n-  I would elaborate on design choices, to help the reader understand the practicality and potential impact of the algorithm:\n\n(i)  I would explain why the observation model is clipped to be between 0 and 1.\n\n(ii)  I would discuss what is the intuition behind the reward in eq. (6). \n\n(iii)  I would discuss under what assumptions there are performance guarantees.\n\n-  I would discuss in more detail the differences of this work from existing works, in particular [27, 28, 29].  In lines 87-89, the comparison is through the statement \"However, these approaches focus on path planning but not on a competing search objective.\"  I understand from this that the difference between [27, 28, 29] and this paper is that this paper uses a different objective function.  I would present the comparison from a more qualitative perspective:  Will the algorithms in [27, 28, 29] not perform well in the proposed setting in this paper, and why? In what scenarios they will not be able to be applied but the proposed algorithm will be?", "extra_scores": {"impact": "2: Unsure, difficult to assess."}, "extra_text": {}}, {"review_id": "3q-0OBSCSW", "reviewer": "Reviewer_b4aX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper a multi-agent active search problem of stationary targets while minimizing exposure to them. The setting also considers the terrain during exploration. The solution must minimize the time to locate the targets with high certainty, and work in a decentralized way with varying degree of communication between agents. The proposed approach is based on maintaining the posterior distribution of environment modeled as a rectangular grid. The prior distribution is taken as zero mean normal distributions independent across cells and with variances distributed according to a conjugate inverse gamma prior to enforce sparsity. The expectation maximization method is used to update it. Thompson sampling of target locations is used by agents to compute their next actions. It optimizes a linear combination of two objectives that trade-off exploration and stealthiness.", "summary_of_recommendation": "The paper considers an important and interesting problem in robotics. It presents a practical approach in the form of a decentralized active exploration method. The paper requires clarification of several technical aspects and improve notation for clarity.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "- Notation is at times ambiguous. The sensing matrix for target and robots use the same symbol while the subscript has different meaning in the two settings. The subscript represents time in one instance, and a target index in the other. See equation (8) as an example.\n-In Sec.3, can a component of vector v_t be zero? If it does, then it would lead to division by zero in equation (2).\n- It is unclear if agents know the targets’ sensing matrices. In case they do not, then how is the visibility objective in equation (8) is computed. The approach only samples the targets’ locations. Does it also sample the targets’ sensing matrices?\n- It is not clear how agent motion constraints are captured by the model. The agents’ motion model is not even described.\n- Is the computational effort growing over time for the EM and action selection procedures?\n- What is the runtime performance of the algorithms per agent action? How does the communication impact performance as a function of update frequency (number of received packages from other agents per time unit)?\n- In the communication model, do agents swap raw observation data? Are data packages sufficiently small for the approach to be practical?\n- Why is the runtime budget 1 hour and 15 minutes?\n- It is unclear from the last sentence if zero-risk search is possible using the proposed method.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Strengths:\n+ Good motivation of exploration setting.\n+ The paper introduces a multi-agent stealthy decentralized active learning method. The contributions are significant.\nWeaknesses:\n- Notation is at times unclear.\n- The agents’ motion model is not described clearly.\n- Runtime performance is unclear.\n- Agent communication model is unclear.", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors' response has addressed most of the reviewers' concerns, and the general assessment is that this paper provides a good contribution to CoRL. In the final version of this paper, the authors should carefully revise the paper to address all reviewers' comments / suggestions.
Decision: Accept (Poster)