Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper a multi-agent active search problem of stationary targets while minimizing exposure to them. The setting also considers the terrain during exploration. The solution must minimize the time to locate the targets with high certainty, and work in a decentralized way with varying degree of communication between agents. The proposed approach is based on maintaining the posterior distribution of environment modeled as a rectangular grid. The prior distribution is taken as zero mean normal distributions independent across cells and with variances distributed according to a conjugate inverse gamma prior to enforce sparsity. The expectation maximization method is used to update it. Thompson sampling of target locations is used by agents to compute their next actions. It optimizes a linear combination of two objectives that trade-off exploration and stealthiness.
review: Strengths:
+ Good motivation of exploration setting.
+ The paper introduces a multi-agent stealthy decentralized active learning method. The contributions are significant.
Weaknesses:
- Notation is at times unclear.
- The agents’ motion model is not described clearly.
- Runtime performance is unclear.
- Agent communication model is unclear.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper considers an important and interesting problem in robotics. It presents a practical approach in the form of a decentralized active exploration method. The paper requires clarification of several technical aspects and improve notation for clarity.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper studies the problem of multi-agent active search. The problem is set to have multiple agents searching for sparsely placed targets, which can be adversarial. Hence, the setting motivates the search strategy to hide the agents from the targets by leveraging the prior information on the terrain. The authors propose a new algorithm named Stealthy Terrain-Aware Reconnaissance (STAR), which is a multi-objective optimization algorithm following the myopic posterior sampling framework that has been studied since 2018. The algorithm helps the agent to search with better decisions with the Thompson sampling technique, which is robust to communication or hardware failures. An ablation study of the STAR algorithm's performance is conducted in simulations by varying several factors, in which STAR shows better performance than existing methods. Experiments using ground robots were also conducted to demonstrate the performance.
review: well I'm not an expert in the domain of multi-agent search. To me, it seems the ablation study should be conducted to see how much the prior helps the proposed problem. The reported ablation study over different map types, reliability of communications, number of search agents, and whether the targets are placed adversarially seems more like evaluating the proposed algorithm over other baselines in different settings. Please use appropriate words to describe the results. 

Also, it is not clear to me what this paper's contribution is. If using the prior terrain information is novel, then has previous research used this information in other papers? If so, how is the proposed algorithm compared to those? If not, then the authors should state that this is the first time that terrain information is used for a stealthy target search algorithm.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Again, I'm not an expert in this domain. My recommendation is based on my limited understanding of this paper and the existing work.

updated on Aug. 18: The statement of contributions has been clearly made now. But since I'm not an expert in this domain, I choose weak accept to let the other reviewers' opinions weigh in.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper proposes an algorithm (STAR) that enables a group of agents to actively search in a specified area for targets without being spotted by these targets. The state space is defined as a grid world, where STAR controls the agents in a decentralized manner and is optimized with a multi-objective parallelized Thompson sampling-based algorithm. The paper presents simulated results to show that STAR is effective and better than baselines.
review: I find there to be substantial room for improvement in the quality and clarity of this work.  

**Major Concerns**
1. Is the method correct? There are confusing setups and missing assumptions. For example, in Sec 3 $\beta$ is defined to be vector of 0s and 1s (line 112), but then it is modeled with a zero-mean gaussian in Sec 4.1 (line 152). As another example, the paper assumes decentralized control, but Eq (4) contains stacked measurements from all agents in Sec 4.1 (line 162) without stating when and how those agents exchange information. Both examples (and there are others) lead me to doubt the correctness of the method.
2. How generalizable is the method? In Sec 2, the paper argues that *"... it is not clear if these RL methods will generalize to different topographies"*. However, the paper does not give results on different terrains either. 
3. How useful is this paper to robot learning researchers? In general, this paper can be presented as proposing a multi-agent exploration task in a grid world, and the stealthy search is one of the applications. A CoRL reader may expect more robot and real-world related components which are currently missing in the reviewer's opinion. For example, depending on the terrain, a vehicle may not be able to visit all the area and may turn over so that it loses mobility, will this affect agent's planning behavior? Lastly, I find the statement in the introduction *"... Our motivation for only presenting simulation results in this paper is to ablate and assess the performance of the STAR algorithm ..."* not convincing.

**Other Concerns**
1. The paper uses a large number of notations and equations that are cluttered and redundant. E.g., Eq (3) and Eq (8) are the same thing.
2. Figures need to be modified and rearranged. Most of the captions and legends are too small to be viewed clearly.
robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: As is stated in the review section, due to
1. Confusing setups and missing assumptions in the method section
2. Insufficient experiments to prove the claimed generalization capability
3. Limited interested to robot learning researchers

I suggest rejecting this paper before all the concerns are addressed.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper studies how multiple mobile robots can stealthily detect immobile targets.  A decentralized algorithm is proposed that relies on a prior of where the targets are, and on a given topological map that quantifies the risk for the robots to be visible from a target.  The number of targets is assumed to be unknown.  Extensive simulations evaluate the performance of the algorithm.
review: Strengths

-  The problem is well-motivated and of relevance to the CoRL community.
-  The simulations are extensive, based on a realistic simulator environment.

Weaknesses

-  I would discuss in more detail the necessary assumptions and their impact on the practicality of the algorithm:

(i)  Although the number of targets is unknown a priori, it seems that the algorithm needs (i-a) a prior on the targets' location, and (i-b) knowledge of their sensing matrix.  I would discuss in the paper how (i-a) and (i-b) can be obtained. 

Also, I would discuss these assumptions in the introduction and in the problem formulation to help guide the reader on what the problem and its limitations are.

(ii)  To my understanding, the algorithm also needs a topological map that quantifies the risk for the robots to be visible from a target.  Similarly to the previous comment, I would explain how such a map can be obtained when the targets are unknown a priori.

-  I would define some used terms/notions:

(i)  I would elaborate on what "information leakage" means.  

(ii)  In eq. (1), I would define what the clip operator does immediately after eq. (1).

(iii)  I would define what a "one-hot vector" is.

(iv)  I would define what "recall" means in Figure 6. 

-  I would elaborate on design choices, to help the reader understand the practicality and potential impact of the algorithm:

(i)  I would explain why the observation model is clipped to be between 0 and 1.

(ii)  I would discuss what is the intuition behind the reward in eq. (6). 

(iii)  I would discuss under what assumptions there are performance guarantees.

-  I would discuss in more detail the differences of this work from existing works, in particular [27, 28, 29].  In lines 87-89, the comparison is through the statement "However, these approaches focus on path planning but not on a competing search objective."  I understand from this that the difference between [27, 28, 29] and this paper is that this paper uses a different objective function.  I would present the comparison from a more qualitative perspective:  Will the algorithms in [27, 28, 29] not perform well in the proposed setting in this paper, and why? In what scenarios they will not be able to be applied but the proposed algorithm will be?
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: The paper proposes an excellent problem and a method that appears promising.  In my opinion, the current version of the paper would benefit from discussing the practicality of the assumptions and from elaborating on the rigorous performance of the proposed algorithm given the design choices. 

UPDATE: Given the rebuttal, I am updating my score to Weak Accept.
review_title: null

Meta-review: The authors' response has addressed most of the reviewers' concerns, and the general assessment is that this paper provides a good contribution to CoRL. In the final version of this paper, the authors should carefully revise the paper to address all reviewers' comments / suggestions.
Decision: Accept (Poster)