Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: In this paper, the authors study the multi-agent navigation problem and propose a control admissibility model (CAM) to output and compose the action instead of directly approximating the optimal policy. To this end, the navigation problem is represented by a graph, and graph neural network (GNN) is employed to compute the admissibility score. The CAM is validated on three navigation benchmarks.
strengths_and_weaknesses: This paper is well-written and easy to follow. The authors propose a simple but potentially effective framework, which breaks down the difficult overall optimal policy learning to multiple admissibility policies. However, the main contributions are not clear. First, the authors mainly introduce two multi-agent navigation difficulties, i.e. ad hoc reward engineering and poor generalization capacity. But why the GNN and decomposition-based CAM can deal with the above challenges is unclear. Second, the proposed method seems independent of the multi-agent system. In the multi-agent navigation problem, what are the key problems can CAM tackle? Additionally, the experiments are less convincing since only one multi-agent navigation baseline is compared. 
robotics_focus: Relevant but unlikely to deploy to hardware in near future
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Although the proposed method is clear and seems potentially effective, the key multi-agent navigation problem to be solved and the empirical results are not clear. Therefore, I suggest rejection of the current version.
issues: 1. In the graph decomposition stage, the sub-graph will be different. How can a single GNN-based CAM deal with graphs with different vertices and edges?

2. In algorithm 1, it shows that CAM does not use reward to guide learning, which can also be seen from Eq. 2. How can CAM learn to navigate if no reward, i.e. goal information provided?

3. What is the influence of three key hyper-parameters in Eq. 2?

4. In Eq. 2, the authors claim the third term enforces the continuity of the CAM values on two consecutive transitions. How to understand it?

5. In the experiments, why the single agent algorithm DDPG outperforms the multi-agent algorithm baseline MACBF?
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes learning a model of admissible controls that would avoid violating specified constraints in an MDP.  The paper calls these models Control Admissibility Models (CAMs), and demonstrate how these can be easily composed to provide a set of admissible actions at any state.  The CAMs are built using a Graph Neural Network, and trained using a time-based back-propoagation, where action that bring into a state that has no admissible actions, even if the state itself is admissible, are also included in the non-admissible state set.

Experiments are performed on a UR5 multi-arm task, a multi-agent driving task, a point-mass chasing task and a drone flight task.  Experiments show significant ability to maintain safety conditions as the number of agents increases, especially in the Car setting.
strengths_and_weaknesses: This paper presents an interesting approach to dealing with dense multi-agent systems, and seems to have promissing results although I'm not an expert in this area and so don't have much experience in what is achievable.

One big weakness however is that I couldn't figure out how the paper uses reward from the environment.  It's clearly taken into account otherwise nothing would work, but the paper seems to skip over this important point or I somehow missed it after multiple re-reads, in which case this should definitely be fixed. 

I would also appreciate a small mention of the inference time of this setup, to understand to what types of systems it could be deployed (i.e. up to what control frequency could it likely function).
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I think this method is easily applicable to real systems, and provides a believable story on how to deal with a large number of interacting agents, which is an important problem in robotics.  
issues: I have mentioned above one blocking issue for me is the discussion of the use of reward, although I recommend an acceptance it is absolutely contingent on addressing this point.  I would also like a brief mention of inference time for the method, as this can drastically impact the types of systems on which it can run.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The authors proposed a strategy to compute admissible sets for multi-agent systems, i.e., a set of actions that agents can take without compromising safety. These sets are computed using GNN which naturally takes into account interaction between different agents. For training, each state-action pair is labeled based on the state of the system and then a relabeling state is applied to make sure that wrong actions that led to unsafe regions are not labeled as safe. Finally, the authors showed that the proposed approach trained in scenarios with only a few agents generalizes to settings where several agents are interacting. 
strengths_and_weaknesses: Strengths
1) Using a GNN for multi-agent navigation makes sense and enables scalability
2) The re-labeling strategy allows the authors to catch failure early on

Weaknesses
1) The authors are basically trying to compute invariant sets. It would be nice to introduce a formal definition, clearly state the objective, and provide an analysis (if possible). 
2) From the first example, it is unclear if the method is actually able to approximate forward invariant sets, even in the very simple example shown in the experiment section.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I think that the paper is well-written and that the method is well-designed. The objective of this work is pretty ambitious as the authors are approximating invariant sets. Clearly, no guarantees can be provided for this problem, so the authors performed a set of experiments to empirically show that the method is working. I think that further experiments are needed. In particular, I would like to see the method applied to high inertia systems where the control action is saturated, see the Issues section for further details.
issues: The method is well presented, but it has no theoretical guarantees (this is totally fine as computing invariant sets is very hard already for a single agent!). So I focused on the examples and I would like to see a few changes to confirm that the method is working:
1) What is the model used for the single agent environment? I would like to see an agent that has a lot of inertia, e.g., a car where you control the acceleration rate and the maximum acceleration rate is saturated. This would allow us to understand if the re-labeling strategy is working and if you are able to catch failure early on.
2) Figure 3 should have some information about the velocity. If you are on the boundary of the set with an outside pointing velocity vector the set is not invariant. It would be nice to see if the set that you are computing is actually an invariant (or a good approximation)
3) Why the set in Figure 3 is not symmetric? As we know that subsets to invariant sets are not invariant, it is unclear if the set shown in Figure 3 is forward-invariant.
4) In the Dubins car examples, are you controlling the velocity? It would be more convincing to control the acceleration and to have a saturation on the acceleration. For the dynamics, you can simply assume that the velocity is the integral of the acceleration. If you are currently controlling the velocity, could you repeat the experiments when controlling the acceleration? I expect to see more collisions.
review_title: null

Meta-review: metareview: The reviewers appreciated the well-written paper and pointed out the ambitious overall objective. They found the proposed method to be an interesting and well-designed approach for dealing with dense multi-agent systems. However, they also raised a number of issues. To address these, the authors should try to improve the explanation of why their approach is actually suitable to tackle the major challenges associated with multi-agent navigation, i.e. why the method is better able to address ad-hoc reward engineering and poor generalization capacity. In addition, it should be clarified, how rewards from the environment are actually taken into account in the approach. To judge applicability, it would be beneficial to mention inference times of the algorithm. Finally, more experimental results would be needed to confirm that the method is working, and to judge how well the method approximates invariant sets. Please note the detailed lists of issues and suggestions of reviewers JipF and XUE1 in this context.

===

Post rebuttal/discussion update:

The authors provided substantial further input and clarifications, including additional experiments which addressed the main concerns of the reviewers. The presented method appears to be a useful and novel contribution for the community.
confidence: 3: The area chair is somewhat confident
Decision: Accept (Poster)