Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper tackles the problem of training policies that control multiple robots simultaneously, and is specifically interested in the case of 0-shot generalization to novel robot team configurations. The proposed approach aims to use awareness and communication of robot capabilities in order to learn a more robust approach. The paper proposes to use a graph network where each node is a robot and each edge is a communication link. The robot's observation are augmented with a vector of its capabilities (payload, sensor, etc). The method is evaluated in sim and real robotarium task where the robots must form a dense sensor network, taking into account each robot's sensor radius. The results show that the graph network and capability communication help the policy generalise to new robot configurations
review: Strengths:
- The paper is written clearly.
- The experiment baselines are thorough
- The proposed method is sound, interesting, and to my knowledge this is a novel application to multi-robot policies. 

Weaknesses:
- The results are only on one kind of task, where only one of the capabilities matters. While this is a good proof of concept, it would be more impactful to push the method further with more complex task/capability relationships.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This is an interesting paper that investigates how enabling robots to use and communicate their capabilities enables generalization to novel robot teams. The evaluation is on a fairly limited set of environments, but the results pan out.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper presents a new algorithm for multi-agent reinforcement learning. The authors propose adding the “capability” of each robot as additional information in MARL, so that the system has capability awareness (each robot knows its own capability) and capability communication (each robot knows about the capabilities of the team). The authors show that this approach significantly improves team performance in a real-world multi-robot test bed (the Robotarium) when generalizing to different team compositions, robot capabilities, and numbers of robots.
review: Strengths: 
- The paper is very well-written and exceptionally clear.
- The motivation is compelling (generalizing to heterogeneity in multi-robot teams without re-training).
- The evaluation is thorough, with systematic investigation of the impact on training and generalization at test-time and a thorough set of baselines.
- Quantitative results are compelling, showing clear benefits of the proposed method.
- Qualitative results are compelling, showing real robots performing the task in accordance with intuition (maximizing surface area while maintaining connection).

Weaknesses:
- The main weakness is that the proposed approach has limited technical novelty. If I understand correctly, one could argue that the algorithm is equivalent to simply augmenting the robot state with the “capability” information. It is then not surprising that critical state information (the sensing radius in the experiment) is instrumental in solving the multi-agent task, and that the learned policy can generalize to different values of the augmented state. Treating this information as “capability” rather than state seems like mere semantics.
- The task is relatively simple in nature and in a single domain. As the authors acknowledge in the limitations, they only test the case where the capability vector is a scalar. The robots are also all identical in the experiment (in terms of hardware rather than capability).
- It seems that while the policy is able to generalize to new *values* of capabilities, it’s unable to generalize to entirely new capabilities, since the capability space C must be known beforehand.

Minor notes:
- “Heterogeneity” is later defined as variation in robot behavior and hardware, but it could be helpful to define this term upfront in the introduction.
- Some additional experiments that are not necessary but could be helpful include (1) ablations justifying the CTDE paradigm and (2) testing the impact of variations in the communication graph structure.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: **Post-Rebuttal Update**: I am keeping my score as a Weak Accept. The authors' rebuttal acknowledges the lack of algorithmic novelty w.r.t. augmented state information.

The paper is well-executed, well-written, and technically sound, but I would like the authors to clarify how their approach differs from augmented state information.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper focuses on improving the generalization of multi-agent RL in terms of different team compositions, team sizes, and different robot capabilities. The author proposes to include robots’ capability in the observation space and use graph networks to enhance communication. The experiments show that the proposed method is effective in improving the generalization of the baselines.
review: Strengths:
* The paper is overall clearly written.
* Improving generalization across heterogeneous robots is an important topic.
* The experiments are pretty comprehensive, including real robot results from Robotarium.

Weaknesses:
* As mentioned in the limitation section, only one dimension of robot capability is tested. 
* The figures in the experiments need improvement - see below for details. 
  * The legends and labels in Figure 3,4,5 are way too small.
  * How is the variance/std calculated in the figures?
  * Figures should be placed closer to the text that describes the corresponding figures.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper presents a method for improving generalization in multi-agent RL. The contribution is relatively minor but it does show improvement against reasonable baselines.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: The paper investigates whether communication of robot capabilities can improve the generalization of RL-based multi-robot policies to different robot-team sizes and robot-type compositions.  To this end, the paper develops a decentralized policy architecture with parameter sharing that allows robots to use their capabilities as implicit role indicators within the multi-robot team.  The role of the policy is to enable the robots to reason about individual robots' capabilities and team capabilities. The proposed approach is evaluated in both simulator and hardware experiments of active sensing tasks with multiple robots.
review: Strengths

-  The paper identifies a learning problem of interest to the CoRL community. 
-  The proposed method is tested on hardware, using the Robotarium multi-robot test bed.
-  The proposed method is observed to outperform current methods that are capabilities *un*aware.

Weaknesses

-  I would elaborate in the introduction what is the role of the task at hand in the learning process, and its impact on the generalization capacity of the learned policies:  How does the generalization capacity get affected when the task changes, for example, from collaborative mapping to multi-target tracking?  How do the robots can "link" the current awareness of the robots' capabilities to the needed capability compositions for the new task?

Answering the above questions may be outside the scope of the paper.  If this is the case, then I would state early on in the paper (or in the limitations) that the focus of the paper is to achieve generalization given a task, and not across tasks.  I believe such a clarification will help readers to understand the paper's scope.

-  In my opinion, additional experiments, across a wider range of team sizes and tasks, could strengthen the paper's conclusions about the generalization capacity of the proposed algorithm.  In more detail:

(i) Fig. 3, which is used to conclude that awareness and communication of capabilities do not hurt performance, is based on experiments over a single task that involves 4 robots only.  In my opinion, additional experiments, across different tasks and more robots, may be needed to substantiate the conclusion that awareness and communication of capabilities do not hurt performance.  Or, could the paper be revised to elaborate on why the presented set of experiments is enough to support the conclusion?

(ii) Fig. 4 demonstrates a nice generalization to robot teams of 3, 4, and 5 robots, given training on teams of 4 robots.  In my opinion, a wider range of team sizes could be used to demonstrate the generalization capacity of the proposed method (e.g., up to 10 or more robots per team).
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper proposes an excellent problem and a method that appears promising. In my opinion, the current version of the paper would benefit from additional experiments, across a wider range of team sizes and tasks, to better support the paper's conclusions about the generalization capacity of the proposed algorithm.

UPDATE: In light of the experimental results provided during the rebuttal process, which they show that both capability and communication awareness may not necessarily increase perfromance, I keep the "Weak Accept" recommendation.  I look forward to future iterations of the proposed algorithm.
review_title: null

Meta-review: Based on the original submission, the rebuttal by the authors and the discussion that followed, the reviewers agree that this paper should be accepted to CoRL. 
To the authors: great job on writing a strong rebuttal and engaging with the reviewers during the discussion period! Please address the remaining comments in the camera-ready version of the paper.
Decision: Accept (Poster)