From: auai.org/UAI/2022/Conference/Program_Chairs
title: Paper Decision
decision: Accept (Poster)
comment: Meta Review: The reviewers felt that this paper offered a valuable contribution to the area of multiagent planning, and specifically in the use of communication in interactive POMDPs to handle certain forms of “open systems” (involving the presence/absence of other agents). There was an appreciation for the novelty of the problem, the new algorithm proposed (an extension of MCT to handle the “open” setting), and the empirical evaluation. The reviewers raised some questions about the general novelty vs. incrementality of the approach, and made a number of suggestions that could greatly improve the presentation and emphasis in the paper. The author response clarified a number of questions and convinced the reviewers that paper revisions would be forthcoming that would enhance the impact of this work. The authors are strongly urged to make the revisions suggested (and discussed in their rebuttal).

From: auai.org/UAI/2022/Conference/Paper475/Reviewer_dNs8
Q1 Summary and contributions: The paper addresses decision-theoretic planning in open multiagent systems in presence of explicit communication between the agents. 
In particular, the paper proposes a variant of the CI-POMDP model that accounts for openness and an algorithm for calculating the optimal actions of an agent in such a model.
Q2 Assessment of the paper: More detailed information regarding each of these aspects is given below:
Q2(1) Originality/Novelty: 2: Fair: The paper contributes some new ideas.
Q2(2) Significance/Impact: 2: Fair: The paper is likely to have moderate impact within a subfield of AI.
Q2(3) Correctness/Technical quality: 3: Good: The paper appears to be technically sound, but I have not carefully checked the details.
Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q2(5) Reproducibility: 2: Fair: Key resources (e.g., proofs, code, data) are unavailable but key details (e.g., proof sketches, experimental setup) are sufficiently well-described for an expert to confidently reproduce the main results.
Q2(6) Clarity of writing: 3: Good: The paper is well organized but the presentation could be improved.
Q3 Main strengths: The proposed approach is potentially interesting and has been shown to work well in an experimental setting.
Q4 Main weakness: The proposed model and algorithms are novel but quite incremental with respect to the state of the art.
The paper is rather clear, but more details (e.g., about algorithms and experiments) could be provided.
Q5 Detailed comments to the authors: The form of openness considered in the paper is rather restrictive: agents can only leave and rejoin the system. The most difficult case in which a previously unknown agent joins the system is not considered. 
How could the proposed model be modified in order to address also this case?

In Section 2, some intuition about the fundamental notion of level-l interactive states should be provided to make the paper more self-contained. Moreover, a discussion on the possible values for l could be added (in the following, only examples with l=1 are made).

The purpose of Section 3 is not fully clear. It lengthly motivates the use of communication and the most relevant part appears to be the second half of Section 3.2. Perhaps some material could be removed and the section could be shortened.

Space saved (see the comment above) could be used to better explain the details of Algorithm 1. 

Experiments show that the proposed approach works in the wildfire suppression setting. 
However, in setup #2, the algorithm I-POMCP-PF_{O} that does not use communication performs better. This is counter-intuitive: the CI-POMCP-PF_{O} algorithm can learn a superset of what the I-POMCP-PF_{O} can (since the former can simply learn to ignore messages), right? Does the problem lie in the incomplete learning process or in the type of messages that agents are exchanging? Maybe messages that are more related to the coordination between agents could help more the system?

Minor issues:
- Use parentheses (\citep{}) for references.
- Page 1: it likely
- Page 4: that optimize that maximize rewards
- References [Mirsky et al., 2022] and [Sunberg and Kochenderfer] are without venues, pages, ...
Q6 Overall score: 6: Weak Accept: Technically solid, moderate to high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations. 
Q7 Justification for your score: The paper provides an interesting but limited incremental contribution, which nevertheless appears to be sound.
Q8 Confidence in your score: 3: Somewhat confident, but there's a chance I missed some aspects. I did not carefully check some of the details, e.g., novelty, proof of a theorem, experimental design, or statistical validity of conclusions.
Q9 Complying with reviewing instructions: 1: Yes.

From: auai.org/UAI/2022/Conference/Paper475/Reviewer_xiSK
Q1 Summary and contributions: The paper puts forward a method for decision-theoretic planning enabling agents to plan with communicative actions in open multiagent systems. An algorithm optimised for large-observation spaces is proposed, and then tested on the wildfire suppression benchmark.
Q2 Assessment of the paper: More detailed information regarding each of these aspects is given below:
Q2(1) Originality/Novelty: 3: Good: The paper makes non-trivial advances over the current state-of-the-art.
Q2(2) Significance/Impact: 3: Good: The paper is likely to have high impact within a subfield of AI OR moderate impact across more than one subfield of AI.
Q2(3) Correctness/Technical quality: 3: Good: The paper appears to be technically sound, but I have not carefully checked the details.
Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q2(5) Reproducibility: 3: Good: Key resources (e.g., proofs, code, data) are available and key details (e.g., proofs, experimental setup) are sufficiently well-described for competent researchers to confidently reproduce the main results.
Q2(6) Clarity of writing: 3: Good: The paper is well organized but the presentation could be improved.
Q3 Main strengths: Overall, the content is interesting and relevant to the conference. I've found the writing very clear to follow and effective in guiding the reader from the background notions to the novel aspects. The idea to integrate the communication acts in the planned strategy seems very promising, as also confirmed by the experimental results.

Q4 Main weakness: No particular weaknesses
Q5 Detailed comments to the authors: My only curiosity concerns this aspect: the second scenario is the only one in which the proposed algorithm did not outperform the baseline. The reasons for these results are expressed clearly: agents' tendency to prefer cooperation instead of individual actions exposes them to unpredictable events (suppressant shortage). Anyway, I would have also expected a discussion on the possible solutions to this problem. For example, changing the simulation hyperparameters has an impact on the results? Do the authors think that changing the set of available communicative messages could help?
Q6 Overall score: 8: Strong Accept: Technically strong paper with, with novel ideas, excellent impact on at least one area of AI or high to excellent impact on multiple areas of AI, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations. 
Q7 Justification for your score: The paper is good.
Q8 Confidence in your score: 4: Quite confident. I tried to check the important points carefully. It is unlikely, though conceivable, that I missed some aspects that could otherwise have impacted my evaluation.
Q9 Complying with reviewing instructions: 1: Yes.

From: auai.org/UAI/2022/Conference/Paper475/Reviewer_KEbY
Q1 Summary and contributions: A decision-theoretic framework for communicating among agents in an open multi-agent system that balances the tradeoff between the overhead of communication with the enhanced ability to coordinate.
Q2 Assessment of the paper: More detailed information regarding each of these aspects is given below:
Q2(1) Originality/Novelty: 2: Fair: The paper contributes some new ideas.
Q2(2) Significance/Impact: 2: Fair: The paper is likely to have moderate impact within a subfield of AI.
Q2(3) Correctness/Technical quality: 3: Good: The paper appears to be technically sound, but I have not carefully checked the details.
Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q2(5) Reproducibility: 3: Good: Key resources (e.g., proofs, code, data) are available and key details (e.g., proofs, experimental setup) are sufficiently well-described for competent researchers to confidently reproduce the main results.
Q2(6) Clarity of writing: 4: Excellent: The paper is well-organized and clearly written.
Q3 Main strengths: The paper contributes new data about the relative costs and benefits of inter-agent communication.

The paper might offer developers of open multi agent systems new strategies for coordination

The paper seems correct in its technical results.

The paper offers real world-based experiments to evaluate their system

The paper is well written and complete.

The motivating application (fire fighting robots) is timely and illustrative of the problem they are trying to solve.
Q4 Main weakness: The paper is an extension to an existing approach and it’s not clear how non-trivial the new ideas are. In other words, it seems like a minor tweak to an existing framework.

Not convinced the impact will be broadly felt in the community.


Q5 Detailed comments to the authors: Section 3.2 seems to drag the paper down with unnecessary detail. It seems to show that nested agent modeling and communication are incompatible, which to me is a flaw in the overall agent model, and the solution presented seems to be somewhat of a hack. There is nothing deeply interesting about the problem being addressed in this section, or its fix, and I would recommend trying to drawing attention to the problem in a single paragraph.

The graphs in Appendix C, although it adds to completeness, does not seem to me to be required. The qualitative summary in the last paragraph in section 5 is probably all you need here.
Q6 Overall score: 7: Accept: Technically solid paper, with high impact on at least one sub-area of AI or moderate to high impact on more than one area of AI, with good to excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations. 
Q7 Justification for your score: It was well written, well motivated with a real world scenario, the experiments clearly demonstrated and explained the advantages of their approach. 

The work does not seem to be terribly original or game-changing with respect to state of the art. Although the results are promising, it's not clear that the approach  will be broadly applicable by the multi-agent community.  
Q8 Confidence in your score: 3: Somewhat confident, but there's a chance I missed some aspects. I did not carefully check some of the details, e.g., novelty, proof of a theorem, experimental design, or statistical validity of conclusions.
Q9 Complying with reviewing instructions: 1: Yes.

From: auai.org/UAI/2022/Conference/Paper475/Reviewer_Pq8Y
Q1 Summary and contributions: The paper extends the recently-developed communicative interactive POMDP framework to model agents that can join or leave the system, and extends a Monte-Carlo approach to solving POMDPs to allow planning using this extended model. The approach is illustrated by experiments in a simulated multi-agent fire-fighting domain.

Q2 Assessment of the paper: More detailed information regarding each of these aspects is given below:
Q2(1) Originality/Novelty: 2: Fair: The paper contributes some new ideas.
Q2(2) Significance/Impact: 2: Fair: The paper is likely to have moderate impact within a subfield of AI.
Q2(3) Correctness/Technical quality: 3: Good: The paper appears to be technically sound, but I have not carefully checked the details.
Q2(4) Quality of experiments (Optional): 3: Good: The experimental evaluation is adequate, and the results convincingly support the main claims.
Q2(5) Reproducibility: 3: Good: Key resources (e.g., proofs, code, data) are available and key details (e.g., proofs, experimental setup) are sufficiently well-described for competent researchers to confidently reproduce the main results.
Q2(6) Clarity of writing: 4: Excellent: The paper is well-organized and clearly written.
Q3 Main strengths: Technically sound and clearly-written paper that describes some enhancements of previous work in this area.

The two contributions are (1) an enhancement of a previously-developed model of multi-agent POMDPs with communicative actions so that it also models of the presence of absence of agents in an *open* multi-agent system, and (2) an extension of a Monte-Carlo tree search algorithm for multi-agent POMDPs that allows planning in this extended model.
Q4 Main weakness: Although technically sound, both contributions (enhanced model and enhanced algorithm) seem to be straightforward and incremental enhancements of a previously-developed model and algorithm, and thus the degree of originality/significance seems modest. For example, I looked at the Eck et al. paper from AAAI-20 that is referenced in this paper, and there seem to be as many similarities as differences between the papers.
Q5 Detailed comments to the authors: Although the enhancements described in this paper are obviously sound, I will mention that I have some reservations about the value of creating increasingly ornate models. As pointed out in the third paragraph of section 3, an open system can be more simply modeled by a state variable that represents whether an agent is present or not, and I read but did not quite follow the explanation for not using this simpler approach. In fact, I’m not convinced that communicative actions cannot be more simply modeled by performing regular state-changing actions on a variable that is observed by the communicating agents. In summary, some enhancements of a model make it possible to do things the original model cannot do, and others are "syntactic sugar" that make the model more convenient to use, but not necessarily more powerful. I suspect these enhancements are of the latter type, but would like to be convinced otherwise. If the simpler model is inadequate for modeling an open multi-agent system, for one reason or another, it may help to introduce an example early in the paper that more clearly motivates and explains why these enhancements of the model are necessary.
Q6 Overall score: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., lack of novelty, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
Q7 Justification for your score: A well-written and technically sound paper that makes an incremental contribution of perhaps modest significance.
Q8 Confidence in your score: 3: Somewhat confident, but there's a chance I missed some aspects. I did not carefully check some of the details, e.g., novelty, proof of a theorem, experimental design, or statistical validity of conclusions.
Q9 Complying with reviewing instructions: 1: Yes.