Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper provides a formal definition of blame for MA MDPs, a set of properties (many from the cooperative game theory literature, but also some for this setting) and an analysis of several methods (both existing methods like Shapley value and a new method that has properties Shapley does not have that are useful in this context. They also discuss uncertainty-oriented extensions to these ideas, and provide an experimental evaluation comparing several of the existing and proposed blame attribution methods.
main_review: Originality: Are the tasks or methods new? Is the work a novel combination of well-known techniques? (This can be valuable!) Is it clear how this work differs from previous contributions? Is related work adequately cited?

The work is of course based in traditional cooperative game theory, but I believe it is new in this setting, with extensions to those concepts fitting for the setting. Previous work is well-cited.

Quality: Is the submission technically sound? Are claims well supported (e.g., by theoretical analysis or experimental results)? Are the methods used appropriate? Is this a complete piece of work or work in progress? Are the authors careful and honest about evaluating both the strengths and weaknesses of their work?

In general I think things are well supported (see clarity issues below), and just about what one would expect for this kind of paper. I’d like to see a little on the complexity of the new method (since Shapley is unusable in practice for this reason). Examples would be even better.

The experimental section is dense and a little unmotivated (or unclearly motivated). It is not always clear what the experiments are (the domains are reasonably described but I’m still a bit unclear on the policies of the participants and how they were manipulate, and what is being shown on the graphs. At a high level, you would be better to describe *how* you intend to evaluate, how the reader should interpret the results, and then finally show the evaluations, since what is being evaluated here is (rightly!) quite different from the typical NeurIPS paper showing the performance of learning algorithm X.

Clarity: Is the submission clearly written? Is it well organized? (If not, please make constructive suggestions for improving its clarity.) Does it adequately inform the reader? (Note that a superbly written paper provides enough information for an expert reader to reproduce its results.)

In general the paper is quite clear (at least, to someone familiar with the traditional cooperative game theory ideas). However, probably due to space issues, I found the run up to the new method in 3.4 to be extremely dense with very little glossing explanation or examples. The high level idea makes sense but not sure I could re-implement. Same for section 4 (where I am less familiar with current approaches to reasoning over uncertain policies). The notational density also starts to wear the reader down (can’t keep the acronyms all straight…need a little cheat sheet to cut out and place by the paper!). Also, comments on experiments above.

Significance: Are the results important? Are others (researchers or practitioners) likely to use the ideas or build on them? Does the submission address a difficult task in a better way than previous work? Does it advance the state of the art in a demonstrable way? Does it provide unique data, unique conclusions about existing data, or a unique theoretical or experimental approach?

Yes, I think this is quite a useful porting of these ideas to this setting--I imagine this could be used as a basis of several different future works.

== Thanks for the clarifications; hopefully those can be added to the final version ==
limitations_and_societal_impact: yes; this is somewhat a core idea in the paper.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper furthers the study of blame in accountability in cooperative multi-agent systems.  They provide many natural desiderata, some natural attribution techniques, and characterizations of how these relate, as well as how they relate to standard game theory ideas.
main_review: I enjoyed this paper, and thought it was generally well written and complete.  My main substantive concern was that many of the desiderata are directly consequentialist in the sense that you can only be to blame if the event would not have happened if you had changed your action.  It seems like this deviates from intuitions around blame; for instance if a pilot took a nap on the job, but the system malfunctioned in a way that would have been unrecoverable even if they were awake, many would still want to blame the pilot for not satisfying the normative commitments we expect of a pilot.  This notion of blame would be more deontological, and would certainly violate many of these desiderata, but is closer to what people do.  The phrase comes to mind "no one was fired for buying ibm"; the claim isn't that ibm gives the highest utility, it's that it is a choice that is hard to blame someone for.

Additionally, I think there is a more causal objection to the desiderata.  You may want to blame the pilot because it would have been their fault if something else where different.  That is, even though nothing they could do would have changed anything, if the autopilot system had worked, or had failed differently, the pilot would have been able to change the outcome.  In a sense this is blame two counterfactual changes away from reality rather than one.

Flagging these kinds of limitations early could further strengthen this work, as it would clearly set out the domain of inquiry.  

On an organizational note, I think that the paper would be greatly improved if the experimental environments were introduced early and used as running examples.  By the time we got to the discussion of the behavior fo the rules in the environments I had forgotten most of the rules, but this would not have been the case if the behavior in the environments where discussed more gradually. It would have also made the rules more intuitive.

minor comment: 
* 152 should be "does not assign less blame for a policy with at least as much performance"

------
**Update to Review**

I appreciate the interesting comments and clarifications both in the response to this review and to the other reviews.  I believe incorporating the proposed changes will be valuable to readers and will improve the paper.

limitations_and_societal_impact: A limitation that you mentioned was the negative externality of unjustly blaming someone, but it is also worth mentioning the negative externality of not blaming someone when you should and thus incentivizing bad behaviors.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In the context of MMDPs, this work presents an overview of blame attribution methods from the game theoretic literature, together with a study of desirable properties. It then contributes a novel blame attribution method to mitigate some of the identified issues. All the methods are compared in an empirical study and analysed in terms of robustness in the face of uncertainty. 


main_review: The paper is nicely written. I could follow along even without being familiar with the literature on blame attribution. 

My first major comment for this work: the paper is written more as a survey, will full overview of approaches and properties, so it is difficult for readers to distinguish what the contribution and the novel concepts are, especially if they are not familiar with all the elements. There is no delimitation between existing literature and newly introduced elements. This chips away from the clarity and quality of the work.

As far as I can tell, the work is technically sound, with no evident issues.

A second major comment concerns the experimental environments. Can the authors explain the choice of the settings? It was especially difficult to grasp Environment 1, and the reasoning behind the manner with which the setting is extended to multiple agents. Is A1 meant to rectify potential mistakes of A2? It was also not clear how the policies of the agents function. For example 'The personal policy of A2 takes an optimal action with a state dependent probability; otherwise it follows an optimal policy that sets the cost of H cells to the cost of F cells'. Do you mean that the optimal policies assumes that H and F cells have the same cost?

All in all, it was an interesting study, my only thought is that this work could have a better impact in a longer format, that would also incorporate all the numerous details from the appendix.


Minor comments:

Lines 115 vs. 119: inconsistent $\mathbb{R}$
Line 103: the expected return is what is presented there

---Post-rebuttal---
Thank you for addressing the raised concerns and for all the clarifications. I will raise my score accordingly. 
limitations_and_societal_impact: No statement for these aspects is present.
review_title: null

Meta-review: All of the reviewers appreciated the general problem being tackled by this paper, and view it as an interesting one for multiagent systems and accountability/interpretability in AI. The axiomatic approach and the novelty and analysis of the new AP mechanism for blame attribution are both novel and interesting. The reviews express some concerns around presentation and explanation of some of the details (e.g., experiment motivation, the remarks around consequentialist vs. deontic perspective on the problem, etc.). The author response helped clarify a few of these issues, and nudged the consensus to acceptance. It is critical however that the authors make the revisions needed to address/clarify the issues raised. While this type of paper is somewhat a bit outside of the (current) set of mainstream topics for NeurIPS, the topic is of relevance to the community (and connects to interpretability and fairness, etc. that are becoming more mainstream in NeurIPS). 

Decision: Accept (Poster)