Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This work introduces "joint Shapley values", which directly extend Shapley’s axioms and intuitions: joint Shapley values measure a set of features’ average effect on a model’s prediction. This work naturally extends Shapley's axioms from a single feature, to sets of features. In a nutshell: joint Shapley values measure the average marginal contribution of a set of features to a model’s predictions. This work presents rigorous mathematical results for the joint Shapley values approach. This approach is then evaluated on several datasets, including: (i) simulated data, (ii) Boston Housing data, (iii) Movie Review data.
main_review: Strengths:
- The code looks quite good. Has walkthroughs, examples, etc. Implementation looks clean.

- The theoretical results look clean, intuitive, and well-presented.

- The work inlcudes some discussions on practical questions, including the employed hardware resources, and algorithm complexity estimates

- The work gives an overview of a good set of other SOTA approaches for Shapley




Weaknesses/Considerations:
- For XAI-based approaches, it is very important to consider their practical advantages, ease-of-use, and intuitiveness. Whilst this work gives a good treatment of the mathematical properties of the proposed approach (and related approaches), these properties are not necessarily tied to practical advantages. For example, emphasis is placed on the fact that "interaction indices are computed with discrete derivatives" (page 1), and yet, it is non-obvious what this implies from a practical standpoint.

- Furthermore, it feels as if this work is lacking a general summary of how to apply this approach for gaining insights on model behaviour. Currently, evaluation is done by running the approach on several values of "k", comparing the different outcomes, and tying it to properties, such as "negation effects". However, such an approach feels quite manually-intensive, involving significant efforts of post-hoc interpretation of the results. This will especially be the case with a larger number of features, and larger values of "k". Including a short description of how to apply this proposed approach in order to gain insights from a model with minimal manual effort involved would be helpful.

- Whilst this work lists numerous related SOTA approaches to feature importance (and Shapley values in particular), a comparison is only made on the toy benchmark datasets. Giving concrete examples of the differences in explanations provided by these approaches on the more complex cases (e.g. Boston Housing), would provide a much clearer view of their relative strengths and weaknesses. At the very least, a justification for why such a comparison is impossible would be helpful.

- There are quite a few explanation properties introduced in the Applications section, such as the "negation effect", or the "cancellation effect". If this is novel terminology introduced in this work, it would be helpful to discuss this earlier on in the methodology section (or in the appendices). If these properties are based on prior work - it would be helpful to include references. If not - giving more intuition on these properties (perhaps with more illustrative toy examples) would help.



Other Remarks:
- "evaluating it at a reference or baseline feature" (page 1) - I think there's a typo in that sentence

- The Introduction section is essentially merged with the Related Work section, making it a little hard to follow. I suggest splitting this into two separate sections for clarity.

- Is there any particular reason for why the "Applications" section is not called "Experiments"?

- Table 2: not sure if so many decimal places are required. Removing some of them could make the table more compact, and improve readability.

summary_of_the_review: The potential impact of this work is not clear as not enough is said about practical advantages.

Gaining some insights into the model behaviour would make the content of the paper stronger.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduces joint Shapley values, which extend Shapley values to measure contributions of *sets*.

main_review: I agree with the theoretical significance of this work, which generalizes the Shapley value for assigning a contribution of *a set of agents*.

Here are some comments:

1. In practice, how can we choose k? It would be great if the authors could provide some rules. 

2. In the simulation, what is the explicit form of the set function? For example, in Table 2, even if there is an explicit form of a function f, it's nontrivial to derive the corresponding set function. 

3. Continuing from 2, I recommend adding discussion on the choice of the set function.






summary_of_the_review: - I agree with the theoretical significance of this work. 

- A few questions/comments on the perspective of practitioners are listed above.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes an extension of the Shapley values, namely, the joint Shapley values which measure a set of features' average effect of a models prediction. The uniqueness of the joint Shapley values is proved. Moreover, training details and tuning parameters are provided in the accompanying code.
main_review: Strengths. The paper is well written and the findings are well explained. The experimental section illustrates that the newly introduced joint Shapley values lead to reasonable results in real applications. Some examples of the joint Shapley values are provided in Table 5 to clarify the obtained results.  

Weaknesses. The method is computationally heavy (but it is not easy to do it less computationally expensive). Indeed, the sampling can be done more efficiently, however, it is difficult to criticise this point much, since the authors mentioned it as their future work.

summary_of_the_review: The topic is important and challenging. The results are novel, and the experimental section provides a nice illustration how the joint Shapley values can be used.
review_title: null

Meta-review: The reviewers think the topic is important and challenging. The results are novel, and the experimental section provides a nice illustration how the joint Shapley values can be used. However, the paper can be improved by including more real world applications and experiments.
Decision: Accept (Poster)