Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper found that the conditional expectation-based Shapley value might fail in some cases, and they thought the problem was caused by the fact that the Shapley value uses the same weight for all marginal contributions. They proposed WeightedSHAP where they assigned a learnable weight to the marginal contributions and make the WeightedSHAP equal to the weighted average of the marginal contributions. 
strengths_and_weaknesses: [Strength]
1. This paper showed a significant problem of the conditional expectation-based Shapley value and provided detailed theoretical discussion about it.

[Weakness]
1. This paper only discussed about the conditional expectation-based Shapley (CES) value, while baseline value-based Shapley (BShap) value was not discussed. [cite 1] has already showed that the CES didn't satisfy the dummy and linearity axioms, while BShap could satisfy all four axioms. Therefore, it is not surprising that the CES might fail in some special cases. 

[cite 1] Mukund Sundararajan and Amir Najmi. The many shapley values for model explanation. In International conference on machine learning, pages 9269–9278. PMLR, 2020

2. The WeightedSHAP lacks of theorectical soundness. In addition to efficiency axiom, due to the nature of CES, the weightedSHAP also does not satisfy the dummy and linearity axioms, thus this method does not satisfy three of the four axioms of the Shapley value. And, the weight vector of weightedSHAP are learned from some utility function which might cause bias towards the model explanation and influence the objectivity of the explanation. Therefore, the WeightedSHAP lacks of theorectical soundness.
3. The experimental results in this paper may not be objective. The weight vector of the WeightedSHAP is optimized w.r.t the utility function AUP, while some similar metric AUC or MSE curve are used to compare the Shapley and WeightedSHAP. 
4. The experiment was not sufficient. Other than the method proposed in [cite 2], some other Shapley-value based explaination methods (like DeepSHAP [cite 3], etc.) should be also compared with WeightedSHAP.

[cite 2] Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, and Ilya Feige. Shapley explainability on the data manifold. arXiv preprint arXiv:2006.01272, 2020.

[cite 3] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in neural information processing systems, pages 4765–4774, 2017.

questions: N/A
limitations: N/A
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper considers the task of feature attribution (specifically, from the perspective of correctly capturing the *order* of attributions). It demonstrates that a common feature attribution method based on the Shapley value (SHAP) is sub-optimal in the sense that there exist instances in which it does not correctly identify the order of important features. It proposes a modification of SHAP that relaxes the efficiency axiom, where the marginal contributions can be weighted arbitrarily, and the optimal weighing is learned from data.


strengths_and_weaknesses: Strengths: The paper is written clearly and related work is discussed properly. Understanding the limitations of current feature attribution methods is an important area of research. 

Weaknesses:

1. One general weakness of this line of work is that by questioning whether the assumptions made by the Shapley value are relevant to the objective of correctly identifying the order of feature attributions ("given that Shapley axioms are often not readily applicable to ML problems"), the choice to stay within the general framework of Shapley seems more questionable. I understand that the efficiency axiom doesn't seem important for correctly identifying the order of important features - is the linearity axiom any more relevant than that? If one could write a similar paper proposing to relax the linearity axiom, then isn't just the takeaway here that the entire approach of using the Shapley value is not optimal? To be clear, this isn't an argument specifically against the current paper, but I do think it's important to keep in mind when considering the contributions & impact of the paper.

2. I think the connection between the observations regarding sub-optimality of SHAP (Section 3) and the proposed method (Section 4) could be articulated better. E.g. the main takeaway from the first negative example in Section 3 is that SHAP fails when the features are highly correlated. How does allowing the weights of the marginal contributions to be non-uniform address this issue?  I'm not sure I understand how the explanation regarding the issue of estimation error in the beginning of Section 4 is technically related to the negative results of Section 3. Ideally, it would be good to revisit the motivational examples from Section 3 and demonstrate how weightedSHAP improves on them (e.g. reduces the region of failure).
questions: 1. The current paper seems to suggest that ideally (if estimation weren't an issue), we would assign higher importance to $\Delta_d$ than the importance assigned by the Shapley value. If I understand correctly, this seems to contradict part of the motivation for removing the efficiency axiom in Kwon & Zou (they show $\Delta_j$ becomes less informative as $j$ approaches $d$, mostly because the marginal contribution of a single element to a large training data diminishes). Can you clarify this point?

2. Not only does the assumption that the set $W$ is finite seems limiting, but I was surprised to see that the cardinality of $W$ in the experiments was very small (13!). Several questions: Was $W$ chosen to be so small for computational reasons, i.e. because the algorithm is basically a for loop over the elements in $W$? How did you select the set $W$? Are the results sensitive to this choice? (the paper points to the appendix for details but I could not locate them).

3. Can you demonstrate how weightedSHAP behaves on the examples from Section 3? (see above)
limitations: Yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper investigates a limitation in the feature attribution method SHAP. To ameliorate this limitation, the authors propose a method WeightedSHAP, which generalizes SHAP to use a weighted average over coalition sizes. The optimal weighting is chosen based on a given utility function and set of possible weights. Experiment results show that the attributions generated WeightedSHAP have better performance than Shapley values on the AUP metric.
strengths_and_weaknesses: 
**Strengths:**
* The investigated problem is important; SHAP is a popular method for approximating feature attributions. The proposed method shows improvement over the baseline method on a key metric, and has a nice game-theoretic interpretation.
* The paper is well-written and well-motivated, with clear and concise notation.

**Weaknesses:**
* Limited comparisons. The only comparison is to the unweighted Shapley Value on AUP. The paper could be improved by including other metrics and/or value functions. For example, it would be interesting to see if improved performance on AUP generalizes to performance on other metrics (e.g. ROAR [1]). There are also other methods that similarly aim to improve the Shapley value's limitations w.r.t. feature correlation. For example, [2] proposes a modified value function. [3] proposes a feature attribution ranking that incorporates feature interactions. Comparisons to such methods would improve the paper.
* No qualitative analysis. The proposed method shows improvement on AUP, however it would be beneficial to see how the actual feature importance ordering changes. Ideally there would be some user study, but even qualitative examples would improve the paper
* No time comparison or time complexity analysis. The main text states that the "[computation cost] is comparable to the Shapley value estimation algorithm", however the WeightedSHAP algorithm must add some nonzero computation expense which likely increases as size of $\mathcal{W}$ and / or number of features increases. The experiments also use relatively small datasets and |\mathcal{W}| seems relatively small.

**Miscellaneous Minor Issues:**
* Typo in Line 132: "efficiency axiom does is not required"
* Line 279: "WeightedSHAP is always better than the Shapley value". Minor nitpick, however this statement could be misleading, given that it's possible for WeightedSHAP to equal performance of Shapley value (as stated in the next sentence). 

**Summary:**
The paper is well-motivated, and the proposed method shows improvement over the unweighted Shapley value on the AUP metric. I have concerns with the limited experiments (see above); including additional qualitative comparisons and comparisons with other metrics would improve on this point.


[1] Hooker et al, A Benchmark for Interpretability Methods in Deep Neural Networks  
[2] Catav et al, Marginal Contribution Feature Importance - an Axiomatic Approach for   Explaining Data  
[3] Masoomi et al, Explanations of Black-Box Models based on Directional Feature Interactions  

questions: * The experiment in Figure 2 shows the relative difference between $\hat \Delta_j$ and $\Delta_j$, but how is this difference reduced by using the weighted mean? Is there a corresponding figure plotting the relative difference after applying the weighted mean?
limitations: Limitations have been adequately discussed in the main text.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper illustrates that regular SHAP does yield a good AUP. They provide examples of why this is the case and provide arguments that regular SHAP is suboptimal. They propose WegihtedSHAP as a solution to improve the AUP, which parametrizes the marginal contributions with a weight that is optimized on the user-specified utility function $\mathcal{T}$. Experiments the demonstrate the advantage of WeightedSHAP.
strengths_and_weaknesses: Positive points:
1. Clearly defined problem - It's well-formulated through empirical evidence
2. Solution is quite simple
3. Experiments convincing 

Negative points:
1. Writing could be clearer


questions: 1. It's not clear what $\mathcal{T}$ exactly means, can you give an example?
2. I've understood that suboptimal refers to that AUP "converges" slower when you add the top k important features. Would it be possible to characterize this in terms of a convergence rate?

------

Post-rebuttal comments:

I'd like to thank the authors for their response and clarification. Happy to keep my score at 6, I think the image explanations are a nice addition.


limitations: The paper considers tabular data for regression and classification. It would be interesting to understand if this could be extended to consider image data as well.
review_title: null

Meta-review: metareview: Strengths:
* paper points out and addresses significant issue in conventional Shapley-based approaches
* theoretical analysis having game-theoretic interpretation
* convincing empirical results (after revision)
* clear writing, good survey of related work

Weaknesses:
* limited empirical comparisons (addressed in revision)
* lacks qualitative analysis (addressed in revision)
* raises general questions regarding suitability of Shapley approaches, and the role of assumptions within


Summary:  
This paper presents a nice generalization of Shapley-based feature attribution that, by introducing weights, mitigates a certain drawback of the standard approach, and provides more flexibility. Most reviewers view the paper’s contributions favorably; theoretical results and interpretations appear to be sound, and the empirical results complement them well (after additions made by the authors in the revised version and in response to reviewer feedback). One reviewer was worried about the soundness of a certain aspect of the proposed approach, but the authors’ response was helpful in clarifying this issue. Another reviewer pointed out that, more generally, arguing for relaxing assumptions (e.g., via weighting) may actually suggest that the overall approach is limited; the authors are encouraged to add a discussion in the paper that addresses this concern.

confidence: Certain
Decision: Accept