Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors show how model multiplicity (i.e. different models which have similar test set performance) can result in explanation multiplicity.  Using Shapely values for feature importance, often models that are underspecified (i.e. have the same hyper parameters setup but different initializations) but have similar test performance can result in different feature importance rankings and even entirely contradictory ones. The authors compare Rashomon effect (vastly different hyper parameters, similar test performance) and underspecification (same hyper parameters, different inits, similar test performance). The Rashomon effect has a much greater impact on agreement and contradiction (as expected), but underspecification also results in fairly high levels of contractiction.
strengths_and_weaknesses: Very strong experimental design tackling a very important issue. Results felt rushed, things are pushed to the appendix that should have been kept in the main paper. I found Figure 2 to be confusing, but the caption was very clear. Also, the right hand side of Figure 5. Overall, paper is very well written and the work is very significant.
questions: The authors note "The odds of a contradiction vary considerably across the experimental settings. They are remarkably high in some of the Rashomon sets, but roughly an order of magnitude lower in the underspeciﬁcation sets", could the authors speculate as to why this is. Are the models in the Rashomon set drastically different? Also, why are the contradict percentages fro the UCI Credit Card dataset in the top-3 so much higher than the other datasets? Did the models for that dataset evaluate more poorly on the test datasets?
limitations: The authors show how the arbitrary selection between similarly performing models can results in drastically different explanations. They couch the work in datasets related credit and insurance, an area which providing good explanations would allow the individual to modify their behavior such that they can score higher. In their work they show how the initial explanation would not often hold-up if a different (but similarly performing) model had been selected.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper studies how model indeterminacy impacts explanation multiplicity. Two aspects of model indeterminacy are considered:  predictive multiplicity (underspecification, rashomon sets), and epistemic uncertainty. As models change, explanations are bound to change as the authors show via experiments. However, authors also show that predictive multiplicity or epistemic uncertainty are not good indicators of the extent to which the explanations agree.

There has been prior work on disagreement between explanations by Lakkaraju et al. where different local explainability methods with respect to a single ML model. There is also follow up bayesian work on quantifying the uncertainty of local explanations by the same authors. Rashomon sets have been used by Rudin et al as the authors point. 
strengths_and_weaknesses: The problem studied by authors is very useful one, especially in practise. The experiments are quite thorough and good. However the results and their implications are perhaps less clear. 

When multiple almost equivalent models have the same prediction for a user, even then local explanations/gradients tend to be different. This is a useful insight. Authors hypothesise that this may be due to the fact that explanations depend on the shape of the predictive function, also seen visually for experiments on small problem sizes. The almost equivalent models are characterised using Rashomon sets and underspecification sets (randomness in training). The results are for one local explanation method -- shap explainer that approximates shapley values. 

The related insight is a bit confusing: i.e. the extent to which the explanations disagree is not determined by the extent to which the predictions of multiple almost equivalent models differ. Is this correct ? Pls clarify

The other insight is: If we consider epistemic uncertainty, this might not be a good predictor of explanation multiplicity for that user across the almost equivalent models. Is this correct ? Pls clarify

The main issues with the paper are: 
(a) Lack of clarity on following 3 terms: predictive multiplicity, model indeterminacy, and epistemic uncertainty. 
(b) Lack of clarity on results.
(c) Experimental results are based on only 1 local explanation method:  SHAP. 
(d) The analytical results are based on linear setting alone. 
questions: a) Can you clarify the key results ? 

b) Can you clarify the 3 terms better: predictive multiplicity, model indeterminacy, and epistemic uncertainty ?

c) How do Rashomon sets as computed by you differ from the ones defined in Rudin, et al? 

c) How is epistemic uncertainty computed here. Are Rashomon sets or underspecification sets used to compute it ? 

d) How would this work/results generalise to other explanation methods -- LIME, counterfactual techniques, etc. 

e) Are there any results for case when predictions of multiple almost equivalent models disagree for a user, how do the explanations disagree in this setting. 

limitations: See above.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper introduces a notion of explanation multiplicity. This is similar to previous works that look at the effects of model indeterminacy such as predictive multiplicity. The authors first show how explanations can vary for models with similar performance on toy examples and theoretically in the linear case. The authors demonstrate that model multiplicity can have noticeable effects on the explanations using shapley sampling explanations, and this can lead to the shapley explanations contradicting in practice for models IRL.
strengths_and_weaknesses: Strengths:
- This paper is very well written and easy to follow. It was a pleasure to read, and I had no issues understanding the work.
- The intuition the paper builds with toy examples before providing real experiments is nice. I enjoyed this aspect. 
- The idea is well-founded. Previous works on things like predictive multiplicity are quite concerning and this work fits into that literature nicely.
- The fig. 1 is quite nice, very clear + descriptive!
- This work also makes sense in the context of explanations, where aspects like stability have been studied extensively in the past (i.e., small perturbations leading to noticeable differences or simply rerunning explanations leading to large differences). So, this work is a nice extension in that regard to a related, but different notion about how explanations that seemingly should be the same are in fact different.
- The experiments are clear 

Weaknesses:

My main critique of this work is that the effects of multiplicity, in this case contradicting explanations, is quite a practical concern, and while the authors have shown this can appear in some cases, the extent to which explanation multiplicity appears in practice is still somewhat left unanswered.  The authors empirically demonstrate their technique using a sampling shap-based approach with the mean as the baseline distribution. Choices around the baseline distribution vary pretty widely and often other variants of shap are used like treeshap + deepshap practically because they're more efficient. It's nice that this work provides evidence this can be a problem in the case presented, but it would be really compelling to evaluate the degree to which explanation multiplicity occurs in other variants of shap that are practically used and other choices of baselines because I suspect readers will be looking for their IRL use cases.

This paper is a weak accept for me because I think the work is quite solid, but extending the evaluation to additional explanation use cases would make the impact much higher.
questions: * how many samples do you use to compute explanations? 
* Sampling-based approaches introduce instability themselves and do you consider whether the contradictions come from not sampling enough or explanation multiplicity?
limitations: Authors have done a sufficient job
review_title: null

Meta-review: metareview: This paper introduces the notion of explanation multiplicity. The authors first show how explanations can vary for models with similar performance on toy examples and theoretically in the linear case. They demonstrate that model multiplicity can have noticeable effects on the explanations using Shapley explanations. This paper puts forth a strong approach tackling a very important issue. So, all the reviewers concur that this paper should be accepted at this time. 
confidence: Certain
Decision: Accept