Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper introduces an evaluation framework for explanatory methods that uses human ability to predict a model's predictions as a measure of the utility of the explanations. The paper considers 3 scenarios for usefulness: bias identification, understanding how to solve too difficult tasks for non-experts, and understanding of failure cases. The paper looks at 6 attribution methods over 1 dataset per scenario.
strengths_and_weaknesses: Strengths
The paper addresses a very important topic: the evaluation of explainability methods.
It is clearly written and has a good research methodology. 
Their evaluation method is applicable to any explainability method. While the principal behind it is not novel (as the authors also point out), the evaluation is done in a more thorough manner than in the previous studies.
Some of the results provide new and unexpected insight, which could potentially have a good impact on shaping the explainability field.


Weaknesses
There isn't much novelty in the evaluation method proposed, just minor improvements over previous studies, although it is still worth having these improvements. 

For each of the 3 scenarios, only 1 dataset was used.

Only attribution methods were tested. I think it is already quite clear for the community that attribution methods have lots of problems (and clearly they cannot provide the "what" that the authors mention) and therefore it would have been more interesting to also test other types of explanatory methods. But even among attribution methods, popular ones such as SHAP and LIME were not tested. 

Some of the conclusions have already been established before and/or are to be expected, as the authors also point out. However, it is still useful to have this paper show them again in a proper manner (e.g., eliminating potential confounds).


Minor typos:
Use \citet when appropriate
L74 "score drop in score"
L81 remove . before citation
L85 citet [5]
L103 This removeS
L107 measure reflectS
L107 and L109, 112 explanationS
L178: Meta-predictorS
L191: space missing ".T"
L220 "ref. [5]" delete ref and citet
L229: if --> is
L320 citation missing
L328: capital for section
L367: vs. remove larger space
L394 one needS
L394: is -> are
questions: How do you think the evaluation method can help the community to design better explainability methods in the first place? Assume that one develops a novel explainer but evaluating it on your method shows that it is less useful than e.g., SmoothGrad. Would your evaluation provide any further insight as to how the novel explainability method could be improved to become more useful?  
limitations: I did not see a direct mention of potential limitations or negative societal impact, which I suggest the authors incorporate in their paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper addresses evaluating explanation methods for image classification based on human experiments. To evaluate them, this study proposes an evaluation metric that quantifies how well humans are able to understand the output of the system after looking at the explanation provided by each explanation method. Then, this study analyzes the explanation methods by considering three scenarios to do bias detection, identification of new strategies, and understanding of failure cases. 
strengths_and_weaknesses: ## Strengths
- The experiment is well designed, the hypotheses tested are appropriate, and the experiment is done with enough participants. Therefore, the results obtained look convincing, except for the points in the weaknesses below.
- Some results are valuable for XAI community. For example, the result of the third scenario can strengthen the statement in previous works, e.g., [57].

## Weaknesses
- I think the impact of the results is limited because the experiment is conducted only on limited scenarios, such as classification on Husky vs Wolf and classification on Kit Fox and Red Fox. Due to the limitation, the scores shown in Figures 4-6 are distributed in narrow areas. Therefore, if more scores are obtained by conducting experiments on more scenarios, the results may change.
- There is a possibility that the predictive performance of the recognition model affects the utility score. When the recognition model has a poor predictive performance, the explanation methods depending on the predictions by the recognition model may also produce incorrect and uncertain explanations. The three scenarios in this study are settings that are difficult for the recognition model to accurately classify images. Therefore, I have a concern that whether the experiment can evaluate explanation methods only, excluding the impact of the recognition model.

questions: NA
limitations: NA
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a quantitative measurement to evaluate explainability methods by relying on how well the methods allows a human to predict what prediction a classifier will make. The authors conduct an experiment using their approach for a variety of image classification explainability tools.
strengths_and_weaknesses: Strengths:
Some sort of statistical approximation around the effectiveness of human-facing explanations is a worthwhile, if challenging goal.
The methods the authors propose, assuming one buys their narrow definition of an effective explanation, seems reasonable and defensible.
The initial experiments produce interesting and potentially useful results.
The paper is reasonably easy to follow, though would benefit from making fewer assumptions about reader's background knowledge in the introduction.

Weaknesses:
The concept of a meta-predictor seems to limit the author's approach to problems in which a model's goal is simply to replicate a human's intelligence, rather than to augment it or enhance it in ways the human could not do on its own.
The authors claim they conducted psychophysics experiments. In what ways does their study design engage with psychophysics? The authors should explicitly say what they are borrowing from the psychophysics literature and why. (From my understanding of psychophysics, the connections seem tenuous and superficial at best).
questions: 1. Might this approach apply for other definitions of an effective explanation?
2. Is this actually a psychophysics experiment?
limitations: The authors should deeply engage with the limitations around how creating a benchmark for their narrowly defined idea of an effective explanation might cause harm by shifting focus away from other ones.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors present a human-centered framework to evaluate the usefulness of attribution explainability methods. They evaluate 6 different attribution methods across 3 setttings. They show that generally, perceptual scores derived from attribution maps can be more useful at predicting failure cases of explainability methods better than fidelity metrics.
strengths_and_weaknesses: Strengths:
- The explainability evaluation framework the authors propose: 
   - is human-centered, which is, ideally, how the usefulness of explanations should be evaluated.
   - includes a control group, which was lacking in previous work

-  The paper is easy to understand and generally well written.
- The proposed metric is original and sound to the best of my knowledge.

Weaknesses:
- The proposed framework poses some reproducitiblity concerns since in order to evaluate any explainability one would have to undergo the entire process described in the paper of hiring and filtering the meta-predictors. This also makes it hard to scale.

- I worry about the effectiveness of the meta-predictor whenever evaluating a dataset with more than 2 classes.

- The authors state that the framework can be applied to other explainability methods and while this in principle makes sense, there are no experimental results supporting this claim. 

questions: In general I recommend rebranding the paper as a study of attribution methods instead of a framework, since the results are hard to scale, hard to reproduce and it is cumbersome to "integrate" a new explainer into this framework. However, the manuscript as is could be improved in the following ways:

#### Critical changes:
 - Provide some experiments with a varying number of classes or at least a small section discussing how the effectiveness of the meta-predictor approach varies when increasing the number of classes.

- The main reason the authors provide for the failure of attribution methods is an intrinsic limitation of said methods, since they can only show what regions in an image should be changed but not how. This is precisely one of  the limitiations counterfactual explainers solve, and since the authors claim that the framework can be applied to any explainer it would be useful to explore if counterfactual explainers fail in the same way, whilst substantiating said claim.
- It seems to me that there is a potential overlap within the 3 scenarios used for evaluation (Bias detection, Identifying an expert strategy, Understanding failure cases). For example: In scenario 2 (Identifying an expert strategy) whenever a meta-predictor is provided with an image that the model has failed to predict, then the task is no longer: _to understand the strategy that was discovered by the AI system_, but instead it becomes either bias detection (scenario 1) or Understanding failure cases (scenario 3). Conversely, in scenario 3 (if the classes are really similar) whenever the meta-predictor is provided with an image that the model has successfully predicted then the task becomes Identifying an expert strategy (scenario 2). An easy fix would be to provide only correctly and incorrectly classified samples for scenarios 2 and 3 respectively, however as stated by the authors this would significantly affect the meta-predictor score if the meta-predictor has some prior knowlege about the domain (dataset) the scenario is being evaluated on. If this overlap does in fact exists, I think it should be discussed in the paper.

#### Minor changes:

- Figure 1 is not referenced anywhere, also there is a missing reference on line 320
- In the section describing the understandability-completeness trade-off it would useful to mention/show examples of how an _entirely faithful_ and _more understandable_ explanations look like.
- In the General study design section it is mentioned that each training session has 5 training samples, but the plots showcasing the results show 5, 10 and 15, this should be clarified. 
- I think it would be useful to incorporate the meta-predictor response time into the metric since an explanation that is correctly predicted instantly is probably better than one that gets correctly predicted after some time. Conversely an explanation that is incorrectly predicted instantly could be considered more "misleading" than one that gets incorrectly predicted after some time.

#### Questions:
- Do the authors have any hypothesis as to why the best method for Bias detection and Identifying an expert strategy differ? I feel like the paper would be more complete with a small section explaining this phenomenon, not unlike the section: __Why do attribution methods fail?__. In which the authors investigate why all methods underperform on the 3rd setting.
limitations: The limitations of the work should be discussed beyond the cost of human evaluation, since the framework itself has some limitations (mentioned above) such as scalability and reproducibility.
review_title: null

Meta-review: metareview: This paper introduces a human evaluation framework for benchmarking current explainers.

There was an engaged discussion between authors and reviewers. Many concerns were clarified and the average score was raised from 4.75 to 5.5. Some concerns remain regarding the intrinsic limit of human evaluation, but overall, there are no major flaws with the current study and the framework is likely to be valuable since human evaluation of explainers is central to the progress of XAI. 

Hence, I recommend acceptance of the paper.

confidence: Certain
Decision: Accept