Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: New Definitions and Evaluations for Saliency Methods introduces intrinsic evaluation metrics for saliency methods --- completeness and soundness --- that do not require additional models or human evaluation. These metrics are grounded in logical proof concepts and force the method to output a saliency map that only explains the class of interest. The paper proposes a mask-based saliency method that optimizes for soundness as well as completeness. Evaluations compare the proposed saliency method to other mask-based saliency methods on soundness and completeness, deletion and insertion game metrics, and saliency metric. 
strengths_and_weaknesses: **Strengths**
* *Intrinsic evaluation methods for saliency methods* --- This paper proposes that saliency methods should be evaluated on completeness and soundness. These attributes are grounded in logical proof systems and defined for saliency methods mathematically and textually in the paper. These concepts provide a formal intrinsic framework to evaluate saliency methods without requiring human evaluations. Human evaluators often measure how well the saliency maps to their representations, which may not always align with the model's representations. Intrinsic evaluations are better suited to evaluating saliency methods on their ability to reflect the underlying model.
* *Defining saliency method requirements* --- The paper reframes saliency by introducing completeness and soundness as two necessary constraints for saliency methods. Previously we only required completeness; saliency justified a model's prediction. By requiring completeness and soundness, saliency justifies a model's prediction but can not justify any other possible prediction. These requirements improve the specificity of what a saliency method should output and make it more straightforward to interpret the results of a saliency method. It also ensures that the saliency map for each class is distinct, which can improve our ability to compare maps and draw meaningful insight between possible predictions.
* *Clarity* --- The paper is very well written and precise. Section 3 is straightforward to follow, despite presenting complex definitions.

**Weaknesses**
* *Novelty of replacement strategy* --- The key novelty of the saliency method is its pixel replacement strategy, where a new pixel value is sampled from another random image. This strategy is known as hot-deck imputation, where replacement values are sampled from the marginal feature distribution. Existing work has used hot-deck imputation as a masking strategy. It has also shown hot-deck imputation and mean imputation (i.e., grey pixel replacement) result in similar changes to model outputs. (see "What made you do this? Understanding black-box decisions with sufficient input subsets" by Carter et al.). Given the similarities to this work, I suggest discussing it in the related work and including it in the comparison to existing metrics and methods.
* *Missing related work* --- Related work on saliency evaluation methods should include model and data randomization tests from "Sanity Checks for Saliency Maps" by Adebayo et al.. Also, consider the saliency method axioms from "Axiomatic Attribution for Deep Networks" by Sundararajan et al.. "Sanity Checks for Saliency Metrics" by Tomsett et al. has a good evaluation of existing saliency evaluations.
* *Lack of reproducibility* --- The checklist indicates the paper does not include the compute details, code, or data. Please include computing details and other details needed for reproducibility. If possible, also release the code.
* *Limited limitations section* --- The paper does not discuss limitations. Understanding limitations is essential for readers who are looking to use this work. Please include a discussion on important considerations when using your method. Also, please incorporate the ethical considerations in Checklist 1 in the main text.

**Minor Issues**
* Line 19: missing space between emdash and words
* Line 37: "and if so one" --> "and, if so, one"
* Line 213: "Procedures for find masking explanations" --> "Procedures for finding masking explanations"
* Line 278: "from original test set" --> "from the original test set"
questions: See the weaknesses section for questions and suggestions that would strengthen the paper.
limitations: Please include a discussion of limitations. Some questions I had were:
* What is the tradeoff between intrinsic and extrinsic evaluations? Do they both have a place in evaluating saliency methods, or are intrinsic evaluations like yours always better?
* Is there a tradeoff between completeness and soundness? Should we optimize for both equally, or is there ever a case where we should prioritize one over another? Can looking at completeness and soundness separately tell us anything different than looking at them together?
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents an additional dimension, *soundness*, for evaluating saliency methods for explainable AI. The authors define this concept, then use it to provide both explanations for why existing heuristic methods work, and to suggest new saliency methods. 

strengths_and_weaknesses: ### Strengths

I find this to be a useful and convincing paper. The paper is well written, but the presentation of the concepts could be made more crisp in parts (see questions, below).

### Weaknessess

Nothing major I could see, but this is somewhat outside my area.
questions: - Lines 44-46 clearly describe the "mask shape" problem. The method introduced in Section 4 mitigates this problem by using pixels from a random image rather than grey pixels or other methods. Does this solve the mask shape problem, or just make it more difficult to solve? The new method requires solving a difficult segmentation problem (that I'm not sure humans would be very good at) but one that is nevertheless solvable.
- Lines 166-168: shouldn't other values for label $a$ enter in here? For example, Figure 3 (right) is for the second best label. Shouldn't there be an index over labels other than $a$?
- Line 184: Should be $\beta$ not $\alpha$.
- Are completeness and soundness similar to (or perhaps, the same as) sensitivity and specificity? If so, mentioning this would help to improve the clarity.
- Figure 4 typo "completenese"
limitations: The authors have adequately addressed limitations.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper presents a method for attributing saliency (in the sense of determining which pixels contribute to a classification outcome) to an image. It does so in a novel fashion that explores the tradeoff between the notions of completeness and soundness, pointing out that prior work in this domain does not address the latter.
strengths_and_weaknesses: The paper itself is reasonably well written albeit with some typos (e.g. completeness is spelled wrong in some places). I find that this addresses an original angle of this type of assessment of how the neural network makes its determination and in a principled way that gives it an advantage over some of its predecessors. I think this is a significant result and generally view the conclusions drawn by this paper as positive.
questions: Figures 1 and 2 present a good case for the issues addressed by the paper. There are many quantitative results included as well. I would have liked to see some additional results on the qualitative side if possible but also understand that there are constraints to the size of the manuscript.

Can the authors comment on the value of a measure of saliency being intrinsic rather than guided by e.g. an external neural network? I see merits for this, but would appreciate a succinct statement of why things should inherently be intrinsic.

Can the authors comment on what is the right tradeoff between completeness and soundness in interpretation since there is an ability to create an arc of success on both fronts. E.g. how should one adjust things to exert the optimal tradeoff for human interpretability?
limitations: There are not strong societal impacts of this work insofar as I can see and to the extent that these do exist the authors have made a good case.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper introduces and formalizes new evaluation metrics to ensure goodness of saliency methods, based on the logical concepts of completeness and soundness. The first ensures that the network's output is unchanged when using the masked (with the saliency map) input in place of the full image, which is what most of current evaluations methods for saliency methods require. The latter requires verifying that the same saliency method cannot be used to produce masked input that make the net output a different label, and therefore ensures that the evaluation of saliency maps appropriately track the model's probability of assigning labels. 
strengths_and_weaknesses: The paper's contributions are clear and significant, and explained in a straightforward and accurate manner. Examples are significant and useful. The originality of the contribution lies in connecting the context of saliency methods to logical proof systems and to formalize an evaluation approach which overcomes limitations of current methods and helps making them more rigorous and theoretically grounded. A simple saliency method based on optimization is proposed, which thanks of a change in the pixel replacement strategy allows to satisfy soundness at a small price in completeness. This is proven to work as expected when validated on various datasets and compared to other saliency methods. Furthermore, thanks to their formal frameworks of definitions, authors provide an intrinsic justification about why methods used heuristically to improve the aspect of masks (TV regularization and upsampling) actually work, in that they improve the soundness.

questions: The paper is very complete and accurate, and I don't have main questions or suggestions. Minor ones are:
- In "Prior approaches" section, a number of extrinsic evaluation metrics are listed. Are there some intrinsic methods previously proposed?
- At line 207, according to Fig. 2, M2 AUC score should be 0.29 and not 0.37.
- In fig.4 caption, I think there is a mistake, since TV penalty factor Lambda_TV in line 5 should be 0.001 and not 0.01.


limitations: I think sharing code and data related to the paper would be beneficial for the scientific community.
review_title: null

Meta-review: metareview: The paper introduces and formalizes new evaluation metrics to ensure goodness of saliency methods,.

Reviewers consensus about the paper was positive. They found that the paper contributions are clear and significant and also appreciated the paper originality. I therefore recommend acceptance.


confidence: Certain
Decision: Accept