Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper tackles the problem of model explainibility from the perspective of heatmap-based visualization approaches. It argues that achieving a good heatmap to explain the model decision is possible by considering two notions, completeness and soundness, borrowed from logic. In fact, the completeness condition is what already considered during the formulation of similar methods. Thus the novelty lies on studying and analyzing the usefulness of soundness. The paper, thus, proposes a metric to evaluate both completeness and soundness that is used to learn a mask that explains the model prediction.
main_review: The paper brings an interesting concept, i.e., soundness, to the model explainability research. There, however, remains a big motivation gap. Why do we need soundness? Given that we are interested to understand what causes the prediction output of a black-box model, why the source of wrong predictions shall matter?

The paper proposes a new score to evaluate the quality of heatmaps. The score is well designed and suited for the proposed approach. It, then, compares some methods with the proposed approach using the same evaluation score tailored for it's proposed algorithm. Thus, one could wonder, why are we lacking a careful comparison and discussions with other scores? Is it fair to compare multiple methods on a score tailored for one?

On the practicality of the method, based on the note in the paper about back-up plan of running other methods, is there any statistics of how often this could happen in the experiments? And how much this could be a risk to the applicability of the approach?

Overall, the paper is interesting. One caveat is that it refers to the supplement so often. It would be nice to read a paper that is self-contained. 
summary_of_the_review: The paper is providing good insight about model explainability and proposes interesting concepts. Reading the paper, it seems the presentation, storyline and experiments could be improved. I find it a bit too much scattered between the supplement and main paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper addresses masking-based saliency methods for providing visual model explanations. It defines several new metrics for evaluating saliency methods: "completeness", "soundness", and "consistency score". It proposes a saliency method that is a variant of prior work. The paper shows through experiments that TV regularization improves soundness of saliency-based explanations.   
main_review: **Strengths:**

- The introduction of soundness and consistency score for use in evaluating saliency maps is a good idea and is explored in several experiments.
- The introduction did a good job of setting up the distinction between saliency methods and saliency evaluation metrics and laying the groundwork for the paper
- The related work gives good background on related saliency methods and evaluation methods; however, the paper could include more context for the past work and its relationship to the current paper, especially in the "arguments about saliency" and "controversies" sections. It would be helpful to have a brief description of what the main points of controversy are and perhaps a note about how this work relates to those arguments. 
- The paper proposes soundness as a quantitative way of discovering artifacts in saliency-based approaches and back this up empirically r.e. TV regularization. 

**Weaknesses:**
- The impact of the work is unclear. The new saliency method is similar to prior work and has mixed results when evaluated against other baselines (Table 1). The introduction of new saliency metrics has the potential to be useful, but I do not understand what new information soundness or consistency score provide that are missing from other measures; furthermore, they are expensive to compute (Section 6). 
- I found the descriptions and equations for completeness and soundness--Definition 3.2 on page 4--to be difficult to understand. I think it would be useful for the paper to include an intuitive description of the equations, how they interacted with each other, and whether alpha and beta should be minimized or maximized. For example, it would be nice to follow up the definition with an intuitive explanation of these metrics to help the reader understand what they are capturing. It may also be preferable to break up these two concepts into separate definitions. 
- This is my current understanding of completeness and soundness: alpha provides a lower bound and 1/beta an upper bound on the ratio of the probability assigned to class a on the masked vs the unmasked input. It is desirable to have both alpha and beta close to 1 (however, additional constraints must be applied to avoid the trivial case where the whole image is considered salient). Alpha and beta close to 1 imply that for all classes, the model's scores on the masked input are similar to its scores on the unmasked input. Is my understanding correct? If not, could the authors please clarify? If so, it may help to include such a discussion in the paper. 
- The paper should justify why completeness is desirable. There is some discussion of this in the introduction, which ties increased soundness to reduction of artifacts in the mask itself. However, there are situations where it would make sense for different masks to significantly change a classifier output. Consider the case where a cat/dog classifier is trying to classify an image with both a cat and a dog in it, then masking out the dog should increase the probability of cat. Similarly, a non-robust model might output very different results in response to different masks. As such, does completeness measure the goodness of the saliency method or the robustness of the model? 
- For the experiments described in Section 6.1, are the authors trying to validate the usefulness of their metric or prove the benefit of their saliency method? The proposed saliency method does not outperform other baselines for most metrics. 

**Additional small notes:**
- The approach was inspired by logical proof systems, which is an interesting source of inspiration! But the connection is not clear--perhaps adding a brief description of how soundness is derived from logical proof systems or some references on this point would be helpful. 
- It would be helpful to readers if the contributions were stated more clearly. The introduction contains a bulleted list of "Other contributions"; I recommend changing this to a list of the top contributions.
summary_of_the_review: The contributions of the paper (the proposed saliency method and the new saliency metrics) are modest and their significance is not clearly explained in the paper or supported by experiments. The paper should explain the concepts of completeness, soundness, and consistency and justify why they are desirable. The experiments should support the main contributions: they should show that the proposed metrics are useful and that the proposed saliency method outperforms baselines on these metrics.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper discusses the completeness of saliency maps and proposes the concept of soundness. Based on this, they propose consistency score to predict the possibility that the quality of the saliency map is consistent with the classification probability. Besides, the author uses these definitions to implement the intrinsic evaluation of masked-based saliency methods.
main_review: Strengths：
1.	The method proposed in this paper does not need to introduce additional manual labeling or training costs, which is an interesting and worth exploring perspective.
2.	The paper provides a lot of intuitive visualization results, which is helpful to guide readers' understanding.

Weaknesses:
1.	We believe that the example in Figure 1 is not comprehensive and does not explain the difference between completeness and soundness well. We want to know what will happen if the saliency maps of the category labels do not overlap?
2.	Section.3 mentioned that this paper uses the images of the training set to fill the pixels outside the saliency map area S. Wouldn't other objects in the background area interfere with the result unpredictably? Perhaps more explanation should be given here.
3.	From the results in Table 1, compared with the method in Phang et al.(2020), the proposed method has no special advantages.
4.	The data in Table 2 confuses me. The completeness of different methods is very different, but the soundness is very close, especially the second column of data. How should we understand this result?

summary_of_the_review: The concept of the proposed method is innovative, but many details of the method have not been explained clearly. In addition, more persuasiveness is needed in the experimental performance.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper introduces two intrinsic evaluation criteria for saliency-based model explanation: completeness and soundness. While completeness indicates a saliency method's ability to justify the correct label,  soundness requires the method to be unable to find a masked input that significantly increases the probability of an incorrect label. A consistency score is introduced to simultaneously evaluate both completeness and soundness. Further, taking into account the soundness factor, a saliency method is proposed to maximize the probability for a given label, rather than all labels. Experiments demonstrate that the heuristic TV regularization method can help with soundness, as suggested by high consistency scores.


main_review: Strengths:

- The paper is well structured, and it is easy to read.

- The proposed saliency explanation approach is simple and general. It leads to extensive results on three datasets: CIFAR-10, CIFAR-100, ImageNet. 

- The analysis of TV regularization is novel and interesting.


Weaknesses:

- While I generally agree with the idea of evaluating and improving the soundness of saliency methods. The premise that the top-1 prediction is considered to be the only correct label and the second prediction is considered to be incorrect, is problematic. Indeed, a well-trained model can achieve high classification accuracy on iconic images and the top-1 prediction usually matches the ground truth. However, in practice, the top-1 prediction is likely to be wrong （as in Table 1), or there are multiple labels that can be correct (as in Figure 4). In the case of Table 1, I am not sure whether the wrong model prediction or the ground truth is considered to be correct. In the case of Figure 4, it is not clearly stated which is the correct label, elephant or zebra or both, and what the completeness and soundness scores of the compared methods are. To avoid confusion, the definition of correctness needs to be clarified. Again, determining what labels are correct or incorrect is critical, which may require a thorough discussion.

- Given this unclear definition, determining what labels are correct or incorrect will be critical. The metrics defined in Eq. (7)-(8) may need to be revised to consider multiple correct/incorrect labels. 

- The experiments show that TV penalty or upscaling generate more smooth saliency maps, which leads to decreased completeness and improved soundness. Can this observation be generalized to other smoothing techniques, such as a simple Gaussian filter? If the effect is mainly attributed to the smoothness of saliency maps, this approach looks a bit trivial. Other baseline methods, such as 1) a single Gaussian blog placed in the image center, 2) a randomly sampled saliency map from other images, or 3) a bottom-up saliency map independent of the labels, should be added, because the current pixel-wise randomization approach does not consider the distribution of image data.

-  It would be interesting to quantitatively evaluate the performance of saliency methods on the ImageNet localization task, and to perform a correlation analysis between the completeness/soundness scores and the localization accuracy.


summary_of_the_review: While I appreciate the motivation and the overall idea of measuring soundness of saliency, the definition of correctness and soundness is a bit unclear, and the effectiveness of the proposed method is not sufficiently demonstrated by the experiments. Considering the strengths and weaknesses of this paper, I don't think it is ready for publication at this moment.
review_title: null

Meta-review: This submission tackles the problem of model explainability from the perspective of masking-based saliency methods. 
Several metrics are proposed for evaluating saliency methods including  a new « soundness »  concept.
Experiments using a consistency score to simultaneously evaluate completeness and soundness are provided.  

Most of the reviewers were not convinced by the approach and have raised several issues. 

After rebuttal and despite the interest in the introduction of the concept of « soundness »  to better explain model decision, the current proposition needs to be improved. In particular, the interest of this soundness concept does not bring out, many details of the method are not clear enough and the effectiveness of the proposed measure is still questionable. It would be interesting that the authors consider the R’s comments as the ones for additional  experiments to demonstrate the relevancy of their contribution.
Decision: Reject