Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper presents experiments for both post-hoc and ad-hoc explainers to better understand their quality and robustness.
main_review: The paper touches on a very important problem - the quality and understanding of (smoothed) explanations. It is very well written and an easy to follow. 

It has many positives. I particularly liked the approaches to assess robustness as they seems very reasonable and widely applicable, for example using LPIPS and perception and a way of giving an interesting new perspective. The main to areas for improvement / clarification are as follows.

- Generalization. How do the results extend beyond the specific networks chosen? It is difficult to understand how relevant the conclusions are in general and whether the example configuration in influencing the conclusions too much. 

- Formalization. For me, the paper lack in the formalization of the problem. After reading the paper I don't fully have a clear notion of what makes an explanation of this form high quality and the exact properties that one should be looking for to assess it. Of course, the metrics reported are proxies to it, but the actual objetive is not clear to me. 


summary_of_the_review: Good paper but I expected a bit more in terms of formalizing quality.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The main contributions of this paper are a series of empirical results on smoothed attribution methods that are designed to show several smoothing techniques in the previous literature may produce worse explanations and these smoothing techniques are also not robust to non-Lp attacks. 

main_review: ### Strength
This paper presents sanity check, non-Lp adversarial attacks, and sparsity measurement on SmoothGrad, Uniform Gradient and $\beta$-smoothing. The results are generally interesting to the community. 
Using the Gini Index to measure the sparsity is an interesting idea that might be useful for the follow-up works. However, I have several questions regarding this metric. I will elaborate in the next subsection.

### Weakness
- It is not surprising that Lp-based defenses (smoothing) are not robust to non-Lp threat models in the community. The authors aim to present evidence that smoothing techniques are not robust enough because the proposed non-Lp attacks can break the explanations, which by itself seems to be an unfair comparison for the prior work. In fact, the contributions showing the proposed non-Lp attacks can break Lp defense is not a new observation.

- The proposed attacks do not seem to produce very different results. In Fig 5, resulting adversarial attacks from new technologies remain very similar to Delta attack. In Fig 4, it seems that the new attacks only make more than 0.05 difference with the Delta attack on $\beta$-smoothing in cosine distance. On other attributions, the improvement seems to be minimal.

- Gini Index for evaluating sparseness. The motivation to use Gini Index seems to be fair, however, a lot of prior works have proposed several different metrics [1, 2, 3, 4] in measuring the sparseness (or the concentrations and the localizations on the relevant features) of attributions. At least some discussions and justifications of the proposed metrics should be included. The robustness-related and fidelity-related evaluations are motivated from the paper’s main idea, understanding if smoothing techniques provide faithful explanations, however, the transition to study sparsity is somewhat sudden to me and it seems to be unconnected from the previous content, only because the following reason authors provide: “To create explanations that are human-accessible, it is advantageous to have a sparse explanation map”

- Some related prior work, i.e. ROAR [5] , that finds smoothing techniques do not create significant degeneration to explanations should be included and discussed, especially when the paper is trying to provide the shortcoming of the smoothing techniques.


[1] A. Chattopadhay, A. Sarkar, P. Howlader and V. N. Balasubramanian, "Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks," 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), 2018, pp. 839-847, doi: 10.1109/WACV.2018.00097.

[2] Poppi, S., Cornia, M., Baraldi, L., & Cucchiara, R. (2021). Revisiting The Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Experimental Analysis. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2299-2304.

[3] Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., & Hu, X. (2020). Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 111-119.

[4] Fong, R., Patrick, M., & Vedaldi, A. (2019). Understanding Deep Networks via Extremal Perturbations and Smooth Masks. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2950-2958.

[5] Hooker, Sara et al. “A Benchmark for Interpretability Methods in Deep Neural Networks.” NeurIPS(2019).

summary_of_the_review: In summary this paper presents a series of interesting empirical results for smoothing techniques. However, I find the empirical results for the goodness of the smoothed attributions are not convincing to me, and the conclusions from the non-Lp attacks are not very new to the community, I remain negative for this paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors empirically evaluate the quality and robustness of 3 post-hoc and
2 ad-hoc approaches for robustification of gradient-based attributions under a
combination of 3 adversarial attack approaches which they use to target the
attributions.
They evaluate the robustness via cosine-distance and LPIPS to a target-heatmap,
and the quality via spearman correlation to the original under cascade
randomization of model parameters, the sparseness of the attribution maps using
the GINI index, and the explanation infidelity.
main_review: ### Strengths
- a broad set of evaluation metrics is used to measure the quality
- hyper-parameters are well-documented, reproducibility is high
- methods from prior work are studied thoroughly for their application
- the paper is structured and written well


### Weaknesses
- the sample set of almost all experiments is very small and should be increased
    - currently for post-hoc approaches on ImageNet:
        - 128 samples for the similarity to the target map
        - 100 samples for cascade randomization
        - 20 (!) samples for the segmentation
        - 100 samples for sparseness with the GINI index
        - 100 samples for infidelity
    - currently for the ad-hoc approaches on CIFAR-10:
        - 320 samples for the similarity to the target map
        - 100 samples for cascade randomization
        - 100 samples for sparseness with the GINI index
        - 100 samples for infidelity

- errors should be reported for the segmentation experiment in table 1, the
  cascade randomization (Figure 2 & 6), the GINI index (Table 2 & 4) and the
  infidelity (Table 3 & 5)

- only a single model on a single dataset is used for each experiment, which should be increased
    - VGG16 on ImageNet for post-hoc approaches
    - ResNet-18 on CIFAR-10 for ad-hoc approaches

- only a single run of the experiments is conducted with only a single seed
  (model parameters)
    - to test for statistical significance, the experiments should, in addition
      to more models and a higher sample size, be conducted multiple times with
      multiple seeds

- are the error bars in Figures 4 and 7 percentiles or the standard deviation?
- the errors in Figures 4 and 7 are very large ("compatible results" most of
  the time), but this is not discussed

- in the segmentation experiment in Table 1 only the 20 pixels with highest
  attribution values are investigated
    - why do the authors not use all pixels' attribution values?
    - if there is any reason behind this, it should be discussed, otherwise all
      pixels should be used
    - this approach measures spikey attribution maps in a different way than
      smooth ones, since the spikey attribution maps will have more of their
      total attribution scores measured

- in Table 3 the infidelity for Smooth and Uniform are only marginally better
  (1.43 vs. 1.42), yet in the text this is simply described as an improvement
  (this marginal improvement will probably even become less significant with
  errors reported)

- it seems a little suspicious that Smooth and Uniform Gradient have the exact
  same values for GINI Index and infidelity in Tables 2 & 3, though this may
  just be by chance

- for the similarity in Figures 4 & 7, it may give more insights to also
  compute the distance to the original, ie. dist(h(x_{adv}, h(x)))

- why is the segmentation experiment is not conducted for the ad-hoc methods?

- minor: a label "higher is better" for figures 4 & 7 could improve readability

summary_of_the_review: While the results of this work are potentially interesting, this paper has many
short-comings in its empirical evaluation, which is especially problematic
since it is the paper's main contribution. With small sample sizes, most errors
unreported, only a single model and dataset, and only a single run, there is a
lot of room to improve this work. As these changes involve running many more
experiments, for the sake of improving this work, I recommend a score of 3:
reject, not good enough.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper evaluates the quality and robustness of explanations of three post-hoc smoothing approaches (Smooth Gradient, Uniform Gradient, B-smoothing), and two ad-hoc smoothing approaches (CURE, Adv). It evaluates the quality of explanations based on the model parameter sensitivity, class sensitivity, sparseness, infidelity. It also evaluates the robustness of explanations to combinations of additive, spatial transformation, and recoloring attacks by comparing similarities between target and explanation maps. All evaluation is performed on publicly available benchmark datasets such as ImageNet and MS COCO. Based on their experimental results, the authors made several claims about the quality and robustness compared to the vanilla gradient method. For example, the authors claim that most of the smoothing methods are less sensitive to perturbation of model parameters so they may not be helpful to debug a model, and they may not be "robust" to non-additive types of attacks.  
main_review: Novelty and Significance:
The novelty and significance of this paper are in the experimental evaluation of "smoothing explanation approaches" using diverse evaluation measures. The major novelty and significance of this paper can come from section 3.2 that aims to show non-robustness of the smoothing methods. However, the experimental results do not show a significant difference between the three attacks, which degrades the novelty and significance. Section 3.1. experiment and result are similar to those in prior works, but still can provide somewhat meaningful observations  (i.e. insensitivity) about the smoothing approaches if error bar is provided.

Technical soundness: The experimental evidence is not fully supporting the authors' claims. Additional experiments and in-depth discussion are needed.
- Table1-3, 5: Please provide error bar (i.e. empirical confidence interval). If error bar is not small enough to distinguish the difference, please use more samples for evaluation.
- Figure 4, 7. The two metrics are not consistent with each other. For example, in figure 4, the non-additive attack is shown as effective for Smooth gradient and $\beta$-smoothing in the left figure but not in the right figure. Why do those two metrics show different results? 
- Figure 4, 7. Each smoothing approach shows no significant differences between the three attacks. It is quite hard to say that those methods are less robust to other types of attacks based on this experiment. Please provide more experimental evidence or use more samples to reduce the error bar. 

Writing and Clarity: Minor corrections are needed.
- Page 1. The first sentence 'Explanation methods attribute a numerical value ... ' is not always true. There are other explanation methods not using attribution scores. Please modify this sentence. 
- Page 2. Infidelity -> infidelity

summary_of_the_review: The novelty and significance of this paper are in the experimental evaluation of robustness of the smoothing explanations that were known to be robust and any quality degradation trade-off. However, the experimental evidence is not fully supporting the authors' claims. The paper should provide experimental evidence showing significant robustness reduction. Also in-depth discussion about the experimental results is needed.
review_title: null

Meta-review: The paper conducts a series of empirical studies to evaluate the robustness of smoothed attribution methods. Although the reviewers think this is an important direction, there are several concerns about the experimental settings, such as the sample size and the models to be tested. Also, one of the main finding that Lp based smoothing methods are non-robust to non-Lp norm perturbations is well known and is not that surprising.
Decision: Reject