Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper presents a new method for contrastive ('why class x and not class y', as opposed to 'why class x') explanations of classification networks. The main idea is to use the attributions of other classes, weighted by the softmax of their logits. The methods's applicability as an extension of existing explainability methods is shown, as well as its computational efficiency: While using information from all of the classes, ir only requires a single backward pass.
strengths_and_weaknesses: **Originality** - while the extension of existing methods is very simple, it has not been used before in the context of explainability (to the best of my knowledge). Related work is properly cited and the main contributions are well separated from previous work.

**Quality** - all the claims are well supported theoretically and the authors provide convincing experimental evaluation. The work is self-contained but proposes directions for future work. 

**Clarity** - the paper is well oeganized, easy to follow and written in good English. Main ideas are explained clearly. Information about details of finetuning required to reproduce the reported numbers is missing (line 269, 'fine-tuned correspondingly').

**Significance** - the method is a simple extension of existing methods, limited to a specific task yet novel and well supported both theoretically and experimentally. The method relies on softmax weights, so is not directly applicable to ie. multilabel classification.
questions: * Figure descriptions - it would help if shortcuts bfa and nf from Fig. 1 were explained in the description. Also, the meaning of the colours (red/blue) is only explained in the first figure. 
* Figure 4 - the names of the top 2 classes are not mentioned, which might be usefull for understandign the method. The visual comparisons only compares the proposed method to non-contrastive methods, comparison with the max/mean contrastive method could provide more insight.
* Linear approximation - not mentioned in related work, is there a reference?
* Backpropagation methods - term used in multiple places in the paper - might lead to confusion with regular back-propagation algorithms in NNs, rather than explainability
* The term 'most possible classes' used ie. in Figure 4 - are these the classes with the highest probability? 
* A couple of typos/grammar mistakes: 'But without biased' on line 260, 'for examples' on line 272, 'why does the model nto classifying' on line 273, ...
* Line 303-304: Could you explain what is meant by 'It should be noticed that here all samples are covered in the experiment thus there is no uncertainty present in the results of table 2.'? Also, there is no Table 2, the reference should be to Table 1.
limitations: Limitations are properly addressed in the conclusions section.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this paper, the authors argued for the need of "class-contrastive" explanations, which not only explain "why an input is classified into a particular class," but also "why the input is NOT classified as others." To obtain class-contrastive explanations, the authors proposed the "weighted contrast" explanation scheme, which is similar to the previous "mean contrast" explanation scheme except for the weights assigned to the non-target classes -- namely, in "mean contrast" explanation scheme all non-target classes have the same weight, whereas in the proposed "weighted contrast" explanation scheme, the weight for a particular non-target class depends on the softmax normalized score for that non-target class against the scores for all non-target classes. The authors showed that their proposed weighted contrastive explanation scheme is equivalent to standard explanation applied to the softmax normalized output of the model, thereby simplifying the implementation of their explanation scheme. Finally, they applied their weighted contrastive method to various back-propagation explanation techniques, including gradient, input x gradient, integrated gradient, GradCAM, and linear approximation (LA), and showed quantitatively that their weighted contrastive explanation scheme can find input regions that are better associated with target-class probabilities, and also qualitatively that their weighted contrastive explanation scheme (when applied to GradCAM and LA) provides more informative explanations.
strengths_and_weaknesses: Strengths:

+ The authors have provided very solid theoretical reasoning for where the current explanation techniques fall short and why we need "class-contrastive" explanations.

+ The authors have provided an easy way to obtain "class-contrastive" explanations -- namely, applying standard techniques not to the logits (unnormalized class scores) but to the predicted probabilities (after softmax normalization).

+ The quantitative measurement of changes in predicted probabilities by perturbing/blurring/masking pixels or input regions identified as important does show that the proposed "class-contrastive" explanation scheme is superior in finding input regions that are relevant for a target class.

Weaknesses:

- I find it difficult to understand equations (3) and (4) -- in particular, why is the weighted sum of standard GradCAM/LA explanations for all classes (where the weight for a class is the derivative of the target class probability with respect to the unnormalized score of that class) "approximately equal" to the weighted contrastive explanation? Please show your proof.

- I also find it difficult to see how integrated gradient with zero baselines is equivalent to the input x gradient method. Please show your proof.

A minor issue regarding related work:

- ProtoPNet (Chen et al., 2019) only provides similar examples but not contrastive examples. In addition, ProtoPNet does not require additional annotations. In fact, ProtoPNet (and a number of other works) does not belong to the "posthoc" explanation family.
questions: See the "weaknesses" above -- I have questions regarding the validity of the last part of equations (3) and (4), as well as the validity of the claim that integrated gradient with zero baselines is equivalent to the input x gradient method. Please provide mathematical proofs of both statements.
limitations: The authors have adequately addressed the limitation that their contrastive explanation method is only applicable to techniques where attributions ("heatmaps") are involved.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors propose a method (Weighted Contrast) for explaining DNN classifier predictions. Rather than focusing on features that change the predicted probability of a given target class, the authors’ method focuses on features that are important for one class and _not_ others.
strengths_and_weaknesses: Strengths:

1. As shown in previous work and by the authors, so-called contrastive explanations can better capture features that discriminate between classes for a given model as compared to non-constrastive explanations.
2. The method proposed by the authors is an extension of gradient-based explanation methods, making it faster than some previously proposed contrastive explanation methods.

Weaknesses (listed in order of importance to my score):

1. Lack of baselines: Many recent works have focused on developing contrastive explanation methods. Although the authors do discuss some of these in their literature review, they do not compare their proposed method against any of these previous works in their experiments; instead, the previously proposed methods are dismissed as being "against Occam's razor". While I acknowledge the authors' point that their proposed method may be computationally cheaper than previously proposed methods, it's necessary to understand whether this speed comes at the expense of explanation quality. For example, with non-contrastive explanations, simpler gradient-based approaches have been shown to produce worse quality explanations than the axiomatically justified (but computationally more intensive) integrated gradients or SHAP methods. Without any comparisons against previously proposed methods, it is difficult to assess the significance of the authors' results.
2. Novelty/additional discussion on the choice of backpropagating with respect to $p$ or $y$: The authors' show that a "weighted contrastive explanation" of logits $y$ is equivalent to a standard explanation of softmax/sigmoid outputs $p$. However, to my knowledge backpropagating with respect to $p$ is already standard practice for some previously proposed methods (see e.g. the official Tensorflow integrated gradients tutorial at https://www.tensorflow.org/tutorials/interpretability/integrated_gradients). Given this, could the authors clarify what exactly the contribution of this work is? Is it just the theoretical perspective suggesting that one should _always_ backpropagate with respect to softmax/sigmoid outputs as opposed to logits or something more? Moreover, for other methods (e.g. input x gradients/LRP) it is specifically recommended _not_ to backpropagate with respect to softmax/sigmoid outputs (see [1] Sections 2.4/2.5). Could the authors comment on this?
3. Presentation of experimental results: In addition to point (1), I found it difficult to assess the authors' experimental results due to issues with figure design. Specifically, for the bar charts in Figure 3, the two darker-colored bars are superimposed on the lighter-colored one for each group of bars, making the chart difficult to read. Moreover, I found Figure 4 hard to evaluate without the labels of the most and second-most likely classes.
4. Writing clarity: I found it difficult to understand the authors' point in parts of the manuscript due to issues with writing clarity. Specifically, Section 3 felt quite rushed, and I had to read the section multiple times to connect the propositions with the preceding text. 

[1]: "Not Just A Black Box: Learning Important Features Through Propagating Activation Differences" https://arxiv.org/abs/1605.01713
questions: See previous "Weaknesses" section.
limitations: The authors briefly discussed the limitations of their work in the conclusion section. I would strongly recommend the authors expand the limitations section and specifically discuss how backpropagating with respect to softmax/sigmoid outputs may lead to issues with some previously proposed methods (see "Weaknesses" point 3).

The authors do not discuss potential negative societal impact for this work, which I think is fine (I don't see any obvious potential for negative impacts).
review_title: null

Meta-review: metareview: Reviewers expressed overwhelmingly positive opinions about the simple, easily implementable, and at the same time innovative procedure proposed in the paper for obtaining gradient-based class-contrastive explanations. Appreciation also transpired for the significance of this work in clarifying some technical points in the gradient-based XAI literature and the potential for future work that the paper opens up.
One of the main criticisms raised in the reviews, the lack of comparisons against other contrastive explanation methods, has been addressed satisfactorily with additional experiments and discussions in the rebuttals.
The most important remaining criticism was a doubt on the merits of one of the key technical points in the paper regarding whether gradient-based attributions should be computed according to the softmax outputs or logits. Reviewers pointed out that computing attributions with respect to softmax outputs instead of logits is already common practice in the field.
Reviewers expressed strongly the opinion that it would be appropriate to characterizing and clarify this situation, as it could be potentially misleading and indeed counterproductive even for the paper to denote attribution methods with respect to logits as "standard", while it's instead the case that some implementation of gradient-based attribution methods already attribute with respect to the softmax output (albeit inconsistently).
In conclusion, the reviewing panel voted for accepting the paper, under the condition that the camera-ready version of the paper explicitly clarify the distinction between the two approaches and discuss the implication of choosing one of the other, without however referring to attribution with respect to logits as standard, but merely pointing out that until now the distinction has been vague and implementations inconsistent. From this technical standpoint, Reviewers ask that the contribution of the paper should then be explicitly characterized as clarifying the distinction between logits and softmax attributions, rather than as the proposal of a new procedure in opposition to an already established standard. This is already perceived as a strong contribution to the community, as phrasing it specifically as indicated would help elucidate the state of affairs in the literature and make the community aware of this outstanding blind-spot.
confidence: Certain
Decision: Accept