“Why Not Other Classes?”: Towards Class-Contrastive Back-Propagation Explanations
Authors: Yipei Wang
Xiaoqian Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: explainable artificial intelligence
attribution explanations
contrastive explanations

Abstract: Numerous methods have been developed to explain the inner mechanism of deep neural network (DNN) based classifiers. Existing explanation methods are often limited to explaining predictions of a pre-specified class, which answers the question “why is the input classified into this class?” However, such explanations with respect to a single class are inherently insufficient because they do not capture features with class-discriminative power. That is, features that are important for predicting one class may also be important for other classes. To capture features with true class-discriminative power, we should instead ask “why is the input classified into this class, but not others?” To answer this question, we propose a weighted contrastive framework for explaining DNNs. Our framework can easily convert any existing back-propagation explanation methods to build class-contrastive explanations. We theoretically validate our weighted contrast explanation in general back-propagation explanations, and show that our framework enables class-contrastive explanations with significant improvements in both qualitative and quantitative experiments. Based on the results, we point out an important blind spot in the current explainable artificial intelligence (XAI) study, where explanations towards the predicted logits and the probabilities are obfuscated. We suggest that these two aspects should be distinguished explicitly any time explanation methods are applied.

"Why Not Other Classes?" Towards Class-Contrastive Back-Propagation Explanations
Yipei Wang, Xiaoqian Wang"
Elmore Family School of Electrical & Computer Engineering
Purdue University
West Lafayette, IN 47906
{wang4865, joynnn, joywang
Abstract
Numerous methods have been developed to explain the inner mechanism of deep neural network (DNN) based classifiers. Existing explanation methods are of- ten limited explaining predictions of a pre-specified class, which answers the question "why is the input classified into this class?" However, such explana- tions with respect not capture features with class discriminatiee power. That is, features that are important for predicting one class may also be important for other classes. To capture features with true llass-discimminativ power, we should instead ask "why is the input classified into this class, but not others?" To answer this question, we propose a weighted contrastive framework for explaining DNNs. Our framework can easily convert any existing back paok-pproag explanation methods to build class-contrastive explanations. We theoretically validate our weighted contrast explanation in general back poekppropagation explanations, and show that our frame- work enables colsssconta explanations with significant improvements in both qualitative and quantitative experiments. Based on the results, we point out an important blind spot in the current explainable artificial intelligence (XAI) study, where explanations towards the predicted logits and the probabilities are obfus- cated. We suggest that these two aspects should be distinguished explicitly any time explanation methods are applied.
single class are inherently insufficient because they do
Introduction
The black-box essence of Deep Neural Networks (DNNs) has been impeding the development and application of powerful deep learning tools in reality. Without providing promising reasoning process, DNNs are never fully trusted by end-users, especially for high-stake areas such as medical analysis, autonomous vehicles, etc. Recently, General Data Protection Regulation (GDPR) even explicitly proclaim the right to explanations for automated decisions (Selbst and Powles 2018). This further stimulates the demand for explainable DNNs. As result, the studies of eXplainable Artificial Intelligence (XAI) thrive explore the inner mechanism of DNNs and to build trustworthy models. Since then, countless explanation methods have been devised to serve such purposes. Methods developed have covered various aspects including pattern recognition, decision making, natural language inference, etc. Albeit seemingly prosperous, the lack of systematicness is an inevitable issue of XAI studies. One of the fundamental questions to be answered is what makes proper explanation to the black-box model? We need to know what end users are longing for before we develop the techniques. For the moment, this still remains open question there is unified framework for XAL. A reasonable workaround is to set specific goals (Wang and Wang 2022) and
"Corresponding author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Weighted Contrastive GradCAM
GradCAM
Engut Imango Albatcous
Black footed albutrous
Northern falenrr
Black Gooted alhatros
Northern fulmar
preducioos
Northern folonar
Black looted albatrora
Predicted probability change after blirring:
Black footed albatron::
0.003
0.221
Narthena fulmar: 0.731 -0953
0.731 0.952
0.731 0.002
0.731 0.962
Figure 1: Comparison between GradCAM and weighted contrastive GradCAM (proposed framework) on an image from CUB-200 dataset. The upper row contains heatmaps for two classes: black footed albatross (bfa.) and northern fulmar (nf.) The bottom row contains the blurred images according to the heatmaps (pixels with negative explanation values are blurred). Red/blue numbers indicate the changed probability with the input blurred images. The blue/red numbers are expected to decrease/increase if the explanations capture pixels with classddssciminnative power.
targeted users (Preece et al. 2018) for specific technique. As user-oriented aspect of AI, one of the most important duties of XAI to serve as the pipeline between the black-box models and the end-users. Therefore, it is crucial to pay attention to what end-users really need in explanations.
Here we take the classification problem as an example. when explaining classifier, most existing explanation methods seek answers to the question "why the sample classified as class t?" However, multiple studies (Lipton) 1990, Barnes, 1994; Miller (019)) have argued that when people seek explanations of classification, they implicitly ask "why is the sample classified as class t, but not others?" Consequently, there is a mismatch between what the end users desire and what current explanation methods can deliver. Building class-contrastive explanations (viz. finding features that distinguish one class from others) plays fundamental role in answering the question end users truly care about. However, class-contrastive explanations are less studied literature.
To address this challenge, in this paper we propose a novel method for class-contrastive explanations. We first introduce the criterion of notations for the sake of lucid presentation. All bold lowercase letters represent tensors. And unless especially specified, x, y, P. Φ represent tensors inputs, predicted logits. probabilitics and explanations, respectively. All normal lowercase letters represent scalars. Especially, d, c, represent the input dimension, output dimension, target class, respectively. Other letters will be clarified when they used.
In machine learning, classification tasks are generally formulated as a prediction model where the output logits E R are the regression predictions for each individual class. Afterwards, a softmax activation (or sigmoid for binary case) is applied to compute the probabilities p [0, 1]e of c classes. This framework is extensively applied in all kinds of classification tasks. However, for classification models, most existing
preppprppaaatoon explanation methods only focus on the output of a single logit given a target class / E [c]- This kind of explanations does not contain any information about other classes E [c). * f. This criterion only adequate for: regression tasks where the relation between y, and does not matter or ii) binary classification tasks (c = 1) when the model is itself contrastive. Otherwise, will raise problems as features important for one class can be equally important, or even more important for other classes. In fig. [ GradCAM and our proposed weighted contrastive GradCAM are compared. We observe that if we focus on the top two classes bfa. and nf. separately, the original GradCAM indeed correctly highlights the object of the image. Here since Yot. and Yata. are both relying on the object, the information provided by the explanations is limited. When features with negative explanation values are blurred according to the each class, it is expected that the probability corresponding class should increase - or at least it shouldn's 't decrease much. However, it can be found that since most of the object features are important for both classes, but more important for nf., the probability of bfa. even drops when the negative features are blurred (second column). As comparison, for the weighted contrastive GradCAM, when features with

---

negative explanations are blurred, the probability increases significantly. Also, since these two classes are dominant, increasing one of them automatically lead to the decay of the other one. More general experiments are in section [ Here we demonstrate via GradCAM as it is one of the most popular explanation methods, while the above issue also exists in other back-propagatioon methods. On the other hand, in most perturbation methods and Integrated Gradient (Sundararajan et al.) (017). the explanations are towards the probability instead of the logit, creating explanations in a far different way from other back-prope pre-pprppgatiin methods. But these methods did not explicitly distinguish the difference between explanations towards probability and logit.
Different as they are, explanations of the logit and the probability are usually obfuscated. Researchers seem to haven't 't noticed the difference caused by this choice. Also, since PyTorch classification models are usually built without the softmax activation, some methods (e.g. Integrated Gradient), even though they are proposed to explain the probability, are still widely deployed to explain the logit. Such problems can be found in popular tools such as captum and torchray This inconsistency can be fatal when evaluating and applying explanations open the black boxes.
Also, it should be noticed that the object area is not necessarily the "correct" area since the models may make the prediction based on unexpected features (such as the person in the top right corner). It is discovered that humans tend to evaluate probabilities only by representativeness (Kahneman et al., 1982), which means we may subjectively treat the consistency between the heatmap and the image as better explanation. Such bias is also referred to as plausibility (Jacovi and Goldberg, 2020), which is a vicious pitfall for evaluating explanations. We summarize our main contributions as follows:
We propose the weighted contrastive scheme, a framework for existing proppaatptntio methods that utilizes the information of all classes to generate class-contrastive explanations.
We validate that the proposed weighted contrastive scheme corresponds to back prok-prooagttion directly from the softmax probability.
We demonstrate the significant difference between explanations towards logits and probabil- ities, and point out the fatal obfuscation in existing XAI studies.
2 Related Work
According to the ways explanations are produced, general post-hoe explanation methods can be roughly split into perturbations, back-propagations, and approximations, Perturbation methods, such as RISE (Petsiuk et al.] 2018), extremal perturbations (Fong et al., 2019), SHAP (Lundberg and Lee, 2017) etc., try to generate explanations by purposely perturbing the input images. However, such methods usually require calling the black-box models multiple times. Generating explanation for one single input image may take up to 20 seconds, which is very inefficient in applications. Approximation methods use an external agent as the explainer for the black box models. Such as LIME (Ribeiro et al., 2016). FLINT (Parekh et al. (021). etc. Such explainers, as independent from the black- box models, may have trouble capturing the inner mechanism of the black boxcs. Back-propagation methods make use of the back-propagation scheme to generate gradient gradient- related explanations. SSmonyyan et al.] (2013) propose to use input gradient with respect to some output value as the feature explanations, which are essentially the local sensitivity of the input sample. Input x Gradient (Shrikumar et al. (0166) use the Hadamard product of the input and the gradient as explanations, such that more information of the input samples is included. Integrated Gradient (Sundararajan et al. (017) applies the gradient theorem to assign attributions to input features. GradCAM (Selvaraju et al. 2017) use the summation of CNN activations across channels weighted by the means of corresponding gradients. There are also rulc-based back- propagation methods that use different ways to propagate the scores, such as LRP (Bach al., 20155). DeepLift (Shrikumar et al. 2017). etc. Apart from general explanations, there are contrastive/ cetiraasiii//uunttrfactttuu explanations that especially focus on contrastive samples, which are similar to the original sample but is predicted differently. Most contrastive explanation methods achieve this by generating contrastive samples (Dhurandhar al. 2018 Liu al.) 2019 Agarwal and Nguyen| 2020; Jung et al. [220]] [Zhoo 2020), Looveren and Klaise| 2021). Such methods are achieved by sophisticated generative models, which can be very complex. This is against Occam's 's razor. Wang and Vasconcelos (2020) argue that is better to find existing samples as the contrastive ones instead of generating them. And Goyal et al. (2019) achieve this by searching algorithm, which is less efficient.
Our work most related to CLRP (Gu al., 2018), where information of multiple classes are taken in to considerations, But our proposed weighted scheme properly assign weights to different
3

---

classes, leading to better contrastive results. Also, CLRP only focuses on modifying LRP, while our scheme is applicable to various back-propagation methods. SGLRP (Iwana et al. 2019) consider the influence of the softmax operation in explanations. But they use the softmax gradient only as the initial relevance, which is insufficiently justified.
3 Motivation
3.1 Sanity Check for Back-Propagation Methods
Arguably acknowledged as self-inttepretaable, linear models' explanations are typical attribution explanations, where each input feature zy is assigned an explanation value, wfx,, for a given target class t. Here we take linear models linear models are the special case of explanations of ReLU networks (Ancona et al. 2017]. However, such attribution explanations are easy manipulate. Given target, score of any feature without modifying either the classification or the inmer mechanism. For example, increasing w (coefficient of the i-th feature for all classes 8 E [c] by the same amount d > 0 does not change the prediction result nor the prediction mechanism since y' = y 5x,1. WLOG, assume e, > O. However, the explanation value for feature 2; changed by dı; for any target class. This is obviously not a proper way to explain classification results. Contrastively, it should be noticed x; stays invariant under the modification. The necessity of contrast here is because z; becomes important for all classes, but single-class explanations will not capture this.
sanity check for existing criterion since attribution values of
can completely change the
that (wj o-I Eeje xx;
3.2 Sigmoid v.s. Softmax
For binary classification, c= 1 would be sufficient. For linear model, this means = wT x E R is an adequate prediction here, and p I+e-v is the probability of the positive class. Here w sufficiently explains the contribution of feature x; in predicting the probability. Compared with the redundant case where c = 2 and y = W x E IR?, neither w. x; nor w x4 is solely sufficient for explaining the classification result any more. They only explain y/1 and 1/2 respectively. Intuitively, (w: w ja; in softmax case (c = 2) plays the same role as W;T; in sigmoid case (c = 1) for binary classification. In fact, we have the following proposition. The proof is available in the Appendix.
Proposition 1. For binary classification, given the same data distribution P(x, u). wi w2 (c softmax) are equivalent to w (c- = 1, sigmoid) under gradient descent optimization.
As a result, the explanations with respect to y under sigmoid activation is equivalent to the explanations with respect 1/1 y/2 under (binary) softmax activation. That is, in the 2 case, need the contrastive explanations with respect to 1/1 -3 Y/2 for sufficiently explaining the classifier. However, most explanation methods either just consider y/ the explanation target, which creates an inconsistency with the binary classification case, or do not distinguish explicitly the difference between and p.
3.3 Weighted Contrast across Classes
For the sake of consistent explanations, it is required to perform class-contrastive explanations even for multi-class classification tasks. For the sake of sanity check, we revisit the linear model y WT x, and let (ar) denote the attribution score of feature towards class t. Then contrastive attributions are usually implemented by mean contrast (Gu et al. 2018) and max contrast (Dhurandhar al. 2018)) which, in linear case, can be summarized follows.
* Mean Contrast: (x)) = (wi T-T Extt wf)x;;
Max Contrast: (1) (wf - w )a, where 8* arg maX,34 Vs
These two schemes degenerates to Y1 - y2 when c = 2. But both of them have information loss. Mean contrast fails when different classes have different attribution scores for the same feature x;, while max contrast fail to capture the feature importance for all other e - 2 classes Hence, the judicious way is to combine the two contrast strategies above by summing over all classes with series of weights {a, ha Therefore, we propose weighted contrast as follows:
Weighted Contrast: (x(x) weighted = (wf Eofz arw D where the weights a, = e / elli)) are the softmax activation of UA E R (the predicted logit vector without the target class t), and Eest ag 1.

---

Softmax
Linear
DNNs
(x) Back-gropagsionn from logsta
(b) Contraative Bock-propagation
Fund-forward Pass
Figure 2: Illustration of the difference between (a) pack-propagation from logits and (b) contrastive back-propagation. In the linear classification layer + most existing methods only take one class into consideration, while contrastive methods cover all of them. A proper combination of such information is equivalent to the back-propagation from p.
This weighted contrast assigns class-wise importance based on the predicted logits y, of contrastive classes. In this manner, Vi [d]. information of all classes is included (for linear model, it is the row vector w, E IR of the coefficient matrix W [Rdxe) in the attribution score of feature T, Extending the weighted contrast to a general classifier beyond linear model, we have:
oi (x) weighted = (x) Σ 2a (x)).
(1)
Proposition 2. Weighted contrastive explanations (in ( towards predicted logits y are equivalent to explanations towards p softmax(y) E0,,11 1] in gradient methods.
In section 4 we extend proposition 2 and show the equivalence under a wide range of other back- propagation explanation methods.
3.4 Weighted Contrast in Back Propaaaaooa Methods
Although linear models are constantly argued to be too simple for theoretical analysis in deep learning research, it is of more significance in back- secck-ropagations. This because as an inductive process, nack-propagation is equivalent at each step. The back nack-propagation steps start from the top layer fi
to the bottom. fact, a classification DNN f x + y can decomposed as f f2 o fi where DNN and f2 and learnable parameters are shared across all classes, which means that the crucial classification process is still achieved by single linear model fz. Truncated from DNN becomes a linear model where the goal is to linearly classify that lies in the learnt manifolds. However, when methods only back-propy prec-propagate from single yı- they only utilize 1/c information of the linear model and the influence of losing of (c (0-1) 1)/c information becomes more significant as the number of classes c increases. And such significant loss will all be back-propagated equally to any lower layers. We prove the following proposition in the appendix.
is the linear classifier. As shown in fig. 2 in features
Proposition 3. For linear bropppropnn rules, the weighted contrast of back pack-propagated explanation from the logit is equivalent to back-propagating the weighted contrast of original explanations of any step.
4 Converting Back- Proppaarppa Explanations to Weighted Contrast
Based on the Jacobian Jyp, weighted contrast scheme is applicable various existing back- propagation-base based post-hoc explanation methods. And in practice, it does not require implementing one explanation method e times. We show that for most methods, it suffices to apply the original explanation schemes towards pæ instead of yı- Therefore, it can be performed using any existing implementations. We split prop-propnn methods into two categories based on the endpoint of the kack-propagation. The first class includes methods that back-propagate until the input space Ed such
5

---

methods include Gradient, Input x Gradient, Integrated Gradient, Layerwise Relevance Propagation, etc. The second class includes methods that back proppggte only activations of some CNN layers. such as GradCAM, Linear Approximation (LAA).
Gradient As the most commonly used attribution explanation method, the gradient (x), = V =ttt captures local sensitivity of yt Sc f((x) with respect the change æ. However, even being the target class, the change of y/ is not decisive to the classification result. It is the change of the relative relations between Y/ and vss E [c]. & f, ys- that decide the result. Note that (' weighted is equivalent to &' (z))p. and hence is the sensitivity of the predicted probability with respect to the input, which truly illustrates the contrastive features.
Input x Gradient (IxG) Since IxG the direct Hadamard product between the input and the gradient, that is. of (x))
O V Vallt. The weighted contrastive IxG is therefore defined as o' (((x) (eightte = (((x))) Σ ofc a x" (x) - (Valh Eefe 0g VaNs), which is equivalent to IxG explanation towards p.
Integrated Gradient (IG) As an application of the gradient theorem, IG requires computing the integral Jo af (Tz)dT, whose closed form is infeasible. (Hesse et al. 2021) show that for dz, ReLU DNNs without bias terms, the model f degenerates to homogeneous function, and thereby 0f(m), Therefore, IG with zeros baselines are equivalent to the IxG method.
So dz.
0f (ræ)dd= Dz.
Rule-Based Methods Different from back-propagation methods that rely on natural gradients, there process. Such as Layerwise Relevance Propagation (LRP)), Deepl.ift (DI.), etc. Due to the special structure of DNNs, such methods actually correspond to natural-gradien methods. LRP is equivalent to IxG in ReLU networks, and LRP, DI., İxG, IG are all equivalent under homogeneous networks (e.g. Rel. networks with no bias terms). Besides, they have implementation drawbacks constraints modules. Moreover, such methods require manually implementing the back- proppprpaggttinn instead of obtaining from the built-in gradient module. As result, their implementation time is longer than general back-propagation method.
are rule-based methods that slightly modify the
GradCAM Suppose at to be the output the CNN layer, where 0 < i, < d' are the spatial coordinates, and le is the channel. Then for GradCAM the explanation of class is
Byt Da
Φ'(x), = Σ
a
(2)
where we remove the last ReLU operation because the negative areas are treated contrastive features (Selvaraju et al., 2017). And the GradCAM explanation to pe is
Op: Uys Sy. dak
Op:
o'(x),
Σ
)a
(xx)
d2
Dy.
sEle]
sElc
Linear Approximation (LA) Different from GradCAM, LA does not use the mean value of global average pooling of the activation gradients as the weight for each channel. Instead, it calculates the activation gradient directly. Note that this is also equivalent LRP with respect the activation instead of the input æc Following the same notations, ø' (x), = Ee a Varyt. Then the weighted contrastive is equivalent propppropaggtion from Pt since
('xx), Lat k
a
Op: V Uv) = Σ Oy.
Op
Σ
k 3 Vat x ('(x)weeghted (4)
Dy.
SE
sEle
5 Experiments
In this section, we perform various experiments to demonstrate the applications of weighted con- trastive explanations. Based on the two genres of back propppaopag methods that we discussed in section 4 we devise different experiments to validate the advantages of using weighted contrastive back poopppropngntioon explanations. All experiments are implemented through Pytorch on Intel Core 19-9960X CPU @ 3.10GHz with Quadro RTX 6000 GPUs.

---

V9216
alexet
resnetle
Gensenet160
gouglene!
moBilenetV
phoud
Oiggtal
Meer
Waigees
Origial
Vian
Max
Weigtte
efficientnet--i
Hear
Max
Mear
Van
Wagnise
Orignel
Poun
Max
We grig
Orghr
Miar
Itaration
10, avg. yt iteration 10, avg. Pi
iteration 1, avg.
iteration avg.
eccuracy
accuraty
10, accurasy
Changes in accuracy, Y/ and pt (t is the target classification class) when certain input features are perturbed. The perturbed features are selected based on four gradient explanations (original, mean, max, weighted contrast), where "original" means explanations w.r.t. the logit directly, We show the results of commonly used CNNs with varying iteration numbers (iteration = 1,2,10) on the ILSV ILSVRC2012 dataset. The lighter-colored bars represent the average changes y-axis), and the two darker-colored bars represent the changes over Pt and accuracy (left y-axis). The results show: 1) the changes of pz and accuracy match each other, but changes in ye do not. This a support for using contrastive explanations; 2) our method shows higher pt and accuracy in all settings, indicating captures important features leading to the largest change in performance.
Figure 3:
y/ (right
5.1 Back-Propagation till the Input Space
recently discovered that explanation saliency maps that are over the original space incline to highlight the textures/silhouette of the input images (Adebayo et al., 2018). As result, visual evaluations of such saliency maps are easy be biased by the representativeness the more accurate the saliency map shows the original object, the better we humans might think it be. But in this case heuristic silhouette detector will ace this test. Besides, since the information of the original input image is dominant in the saliency map, changes in the qualitative results with visualizations are almost invisible because the scale differences. Therefore, we focus on quantitative results for such methods in the main paper. We leave the visualizations results and detailed descriptions in Appendix.
As all such methods are based or partially based on the input gradient, we propose to compare the recourse application of weighted contrastive gradient (Weighted), original gradient (Original), mean contrastive gradient (Mean), and max contrastive gradient (Max) methods by performing the input modification based on the gradient sign perturbation over 11.SVR20012 (Deng et al.) 2009) validation set, where 50000 images from 1000 classes included. The gradient sign perturbations are implemented following the projected gradient descent (Madry et al. 2017)
x n+1 xh + sign( (6 (a:)
x n+1 clamp( min(x e, 0), max( e,,
where we set 21,2, 10] to be the number of iterations, c 10 3 is the perturbation limit, and c/n is the step size. 9 commonly used CNNs are tested. The results are shown in fig. For each model, gray-ish, red-ish and blue-ish bars represent results of 1, 2. 10 iterations respectively. For given step, the wide shallow bar represents the change in yt. the thin median bar represents the change in Pt, and the thin deep bar represents the change in accuracy. It can found that weighted contrast outperforms all other methods in improving the probability of the target class and the accuracy. And
7

---

Dypart
Figure 4: Comparison between the back-propage propapptiaannn from logits (Original) and weighted contrastive back propagation (Weighted) over GradCAM and LA on images from the CUB-200 dataset. For each image, the top 4 images are from GradCAM and the bottom 4 images are from LA. The left/right column are explanations to the most/second possible class, respectively.
these two measures actually correspond with each other. On the contrary, the change in ye does not match the change in Pr or accuracy, since higher yr does not guarantee higher accuracy or pt.
5.2 Back- Pocc--Propggatoo till the Activation Space
For the methods that back-propagate to activation space, the activation is set up the last CNN layer. Due to the spatial invariance property of CNNs, the areas of activations are directly (but in an unknown manner) related to the corresponding areas of the original input. But without being biased by input silhouettes and textures, such methods' heatmaps usually demonstrate better localization power. Since one of the main goals of contrastive explanations is seeking fine-grained distinctions between features (Wang and Vasconcelos, 2020), here we carry out experiments over fine-grained datasets: CUB-200 (Wah et 1201 containing 200 classes of birds, SraenGr Visual Classification of Aircraft (FGVC) (Blaschko et al., 2012) containing 100 classes aircraft, Food- 101 (Bossard et al.) 2014) containing 101 classes of food, Flower- 1ow (Nilsback and Zisserman, 2008) containing 102 classes of flowers, and Stanford Cars (Krause et al. 2013) containing 196 classes of cars. We test
the results VGG-16 (Simonyan and Zisserman, 2014) and AlexNet (Krizhevsky) 2014) that pretrained by PyTorch on ImageNet (Deng et al.) 2009) and fine-tuned correspondingly.
Here we mainly focus on the contrastive explanations of the top 2 classes with the highest predicted probabilities, which conforms to what humans usually pay attention to in classification tasks. For example, people are usually interested in understanding why the model does not classify apple_pic as bread_ peradd_pudiing rather than why does the model not classifying apple_pil as greek_salass We clarify that our method is not limited to explaining 2 classes. In fact, our method can provide contrastive explanations with respect to any subset of classes of interest.
5.2.1 Visualizations
GradCAM and Linear Approximation results under the original form and the proposed scheme shown in fig. 4 Images are selected so that the second possible class has probability higher than 0.1. Comparing the two columns of each input image, we can find that for the two most possible classes, the original methods back- pock-propagates from the logits, which will always give very similar explanations, because two explanations are independent. Such explanation pairs convey limited information about the classification to the audience. On the contrary, weighted methods have focus on specific parts in explanations of different classes. Also, although the weighted explanations of the top classes seem to be complementary. they do not necessarily need to be. This phenomenon is because these two classes dominant over all classes. We also present very detailed analysis of the five bird images in appendix D Visualizations of other datasets can be found in appendix
5.2.2 Blurring/Masking
perform blurring experiments, where we blur positive and negative features decided by original and weighted GradCAM/LA, respectively. Since CNNs take input of fixed size, missingness is

---

Table 1: Comparisons between weighted contrastive method (wtd.) and original method (ori.) in blurring/masking experiments on 5 datasets. 3 baselines (Gaussian Blur, Zeros, Channel-wise Mean), and 2 methods (GradCAM, LA) are included. f1, 12 represent the classes with the highest and the second highest probability respectively. In each line, we show the changes of expected relative probability. Pos./Neg. Features mean that only positive/negative features are kept with respect to the corresponding i, class. is expected that when the positive/negative features corresponding to 4 are kept, the expected relative probability pj is expected to increase/decrease.
Cauvsian Bhr Neg. Fentures wid.
Charnel
Pt
Fox.
Festures
Fos. Features
Nog.
Feutures
Pos. Features
Neg.
Features
OEL
on.
wid.
oz.
w.d.
wtd.
wid.
LA
81
0,710
0.753
0.820
0.447
0.280
0.730
0.795
0.461
0.301
0.741
0.815
0.457
0.294
CUB-200
0.290
0.710
0.450
0.752
01845
0.749
0.417
0.178
0.447
0.830
0.723
0.444
0.424
0.205
0.418
0.844
0.741
0.446
0.410
0.187
0.250
82
0.290
0.406
0.776
0.418
0.137
0.410
0.762
0.416
0.152
0.403
0.773
0.409
0.141
LA
81
0.693
0.757
0
0.397
6.18
0.742
0.435
0,312
0736
0,772
0.45
0.287
PGvC
GC
0.695
0.307
0.617
0.792
0.627
0.826
0.862
0.775
0.352
0.377
0.368
0.193
0.192
0.606
0.757
0.763
0,813
0.711
0.302
0.417
0,421
0.261
0.256
0.709
0.61
0.835
0.742
0.379
0.405
0.404
0.179
0.230
0.251
T.A
0.714
0.805
0.572
0.344
0.795
0.836
0.575
0.848
0.361
0.200
Food 101
0.286
0.696
0,838
6,684
0.352
0,164
0.099
0.817
0.348
0.150
GC
61
0.714
0.828
0.907
0.339
0.128
0.211
0.891
0.353
0.151
0824
0.896
0.342
0.139
0,286
0.669
0,5888
0.317
0J185
0.668
0,865
0.334
0.106
0.689
0.874
0.328
0,098
LA
81
0,715
0.778
0.869
0.423
0.290
0,700
0,776
0.350
0.268
0.719
0.785
0.390
0.274
Flower-102
GC
f1 0.715
0.285
0.598
0.796
0.746
0.570
0.386
0.133
0.632
0.744
0.858
0.450
0.315
0.234
0.177
0.736
0.868
0.370
0.448
0.223
0.194
1g
0.285
0.539
0.505
0.305
0.110
0.545
0.835
0.416
0.170
0.544
0.827
0.411
0.164
Stanford
LA
4g 0.721
0,279
0.574
0.777
0,835
0.765
0.426
0.412
0.156
0.774
0.589
0,762
0.830
0,422
0.437
0.263
0.162
0.773
0.592
0.769
0.836
0.112
0:438
0.159
0.360
Cars
GC
0.279
0.721
0.552
0.810
0.899
0.830
0,405
0.416
0.199
0.565
0.803
0.894
0.829
0,421
0.407
0.296
0.804
0,898
0.419
0.402
0.205
not a well-defined concept and is achieved by removing original pixel values and imputing with heuristic baseline values. [Sturmfels et al. (2020) demonstrate that the baseline values have non- negligible impact on the results. Hence we include three different baselines respectively: 1) missing pixels are blurred; 2) missing pixels are imputed with zeros; 3) missing pixels are imputed with the (RGB) channel -wise mean values. Here VGG-16 is applied as the explained classifier, and heatmaps 14 14 are generated at the output of the CNN layers but before the last MaxPooling layer for higher resolutions. Then they are upsampled to the input space 224 x 224 by bilinear upsampling. The results are shown in table 1. we present the relative probability Pt, = Ble,",, /(e0hr + e )), 2, where I E [c] represent the i-th possible class. Expectations are calculated over samples where Pta > p. P is a threshold we fixed to be 0.1 here to ensure that the t class not to be too dominant. It can be found clearly that weighted methods outperform original methods in altering the probability. This result has verified the fact that back-propagation explanations from the logits fail to capture the difference between exclusive features and features shared among classes. In contrast, contrast method effectively captures positive and negative features for both t1 and f/2 classes with discriminative power. It should be noticed that here all samples covered in the experiment thus the results shown in table [ are deterministic and there is no uncertainty present table
weighted
5.3 Comparison with Mean/Max Contrast
Recall that mean contrast and max contrast have been applied to generate contrastive explanations. We compare the explanation results between the proposed weighted contrast and these two schemes. As discussed in section 3.3, both mean and max contrast schemes have inevitable information loss in explaining predictions of different classes. As shown in fig. [] for the mean contrastive, since the explanation is obtained by Dinean = Φ² T-I Exet Ø², where the c- non-target classes are treated equally. The classes with positive explanations and classes negative explanations are neutralized with each other by the simple summation. The denominator e 1 = 199 also decreases the contrast term a lot. As result, we can find that the mean contrast is almost the same as the original GradCAM. As for the max contrastive scheme, it only cares for the max non-target class, which is insufficient. As shown in the red boxes, we can find that the max contrastive explanations for these two classes are simply the additive inverse of each other. This is because when targeting one of them, the other automatically becomes "max non-target". Compared with the weighted contrastive explanations, does not contain any information of the 3rd 3rd-possible class (third row). For instance, in the top left
9

---

All grene
oogen
wAih
Grows biledani p 1:d8
p-m 0.622
hyboattinnon Ruk cow p
Mack wetker
bypoctionne for skby conlird
Enstrnttio fan 0.388
than
FREA
mann
Bronoed cowbine p- 1.204
Logpohwed stric
F-
Explasation for belled p
Exporation for Saperik 3
Oplimatoniioo for whit reched litr
Explinatoo Black thouted 0.15
Figure 5: Comparison with mean/max contrast. 4 images from CUB-200 are presented. For each image, we present the explanations for the top 3 classes predicted by the classifier in three rows in the top-down order. And in each column, explanations from the origin, mean contrastive, max contrastive and weighted contrastive GradCAM are presented.
block, when the head of the bird contributes both to the 1st Ist-possible class "groove billed ani" and the 3rd-possible class "shiny cowbird". the weighted contrastive explanation manages to find out "shiny cowbird". However, due to the absence of information from the
that it contributes 3rd-possible class, the max contrastive explanation fails to find this difference. Such explanations are insufficient to help understanding the behaviors of the black-box classifier.
Conclusions & Future Work
In this work, we propose a weighted contrastive scheme for back bock-propagation explanations that make proper use of information from all classes. The proposed scheme is applicable to wide range of existing nack-propagation methods and does not require additional explainers. We prove that the proposed weighted contrast, though include all classes, is actually equivalent directly explaining the probability (softmax activation), which only requires one back baok-propag for one sample. This makes it easy implement compared with existing contrastive explanation methods. We also demonstrate the significant difference between explanations to the logit y and to the probability p and point out that it is important distinguish them. Our experiments show that y-based explanations have better localization capability, while p-based explanations focus more on the contrast among classes. This should be decided based the desiderata of explanations case by
Albeit the desired results of the weighted contrast, admit that, due the simplicity of our model, the form of explanations are still limited to the heatmaps of the input sample. But is worth noticing the here the heatmaps can provide guidelines for generating contrastive explanations in complex models, since the highlighted areas are verified to be directly related to the classification results. We will explore the role of weighted contrastive explanations in generating/searching for contrastive samples in future work.
Acknowledgements
This work was partially supported by NSF IIS #1955890, IIS #21 #2146091, Purdue's Elmore ECE Emerging Frontiers Center.
10

---

References
Adebayo. J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B. (2018). Sanity checks for saliency maps. Advances in neural information processing systems, 31.
Agarwal, C. and Nguyen, A. (2020). Explaining image classifiers by removing input features using generative models. In Proceedings of the Asian Conference on Computer Vision.
Ancona, M., Ceolini, E., Öztireli, C., and Gross, M. (2017), Towards better understanding of gradient-based attribution methods for deep neural networks. arXiv preprint 1711::1711.000000
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., and Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, :0(7):+0
Barnes, E. (1994). Why p rather than q? the curiosities of fact and foil. Philosophical Studies, 73(1):35-53.
Blaschko, M., Girshick, R. B., Kannala, J., Kokkinos, I., Mahendran, S., Maji, S., Mohammed, S., Rahtu, E., Saphra, N., Simonyan, K., et (2012). Towards detailed understanding of objects and scenes in natural images.
Bossard, L., Guillaumin, M. and Gool, L. V. (2014). Food-101-mining discriminative components with random forests. In European conference on computer vision, pages 446 146-461. Springer.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248 248-255. leee.
Dhurandhar, A., Chen, P.-Y,, Luss, R., Tu, C.-C., Ting, P., Shanmugam, K., and Das, P. (2018). Explanations based the missing: Towards contrastive explanations with pertinent negatives. Advances in neural information processing systems, 31.
Fong, R., Patrick, M., and Vedaldi, A. (2019). Understanding deep networks via extremal perturba- tions and smooth masks. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2950-2958.
Goyal, Y., Wu, Z., Emst, J., Batra, D., Parikh, D., and Lee, S. (2019). Counterfactual visual explanations. In International Conference on Machine Learning, pages 2376-2384. PMLR.
Gu, J., Yang, Y., and Tresp, V. (2018). Understanding individual decisions of cnns via contrastive backpropagation. In Asian Conference on Computer Vision, pages 119-134. Springer.
Hesse, R., Schaub-Meyer, S., and Roth, S. (2021). Fast axiomatic attribution for neural networks. Advances in Neural Information Processing Systems, 34.
Iwana, B. K., Kuroki, R., and Uchida, S. (2019). Explaining convolutional neural networks using softmax gradient layer-wise relevance propagation. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pages 4176-4185. IEEE
Jacovi, A. and Goldberg, Y. (2020). Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? arXiv preprint arXhv:2004.03685.
Jung, H.-G., Kang, S.-H., Kim, H.-D., Won, D.-O., and Lee, S.-W. (2020). Counterfactual explanation based on gradual construction for deep networks. arXiv preprint arXiv:2008.01897,
Kahneman, D., Slovic, S. P., Slovic, P., and Tversky, A. (1982). Judgment under uncertainty: Heuristics and biases. Cambridge university press.
Krause, J., Stark, M., Deng, J., and Fei-Fei, (2013). 3d object representations for fine-grained categorization. In 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13), Sydney, Australia.
Krizhevsky, A. (2014). One weird trick for parallelizing convolutional neural networks. arXiv preprint ar Xiv:: 104.59444

---

Randomized input sampling for explanation of Lipton, P. (1990). Contrastive explanation. Royal Institute of Philosophy Supplements, 27:247-266.
Liu, S., Kailkhura, B., Loveland, D., and Han, Y. (2019). Generative counterfactual introspection for explainable deep learning. In 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pages 1-5, IEEE.
Looveren, V. and Klaise, J. (2021). Interpretable counterfactual explanations guided by prototypes. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 650-665. Springer.
Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in neural information processing systems, 30.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv preprint /rX6v:/706.0000 06083.
Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1-38.
Nilsback, M.-E. and Zisserman, A. (2008). Automated flower classification over large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing. pages 722-729. IEEE.
Parekh, J., Mozharovskyi, P., and d' Alché Buc, F. (2021), A framework to learn with interpretation. Advances in Neural Information Processing Systems, 34.
Petsiuk, V., Das, A., and Saenko, K. (2018). Rise: black-box models. arXiv preprint
Preece, A., Harborne, D., Braines, D., Tomsett, R., and Chakraborty, S. (2018). Stakcholders in explainable ai. arXiv preprint arXiv:
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144.
Selbst, A. and Powles, J. (2018). "meaningful information" and the right to explanation. In Conference on Fairness, Accountability and Transparency, pages 48-48. PMLR.
Selvaraju, R., Cogswell, M., Das. A., Vedantam, R., Parikh, D., and Batra. D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618-626.
Shrikumar, A., Greenside, P., and Kundaje, A. (2017). Learning important features through propagat- ing activation differences. In International conference machine learning, pages 315531553 PMLR.
Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. (2016). Not just black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713
Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualis- ing image classification models and saliency maps. arXiv preprint arXiv:1312.6034.
Simonyan, K. and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409. 155..
Sturmfels, P., Lundberg, S., and Lee, S.-I. (2020). Visualizing the impact of feature attribution baselines. Distill, 5(1):e22.
Sundararajan, M., Taly, A., and Yan, Q. (2017). Axiomatic attribution for deep networks. In International conference on machine learning, pages 3319-3328. PMLR.
Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011). The caltech-uesd birds-200- 2011 dataset.
12

---

Wang, P. and Vasconcelos, N. (2020). Scout: Self-aware discriminant counterfactual explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8981-8990.
Wang, Y. and Wang. X. (2022). A unified study of machine learning explanation evaluation metrics. arXiv preprint arXiv:2203.
Zhao, Y. (2020). Fast real-time counterfactual explanations. arXiv preprint arXiv:2007.05684.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [No]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [No]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releusing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [No]
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're
using/curating? [No]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [No]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "hO8bs2ismCy", "reviewer": "Reviewer_PhZB", "summary": "The authors propose a method (Weighted Contrast) for explaining DNN classifier predictions. Rather than focusing on features that change the predicted probability of a given target class, the authors’ method focuses on features that are important for one class and _not_ others.", "questions": "See previous \"Weaknesses\" section.", "limitations": "The authors briefly discussed the limitations of their work in the conclusion section. I would strongly recommend the authors expand the limitations section and specifically discuss how backpropagating with respect to softmax/sigmoid outputs may lead to issues with some previously proposed methods (see \"Weaknesses\" point 3).\n\nThe authors do not discuss potential negative societal impact for this work, which I think is fine (I don't see any obvious potential for negative impacts).", "rating": 4, "confidence": 4, "soundness": 1, "presentation": 1, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. As shown in previous work and by the authors, so-called contrastive explanations can better capture features that discriminate between classes for a given model as compared to non-constrastive explanations.\n2. The method proposed by the authors is an extension of gradient-based explanation methods, making it faster than some previously proposed contrastive explanation methods.\n\nWeaknesses (listed in order of importance to my score):\n\n1. Lack of baselines: Many recent works have focused on developing contrastive explanation methods. Although the authors do discuss some of these in their literature review, they do not compare their proposed method against any of these previous works in their experiments; instead, the previously proposed methods are dismissed as being \"against Occam's razor\". While I acknowledge the authors' point that their proposed method may be computationally cheaper than previously proposed methods, it's necessary to understand whether this speed comes at the expense of explanation quality. For example, with non-contrastive explanations, simpler gradient-based approaches have been shown to produce worse quality explanations than the axiomatically justified (but computationally more intensive) integrated gradients or SHAP methods. Without any comparisons against previously proposed methods, it is difficult to assess the significance of the authors' results.\n2. Novelty/additional discussion on the choice of backpropagating with respect to $p$ or $y$: The authors' show that a \"weighted contrastive explanation\" of logits $y$ is equivalent to a standard explanation of softmax/sigmoid outputs $p$. However, to my knowledge backpropagating with respect to $p$ is already standard practice for some previously proposed methods (see e.g. the official Tensorflow integrated gradients tutorial at https://www.tensorflow.org/tutorials/interpretability/integrated_gradients). Given this, could the authors clarify what exactly the contribution of this work is? Is it just the theoretical perspective suggesting that one should _always_ backpropagate with respect to softmax/sigmoid outputs as opposed to logits or something more? Moreover, for other methods (e.g. input x gradients/LRP) it is specifically recommended _not_ to backpropagate with respect to softmax/sigmoid outputs (see [1] Sections 2.4/2.5). Could the authors comment on this?\n3. Presentation of experimental results: In addition to point (1), I found it difficult to assess the authors' experimental results due to issues with figure design. Specifically, for the bar charts in Figure 3, the two darker-colored bars are superimposed on the lighter-colored one for each group of bars, making the chart difficult to read. Moreover, I found Figure 4 hard to evaluate without the labels of the most and second-most likely classes.\n4. Writing clarity: I found it difficult to understand the authors' point in parts of the manuscript due to issues with writing clarity. Specifically, Section 3 felt quite rushed, and I had to read the section multiple times to connect the propositions with the preceding text. \n\n[1]: \"Not Just A Black Box: Learning Important Features Through Propagating Activation Differences\" https://arxiv.org/abs/1605.01713", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "_W1xd5pvaPF", "reviewer": "Reviewer_aXtB", "summary": "In this paper, the authors argued for the need of \"class-contrastive\" explanations, which not only explain \"why an input is classified into a particular class,\" but also \"why the input is NOT classified as others.\" To obtain class-contrastive explanations, the authors proposed the \"weighted contrast\" explanation scheme, which is similar to the previous \"mean contrast\" explanation scheme except for the weights assigned to the non-target classes -- namely, in \"mean contrast\" explanation scheme all non-target classes have the same weight, whereas in the proposed \"weighted contrast\" explanation scheme, the weight for a particular non-target class depends on the softmax normalized score for that non-target class against the scores for all non-target classes. The authors showed that their proposed weighted contrastive explanation scheme is equivalent to standard explanation applied to the softmax normalized output of the model, thereby simplifying the implementation of their explanation scheme. Finally, they applied their weighted contrastive method to various back-propagation explanation techniques, including gradient, input x gradient, integrated gradient, GradCAM, and linear approximation (LA), and showed quantitatively that their weighted contrastive explanation scheme can find input regions that are better associated with target-class probabilities, and also qualitatively that their weighted contrastive explanation scheme (when applied to GradCAM and LA) provides more informative explanations.", "questions": "See the \"weaknesses\" above -- I have questions regarding the validity of the last part of equations (3) and (4), as well as the validity of the claim that integrated gradient with zero baselines is equivalent to the input x gradient method. Please provide mathematical proofs of both statements.", "limitations": "The authors have adequately addressed the limitation that their contrastive explanation method is only applicable to techniques where attributions (\"heatmaps\") are involved.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n+ The authors have provided very solid theoretical reasoning for where the current explanation techniques fall short and why we need \"class-contrastive\" explanations.\n\n+ The authors have provided an easy way to obtain \"class-contrastive\" explanations -- namely, applying standard techniques not to the logits (unnormalized class scores) but to the predicted probabilities (after softmax normalization).\n\n+ The quantitative measurement of changes in predicted probabilities by perturbing/blurring/masking pixels or input regions identified as important does show that the proposed \"class-contrastive\" explanation scheme is superior in finding input regions that are relevant for a target class.\n\nWeaknesses:\n\n- I find it difficult to understand equations (3) and (4) -- in particular, why is the weighted sum of standard GradCAM/LA explanations for all classes (where the weight for a class is the derivative of the target class probability with respect to the unnormalized score of that class) \"approximately equal\" to the weighted contrastive explanation? Please show your proof.\n\n- I also find it difficult to see how integrated gradient with zero baselines is equivalent to the input x gradient method. Please show your proof.\n\nA minor issue regarding related work:\n\n- ProtoPNet (Chen et al., 2019) only provides similar examples but not contrastive examples. In addition, ProtoPNet does not require additional annotations. In fact, ProtoPNet (and a number of other works) does not belong to the \"posthoc\" explanation family.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "E_utbVinM6I", "reviewer": "Reviewer_Ca7X", "summary": "The paper presents a new method for contrastive ('why class x and not class y', as opposed to 'why class x') explanations of classification networks. The main idea is to use the attributions of other classes, weighted by the softmax of their logits. The methods's applicability as an extension of existing explainability methods is shown, as well as its computational efficiency: While using information from all of the classes, ir only requires a single backward pass.", "questions": "* Figure descriptions - it would help if shortcuts bfa and nf from Fig. 1 were explained in the description. Also, the meaning of the colours (red/blue) is only explained in the first figure. \n* Figure 4 - the names of the top 2 classes are not mentioned, which might be usefull for understandign the method. The visual comparisons only compares the proposed method to non-contrastive methods, comparison with the max/mean contrastive method could provide more insight.\n* Linear approximation - not mentioned in related work, is there a reference?\n* Backpropagation methods - term used in multiple places in the paper - might lead to confusion with regular back-propagation algorithms in NNs, rather than explainability\n* The term 'most possible classes' used ie. in Figure 4 - are these the classes with the highest probability? \n* A couple of typos/grammar mistakes: 'But without biased' on line 260, 'for examples' on line 272, 'why does the model nto classifying' on line 273, ...\n* Line 303-304: Could you explain what is meant by 'It should be noticed that here all samples are covered in the experiment thus there is no uncertainty present in the results of table 2.'? Also, there is no Table 2, the reference should be to Table 1.", "limitations": "Limitations are properly addressed in the conclusions section. \n", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Originality** - while the extension of existing methods is very simple, it has not been used before in the context of explainability (to the best of my knowledge). Related work is properly cited and the main contributions are well separated from previous work.\n\n**Quality** - all the claims are well supported theoretically and the authors provide convincing experimental evaluation. The work is self-contained but proposes directions for future work. \n\n**Clarity** - the paper is well oeganized, easy to follow and written in good English. Main ideas are explained clearly. Information about details of finetuning required to reproduce the reported numbers is missing (line 269, 'fine-tuned correspondingly').\n\n**Significance** - the method is a simple extension of existing methods, limited to a specific task yet novel and well supported both theoretically and experimentally. The method relies on softmax weights, so is not directly applicable to ie. multilabel classification.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept