Pixab-CAM: Attend Pixel, not Channel
Authors: Jaeeun Jang
Seokjun Kim
Hyeoncheol Kim
Venue: ICLR 2022 Submitted (2022)
Keywords: Explainable AI
Interpretable ML
Visual explanation of CNN
Class activation maps
Computer Vision

Abstract: To understand the internal behaviors of convolution neural networks (CNNs), many class activation mapping (CAM) based methods, which generate an explanation map by a linear combination of channels and corresponding weights, have been proposed. Previous CAM-based methods have tried to define a channel-wise weight that represents the importance of a channel for the target class. However, these methods have two common limitations. First, all pixels in the channel share a single scalar value. If the pixels are tied to a specific value, some of them are overestimated. Second, since the explanation map is the result of a linear combination of channels in the activation tensor, it is inevitably dependent on the activation tensor. To address these issues, we propose gradient-free Pixel-wise Ablation-CAM (Pixab-CAM), which utilizes pixel-wise weights rather than channel-wise weights to break the link between pixels in a channel. In addition, in order not to generate an explanation map dependent on the activation tensor, the explanation map is generated only with pixel-wise weights without linear combination with the activation tensor. In this paper, we also propose novel evaluation metrics to measure the quality of explanation maps using an adversarial attack. We demonstrate through experiments the qualitative and quantitative superiority of Pixab-CAM.

Under review as a conference paper at ICLR 2022
PIXAB-CAM: ATTEND PIXEL, NOT CHANNEL
Anonymous authors Paper under double-biee brind review
ABSTRACT
To understand the internal behaviors of convolution neural networks (CNNs), many class activation mapping (CAM) based methods, which generate expla- nation map by a linear combination of channels and corresponding weights, have been proposed. Previous CAM-baseed methods have tried to define channel-wise weight that represents the importance of a channel for the target class. However, these methods have two common limitations. First, all pixels in the channel share a single scalar value. If the pixels are tied to specific value, some of them are over- estimated. Second, since the explanation map the result a linear combination of channels in the activation tensor, it is inevitably dependent on the activation ten- sor. To address these issues, we propose gradient-free Pixel-wise Ablation -MMM FrxxabCAAM which utilizes pixel-wise weights rather than channel-wise weights to break the link between pixels in a channel. In addition, in order not to gener- ate explanation map dependent is generated only with pixel-wise weights without linear combination with the activation tensor. In this paper, we also propose novel evaluation metrics mea- sure the quality of explanation maps using an adversarial attack. We demonstrate through experiments the qualitative and quantitative superiority of Pixab-CAM.
the activation tensor, the explanation map
INTRODUCTION
As the performance of convolution neural networks (CNNs) converges to some extent Szegedy al.
2015 Simonyan & Zisserman 2014] Huang et al. 2017 Howard et al. |017 Chollet] 2017), 7), re- searchers have become interested in interpreting their predictions(Sprineebbre
[20 S
et al. 2017 Fong & Vedaldi [0177] Bach et al. 2015 Lundberg & Lee 201 Ribeiro et al. 2016 Petsiuk et al. [2088] Shrikumar al. 2017). One of them class activation mapping (CAM) based approach(Zhou et al.) (016), which generates an explanation map by a linear combination of chan- nels the activation tensor and corresponding weights. The explanation map L2 for the target class
e is calculated as Eqn.] 1 A represents activation of pixel at spatial location (i,j) for k-th channel of the activation tensor A, and wi denotes the coefficient of the k-th channel for class
Di Σ w A
The activation tensor A fixed since it is determined by the given input image and pretrained (LCuu et al. 1998). Therefore, L depends on how w is defined. Thus, different wf. have been proposed in many CAM-based methods to generate high-quality explanation maps. The previ- ous CAM-based methods can be divided into two categories, one is gradient- based CAMs(Sellvaajuu Chattopadhay et al. 2018 Fu et al. 2020), and the other is gradient-fred CAMs Wang al. 2020 Ramaswamy al. 2020). Gradient- based CAMs and gradient-free CAMs be dis-
let al. 2017
tinguished by whether or not gradients are used the process of deriving weights.
1.1 GRADIENT-RASED CAMS
al. 2017) obtains the weight of k-thh channel by averaging their gradients. However, it has a problem in that the weight is crushed due to the offset of positive and negative gradients. (Z is the number of pixels in a channel and y is the classification score for class c)
Frad-CAMMSSdde
wf ΣΣ aYe
DA

---

Under review as a conference paper at ICLR 2022
Grad-CAM++ Chattopahhay al. 2018) introduced higher-orde derivatives at in Grad-CAM equation to solve the problem that channels with fewer footprints fade away in the explanation map.
-x
ΣΣ
kc aij
ReLU
a
XGrad-CAM(Fu et al. 2020) points out that previous CAM-baser methods lack sufficient theoreti- cal support and propose weight that satisfies two basic axioms: Sensitivity and Conservation.
AN Σ,E, AM a
eY:
W = ΣΣ
(4)
However, Gradient-based CAMs suffer from gradient saturation problems, leading to localization
failures for relevant regions. WWag et al. 2020
Ramaswamy al. 2020 Jung & Oh] 2021
1.2 GRADIENT-FREE CAMS
Mrore-CAM(V) Wang et al. 2020 defines "increase of confidence" as the coefficient of the channel. For input X and baseline input X', the weight of the k-th channel for class e is defined
wf = J(X su(( ())) - f(X,)
(5)
where ua-) indicates the upsampling operation into the input size, s(-) is function that normalizes all elements to [0, 1] and is the "Hadamard product".
et al. 2020), which inspired our research, defines the weight of spe- eific channel as the fraction of drop in activation score of target class when that channel removed. In the equation below, Yg is the score of class e when the k-th channel has been replaced by zero.
Aitation-CAMMaamaswwaaaa
Y Y
Ye
Gradient-free CAMs remove the dependence of gradients, but they are time-comsuming because the prediction process must be repeated as many times as the number of channels to acquire the weights.
Regardless of whether gradient is used or not, all CAM-based methods so far have two things in common. First, they all use channel-wise weights w (see Eqn However, since the channel-wise weights allow all pixels in specific channel to share single weight, it is an unfair weighting method from the pixel perspective. (Although ke Grad-CAM++ (Eqn(3) modifies the gradients at the pixel level, but these gradients are eventually crushed into single scalar value, so it doesn't 't solve the problem of pixels being tied together.) Furthermore, channel- -wiieel--wieeeee weight-based methods tend to assign high weight if there pixels that have high contribution to the target class among the active pixels in the channel. Therefore, within a specific channel, other pixels that are active with the pixels related to the target class are also given high weight. If channels containing these overestimated pixels are accumulated, overestimated pixels also occur in the explanation (spe Sec Second, when generating the explanation map L all CAM-based methods follow Eqn which contains the activation tensor A. Therefore, the explanation map has no choice but to depend on the activation tensor, and eventually may capture too much meaningless information since the channels in the activation tensor are not necessarily related to the target category (Zhang et al. 2021). Sec 3.2
2 PROPOSED EVALUATION METRICS OF THE EXPLANATION MAP
Since it is hard to know where CNN looks and makes a specific prediction, the existing evalua- tion metrics of the explanation map have limitations. AD & IIC/Chattopadhay et al.) 2018) and Deletion & Insertion(Petsiuk et al. 2018) are coarse proxy evaluations because removing pixels by zero-ing/gren pero-ing/graying is unnatural in real photos(Dabkows) & Gal 2017). Also, in Localization Evalu- ationQWan al. 2020), there may not be correlation between localization ability and the quality of the explanation map since the human-annotater bounding box may not match where CNN is ac- tually focusing. However, we could know where CNN is looking, we could make a more accurate assessment. We use an adversarial attack for this. (we use FGSM attock et al. 2014).) If a targeted adversarial attack is made on a specific patch in the zero image, it is self-evident that CNN sees that patch and determines that it is target, the performance can be evaluated by whether the explanation map correctly captures that part. We propose 2 evaluation tasks (DAPI and DAP2) from this idea. Before introducing them, will explain 2 adversarial examples (AE1 and AF2) first.
2

---

Under review as a conference paper at ICLR 2022
2.1
ADVERSARIAL EXAMPLE (AE1)
(a) 7em Image
(h) Adversarial Parch
(c) Input Image
(d) Zero-ing Patch
(e) Adversarial Example
Figure 1: Adversarial patch in (b) was attacked to become "cucumber".
It is simple attach adversarial patch classified as given target class to input image. First, we apply a targeted adversarial attack so that the patch at a specific location in the zero image is classified into the class we want. Since the attack is applied only to the patch, the rest of the area is still zero as in (b). Then, the imput image with only the patch area of zero((d)) is added to the image (b). The finally obtained adversarial example AEI((e)) used for our evaluation task DAPI.
2.2 ADVERSARIAL EXAMPLE (AB2)
(a) Adversarial Patch Adversarial Patch (c) Adversarial Patch
(d) Adversarial Patch 4
(e) Adversarial Example
Figure (a): "chihuahua", (b): "tusker", (c): "soccer ball",, (d): "broccoli"
For generating a second adversarial example AE2, only adversarial patches are used without input images. This can also be easily created. After generating four adversarial patches in the same way as in Sec combine them to make the final adversarial example AE2. (Each adversarial patch is attacked to become different class.) AE2 created in this way is used for evaluation task DAP2.
2.3 DETECTING ADVERSARIAL PATCI
Explanation Map A
Pointng Game A 871 Object Localication K1
Explanation Map
M2
Pointing Came A
Object Localication AX2
Figure 3: Columns 1-3 and 4-6 correspond to the tasks "Detecting Adversarial Patch 1 (DAPI)" and "Detecting Adversarial Patch 2 (DAP2)". Also, each is explanation of "cucumber" and "tusker".
In this paper, "Pointing Game" and "Object Localization" used metrics in "Detecting Adver- sarial Patch (DAP1)" and "Detecting Adversarial Patch (DAP2)" tasks.
Pointing Game (PG) (Petsiuk et al., 2018): The Pointing Game (PG) is metric that measures whether the maximum point of the explanation map is inside the human-annotated object bounding box. As in columns 2 and 5 of Figure 3 if the point lies inside the target bounding box, it is counted #Hits
as hit. Pointing Game accuracy is calculated as #Hits+ Misess
Object Localization (OL) (Wang al. 2020): Object Localization (OL) is metric that measures how much the energy of the explanation map falls into the target bounding box instead of using only
Σ L.
maximum point. This metric can be formulated as Σ Tijeebox +2 L Liggsoon

---

Under review as a conference paper at ICLR 2022
However, in both DAP1 and DAP2, the ground truth bounding box of the target object is the patch,
bounding box. So, the ground truth bounding box in Sec 5.2.2 is the not the bounding box, and the ground truth bounding box in See 5.2.3 is the patch.
MOTIVATION
3.1
CHANNELS WITH OVERESTIMATED PIXELS (COP)
# COP
Gad-CAM
Gad-CAMes
Score-CAM
26
AMMrree-CAM
53
XGad-CAM
Input Image
(b) Target Only
(c) Target non- mon-Target (d) non-Target Only
Table 1: # COP
Figure 4: Target: "chihuahua",
non- nar-et: "tusker", "soccer ball", "broccoli"
In Figure 4 (b)-(d) indicate which part of the input image the specific channels of the activation tensor focus on (Masking input image with upsampled channels.). Channel (b) is focusing only on the target part, channel (c) is focusing on both the target and non-target part and channel (d) is focusing only on the non-target part. We investigated the weights for channels (b)-(d), and was confirmed that Grad-CAM, Grad-CAM++ Score-CAM, Ablation-CA.M, and XGrad-CAM all gave high weights to channels (b) and (c) and low weights to channel (d). In other words, regardless of whether specific channel contains non-target part not, long as it contains target part, that channel tends to be given a high weight. However, since channel-wist weight-based methods assign one weight per channel, all pixels in the channel share single weight. As result, for channel like (c), pixels in the non-target part are also given a high weight. As shown in Appendix [A] these channels are commonly found in the activation tensor. Since the explanation map of previous methods is generated by a linear combination of channels, if the number of channels with overes- timated pixels(COP) increases, the overestimated pixels also appear in the explanation map. These overestimated pixels suppress other pixels, so the explanation map may highlight the wrong places or only part of the target. [ablee[shows the average number of channels focusing on both the target and non-target part among the channels with the top 100 weights((nception V3 has 2048 channels). (If the channel contains more than 80% of the patch, the channel considered to focusing on that patch.) The values in Table I are averaged for 500 randomly generated AE2s. Among the five methods, Score-CAM best distinguishes the channel focusing only on the target part and the channel focusing on both the target and non-target part. However, this is also not satisfactory value.
3.2 DEPENDENCY ON ACTIVATION TENSOR (DOA)
Reason for removing the dependency activation tensor: To check the dependence between the explanation map and the activation tensor, we compare Pixab-CAM, which will be covered in Sec with the previous methods for the "Detecting Adversarial Patch (DAP1)" task presented in 2 In addition, in order to check where the activation values are intensively distributed in the activation tensor, we defined Activation-CAM, which is expressed Σ, 1 At, Since the channel- wise weights are fixed at 1, Activation- CAM can indicate where the activation tensor focuses.
If there is only an adversarial patch in the image. like the first row Figure 5 all methods detect this patch well. However, if features are evenly distributed in the image, such as when natural image and an adversarial patch are together (the result of Activation-CAA Figure (5), the channel-wise weight-based methods cannot capture the patch properly even if the same patch is in the same location. Since the explanation map generated from a linear weighted combination of channels in the activation tensor, it is difficult properly focus target if the target located where the activation tensor is sparsely distributed (= the number of channels capturing the target is small), such the result of Artivattio-CAM in the second row of the Figure 5 (In this case, the target is a patch.). This situation is common in the real world since channels in the activation tensor are not necessarily related to the target class(Zhane et al. 202T). On the other hand,
in the second row of
4

---

Under review as a conference paper at ICLR 2022
Input Image
Activation-AAm
Grad-CAM
Crad-CAM++
Scere-CAM
Ablation-Caa
XGrad-CAM
Pixab-CAM CAM
Figure 5: Both the first row and the second row are explanations of "keypad".
Pixab-CAM can handle any situation flexibly regardless of how the activation tensor is distributed on the image because the activation tensor does not participate in the process of generating the explanation map (sce 4ec Another finding is that the explanation maps generated by channel- wise weight-based methods except Grad-CAM and Ablation CAM do not deviate much from the result of Activation-CAM, Score-CAM is very similar to the results Activation-CAM From this, can be inferred that the Figure 5 produced the
shown in Figures 6 and 8 In particular, the explanation map of
reason that all the channel-wise weight-based methods in the first correct explanation map is that the features are distributed only in the adversarial patch, as shown in the result of Activation-CAM. In other words, since there were no features that prevented the detection of the target in the first place, all of them were able to provide proper explanation.
How measure dependence with activation tensor: To quantitatively check how dependent the explanation maps of CAM-based methods are on the activation tensor, we measured the cosine distance between Activation-CAn ImageNet, this cosine distance may not give reliable results the activation tensor may be properly distributed on the target. As the first row of Figure 5 if the activation tensor is distributed only on the target, it is impossible distinguish correct explanation map from explanation map that depends on the activation tensor. To solve this, we use "Adversarial Example (AE2)" introduced in Sec 2.2 We calculate the cosine distance after making the activation tensor evenly distributed over the four patches, such as the result of Activation-Ca. CAM Figure [ This approach ensures that the activation tensor is not distributed only the target, the dependency more accurately (Generatin an explanation map similar to Activation-CAM (= Low cosine distance) means is not capturing the target properly.) We measured the cosine distances for 500 AE2s.
and CAM-based methods. However, for natural images like
be computed
Input Image
Activation-CAM
Grad-CAM
Grad-CAM++
Score-CAM
Ablation-CAM XGrad-CAM
Pixab-CAM
Figure 6: 1st row: "chihuahua", 2nd row: "tusker", 3rd row: "soccer ball", 4th row: "broccoli",
The result for "Dependency on Activation Tensor (DOA)" is in Table Grad-CAM++. CcAr and XGrad-CAM have very low DOA values. In particular, Score-CAN has value close to O. In

---

Under review as a conference paper at ICLR 2022
other words, the explanation map they generate is highly dependent the activation tensor. We also observed that the less the activation tensor is distributed in the target, the more they depend on the activation tensor. This result implies that in the case of very weakly activated feature the image, the explanation maps these methods may not be able to capture that part even though is target. The explanation maps of Grad-CAM and Ablation- tensor, but they focus on the wrong place or only part of the target, as shown in Figures 6 and
do not depend the activation
PROPOSED METHODOLOGY
ablation bowes
plarmigan
C
C
DOKY
CC
N
N
ptarmigan
V
V
V
Figure 7: An overview of Pixab-CAM
Pixab-CAM has three main features. The first that doesn't use gradients to avoid problems with gradient saturation that negatively affect visualization quality (Wang et al. 2020 Ramaswamy al. 2020). Second, we define pixel-wise weight rather than channel-wise weight to solve the problem that pixels within specific channel are tied to a single scalar value. (First motivation in SecB.I) Finally, in order to completely remove the dependency on the activation tensor, we do not include the activation tensor in generating the explanation map. (Second motivation in Sec 3.2)
As in Equ 7 we define the weight by using the idea of Ablation-CAM in terms of pixels. Although Ablation-CAM uses only the channel contribution from the deletion(=abllation) perspective when deriving the channel PeeffiiientRamaswaaal
et al. 2020), we define pixel coefficient using both the pixel contribution from the deletion(fisst term) and preservation(secom term) perspectives. If specific pixel is associated with target class, the class score will decrease when it is removed(firtt term) and increase when appears(second term). These two terms are complementary to each other because they reflect the contribution of pixels from different perspectives. (It is a strict weight assignment method because a high coefficient can be obtained only by showing high contribution from both perspectives.) As shown in Figure 7 Eqn can be expressed as a tensor composed of x charnel _size, moving
Os(first term) and a tensor composed of sssecond term), each of size on the activation tensor in sliding window manner. We'll call this tensor ablation box, and use four ablation boxes-1 x 1, x 2, 2 x 2 AAn evaluation each design choice and why these four ablation boxes were used are covered in Appendix B The reason for using multiple scale boxes is to detect objects of various sizes and shapes as in Faster R-CNN et al., (201)). For reference, used, the prediction process only 1/4 of Ablation CAM.. Also, regardless of
if all boxes the size of the box, stride size is 1, and padding is used all except the 1 1 box to reflect the pixel contribution fairly. Then, we average over all boxes of each perspective. Finally, the explanation map is obtained after applying ReLU and the normalization function as in Eqn What should be noted here is that the activation tensor is not used in the process of generating the explanation map. yo
Y
Y
Y
pig (-
Yc
Ye
L - (ReLU (PE)))
(8)
where y is the score of class e when the pixel at spatial location (i.j) has been replaced by zero, conversely, Ve is the score of class e when the pixel at spatial location (i.j) is the only remaining pixel and ye is the score of class when tensor with all values of O is input the model.

---

Under review as a conference paper at ICLR 2022
5 EXPERIMENTS
In this section, we mainly compare Pixab-CAM with state-of-the-ar CAM-based methods. Quali- tative (see Sec(5.1) and quantitative evaluations (see 5.2 performed to evaluate the quality of the explanation map. addition, we use Inception V3 network(Szegele et al. 2016) pretrained on the ImageNet provided by TensorFlow black box model.
5.1
QUALITATIVE EVALUATION (MORE EXAMPLES ARE PROVIDED IN APPENDIX
Inpat Image ActivationCCAM
Grad-CAM
Gra-CAMM++
Score-CAM
Ablation-CAM
XGrad-CAM
Pixab-CAM
Figure 8: Explanation Map Comparison. (Rows 1-2: Natural Image,
Rows 3-4: AEI)
Input Image
Guided-BI
Grad-CAM
Grad-CAM++
Score-CAM
Ablation XGrad-CAM
Pixab-CAM
Figure 9: Guided CAM-based Explanation Comparison (Springenberg et al. 2014)
As shown in Figures and 10 we qualitatively compare the explanation maps, guided CAM- based explanations, and weekly supervised localization results generated by Activation-CAMM five state-of-the-ar CAM-based methods, and Pixab-CAM. Compared to previous CAM-based methods, Pixab-CAM produces higher quality explanation maps in all examples, whether or the activation tensor is focusing on the target. On the other hand, the explanation maps of previous CAM-based and Ablation-CAM tend to
methods tend not to focus properly on the target. First, generate insufficient explanation maps, and even Ablation-CAN captures places that are not related to the target class at all. Second, if part not related to the target class is emphasized in the result of Activation-CAM, that part is also highlighted in the explanation maps of Grad-CAM++, Score- CAM, and XGrad-CAM, which means that they are greatly affected by the activation tensor.
5.2 QUANTITATIVE EVALUATION
We scraped 2,000 images with two objects with different labels (MOD) to evaluate the performance of the explanation map in situations where fine- anee-rrined weight adjustment is required. MOD images consist of objects with different labels but similar textures, such as cats and dogs,
7

---

Under review as a conference paper at ICLR 2022
Inpus Image
Grid-CAM
Gmd-CAMM++
Score-CAM
Ahlation-CAr
XGrad-CAM
Pixah-CAM
Figure 10:
Weakly Supervised Localization Comparison (Qquab et al. 2015
lions and tigers, trucks and cars, the explanation map cannot properly capture the target unless the activation tensor is precisely adjusted by weight. In addition, ILSVRC 2012 validation set (IMGN) ( et al.] (013)). PASCAL VOC 2007 test set (vvrrnngeng et al. 2010), and COCO 2014 validation set (COCO)(Lin et al. 2014) were used for evaluation.
5.2.
FAITHFULNESS EVALUATION
Table 3: AD, IIC, and Compactness
Table 2: DOA
AD (9)
(9)
Compuctness
Name
DOA
Name
IMGN
COCO
COCO
VOC
MOD
0.39]
Grad CAM+++
0,028
Grad-CA
34.8
24.5
31.3
33.1
37.6
13.7
11.3
Scor-CAM
B:014
Grad CAMe+
22.4
22.8
21.3
33.3
31.0
35.2
19.8
16.9
Ablation-CAM
0.223
Scare-CAM
33.2
22.8
23.6
22.8
22.2
32.5
30.8
35.0
19.4
21.1
21.7
19.4
CAM
0.031
Abimion-CAM
32.7
21.2
22.4
20.1
34.5
33.3
32.9
37.6
16.3
17.9
18.3
14.5
Pixab-CAM
0.402
XGrad CAM
33.7
22.1
23.2
20.3
32.7
33.0
36.8
17.8
18.7
19.4
16.2
Paub-CAM
30.2
19.4
19.1
18.9
36.2
35.9
39.7
15.4
16.1
12.6
AD, IIC, and Compactness: Average Drop (ADtChaatopa et al. 2018) is metric that measures how much the explanation image, which is generated by point-wise wise multiplication of the upsampled explanation maps with the original image, maintains the part that plays essential roles in correct decision-man making.. Increase in Confidence (Chatttoopayy et al.) 2018) measures whether the part that confuses correct decisions in the image has been sufficiently removed. Compactness is the metric proposed for the first time in this paper to measure how large the area the explanation map occupies an image. An ideal explanation map should be of a suitable size cover the target com- pactly. Compactness is expressed as NV EE E. I where T.i is the explanation map value at spatial location (a, b) for the i-th input X; and N is the number of evaluation images. Results for 1,000 images randomly sampled for each dataset are in Table From the results of IIC and AD, except for IIC in IMGN, Pixab-CAM outperforms other CAM- methods by large scale on all metrics. This result reveals that Pixab-CAM can successfully leave only the essential parts for the correct decision while sufficiently removing the parts that interfere with decision mak- ing. In addition, Pixab-CAM has the second smallest Compactness value after Grad-CAM. means that Grad-CAM generates an explanation map with size less than necessary, and other previous CAM-based methods generate an explanation map with excessive size. Meanwhile, Pixab-CAM generates an explanation map of appropriate size enough cover the target compaetly.
The deletion metric(Petsiukk et al., 2018) measures the decrease in the probability of the predicted class more and more important pixels are removed. The insertion metriic et al., 2018), on the other hand, measures the increase in probability as more and more pixels are introduced. Table 4 reports the average result of 1,000 images randomly selected from ImageNet. Pixab-CAM exhibited the highest insertion AUC and lowest deletion AUC. This experiment shows that Pixab-CAM successfully assigned coefficients in order of importance.
Deletion and Insertion:
5.2.2 ОВJЕCT LOCALIZATION EVALUATION
In this section, we compare the localization ability of the generated explanation maps. The "Object Localization (OL)" metric was discussed Sec 2.3 and therefore omitted from this section. The values in Table 5 are averaged for 1,000 images randomly selected from ImageNet. Pixab-CAM has the highest proportion compared to other CAM- based methods. This means that the explanation maps generated by Pixab-CAM compactly focus the essential parts of the image.
8

---

Under review as a conference paper at ICLR 2022
5.2.3 "DETECTING ADVERSARIAL PATCH 1" & "DETECTING ADVERSARIAL PATCH 2"
We also evaluate the performance of explanation maps for tasks "Detecting Adversarial Patch (DAPI)" and "Detecting Adversarial Patch 2 (DAP2)", which are proposed for the first time in this paper (see Section 2). The number of patches in the image is randomly selected from 1-9, and the width and height of the patches are also randomly selected from 15%-20% of the input image size. DAP1 and DAP2 were tested on 500 AEIs and 500 AE2s, respectively, and the results are in 6 We observe that Pixab-CAM outperforms previous CAM-based methods in both DAPI and DAP2.
Table 6: DAPI & DAP2
Table 4: Deletion & Insertion
Table Object Localization
DAPI
DAP2
Name
Deletion
Insertion
Name
Proportion (%)
Name
PG
OL
PG
OL
Grad-CAM
0.090
0.433
Grad-CAM
46,39
Grud-CAM++
0.094
0.428
Gra+-CAM++
48.17
Grad. CAM
46.5
42.7
45.3
43.1
Score: CAM
0.101
0.425
CAMe-CAM
50.77
40.2
Ablation-CAN CAM
0.088
0.457
Ablation-CAM
51.32
Score-CAM
43.2
40.2
42.9
39.5
XGrad-CAM
0.434
XGrad-CAM
47.19
Ablation-CAM
46.8
43.1
46.7
4499
Pixab-CAM
0.082
0.469
Pixab-CAM
53,22
XGmd-CAM
44.0
43.1
40.6
Pixab-CAM
55.3
50.1
54.9
51.3
5.3 SANITY CHECK (ADEBAYO ET AL. 2018
Model parameter randomization test was proposed compare the explanation map on trained model to the explanation map on a randomly initialized, untrained network of the same architecture. As in Figure Fixab-CAM is sensitive to model parameters and can judge the quality of the model.
Input Image
Logit
Mixed_7b
Mixed_6c
Mixed_6a
Mixed_5c
Figure 11: Cascading randomization from top to bottom layers on Inception v3 trained on ImageNet
5.4 APPLICATIONS
To prove the applicability of Pixab-CAM in downstream tasks, we applied it to the Brain Tumor Detection task. Since the Brain Tumor dataset is small (N: 253), we used "Transfer Learning" (Pan & Zisserman, 2014) pretrained on the ImageNet
Yang, 2009). downloaded VGG- VGG-16(Simonyan dataset, then trained only the final fully connected layer while keeping the weights of all ConvNets fixed. Figure 12 shows the explanation map on a trained VGG- 16 for brain tumor images. Pixab- CAM detects tumor better than other CAM-based methods. This experiment demonstrates that Pixab-CAM can also be applied to fine-tuned models using small datasets.
Input Image
Grad-CAM
Grad +ru++CAM++
Score-CAM
Ablation-CAa CAM XGrad-CAM Pixa-CAA
Figure 12: Explanation Map of Brain Tumor images
CONCLUSION
We proposed Pixab-CAM, a novel CAM-based approach with three characteristics. First, we use pixel-wise weights rather than channel-wise weights that the importance of the pixels is not dis- torted. Second, we exclude the activation tensor to create an explanation map independent of the activation tensor. Third, to avoid problems with gradient-based CAMs, gradients not used when obtaining pixel-wise weights. In addition, we propose new evaluation metric using adversarial at- tack and demonstrate the superiority of Pixab-CAM through qualitative and quantitative evaluation.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. arXiv preprint arXi::1810.03222 2018.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Rober Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer- layer-wise relevance propagation. PloS one, 10(7)ce0130400, 2015.
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad- cam++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 839-847. IEEE, 2018.
François Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1251-1258, 2017.
Piotr Dabkowski and Yarin Gal. Real time image saliency for black box classifiers. arXiv preprint 2017.
Mark Everingham, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2): 303-338, 2010.
Ruth C Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful perturba- tion. In Proceedings of the IEEE International Conference Computer Vision, pp. 3429-3433 2017.
Ruigang Fu, Qingyong Hu, Xiaohu Dong, Yulan Guo, Yinghui Gao, and Biao Li. Axiom- based grad-cam: Towards accurate visualization and explanation of cnns. arXiv preprint arX1v:2008.02312, 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. 2014.
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: arXiv: 1704004611 2017.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700 2017.
Hyungsik Jung and Youngrock Oh. Lift-cam: Towards better explanations for class activation map- ping. arXiv preprint arXiv:2102.05220 2021.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-233) 1998.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr
Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Scott Lundberg and Su-In Lee. A unified approach to interpreting model predictions. arXiv preprint arXiv:1705.07877, 2017.
Maxime Oquab, Léon Bottou, Ivan Laptev, and Josef Sivic. object localization for free?-weakly- supervised learning with convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 685-694, 2015.
Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on knowledge and data engineering. 12410)1345-1359, 2009.
Vitali Petsiuk, Abir Das. and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv: 08011880007221 2018.
10

---

Under review as a conference paper at ICLR 2022
Harish Guruprasad Ramaswamy et al. Ablation-cam Visual explanations for deep convolutional network via gradient-fre localization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 983-991, 2020.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-enm: Towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01997 2015.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagener large scale visual recognition challenge. International journal of computer vision, 11(())211-252) 2015.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- gradient-based based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International Conference on Machine Learning, pp. 3145- 3153. PMLR, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint 1rXiv:1409.1555 1456.1556,1,, 2014.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv: /rtxiv:706...e 1706.03255 2017.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint 1rXiv:1412.68000 62066000 2014.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1-9, 2015.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-28222 2016.
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-cam: Score-weighted visual explanations for convolutional neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pallern Recognition Work- shops, pp. 24-25, 2020.
Qinglong Zhang, Lu Rao, and Yubin Yang. A novel visual interpretability for deep neural networks by optimizing activation maps with perturbation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 3377-3384, 2021.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2921-2929, 2016.
11

---

Under review as a conference paper at ICLR 2022
A CHANNELS WITH OVERESTIMATED PIXELS
As explained in Sec overestimated pixel occurs because pixels within a particular channel share single value. We visualize channels with overestimated pixels(COP) through Adversarial Example 1 (AEI), Adversarial Example 2 (AE2), and natural image. (Descriptions of AEI and AE2 are in 2. and S.2 respectively.) For each of Grad-CAM, Crad-CAM+++, Score-CAM, Ablation- CAM, and XGrad-CAM, we find the top 20 channels by weight among the 2048 channels in the activation Iensor(nnception V3 has 2048 channels(Szegedy et al. (016)), and then select channels focusing on both the target and non-target part. The visualization results are in Figures 13 14 and 15 The target in Figures 13 14 and are the patch, the patch in the upper left, and the "pug" and "labrador retriever". respectively. (The first row, second row, third row, fourth row, and fifth row are the results of Grad-CAM, Grad-CAM++, Score-CAM, Ablation-CAM and XGrad-CAM respectively, and each column represents channel.) This result shows that more channels with overestimated pixels(COP) than expected are in the activation tensor.
B DESIGN CHOICE
We will find the optimal combination of ablation boxes with the evaluation tasks (DAPI and DAP2)
presented in 2.c Since is difficult evaluate all possible ablation box combinations(For Inception (3eezgyy et al. 2016), since the size of the activation tensor is 8 x 8, the possible ablation box sizes are 1 1,
x 2,2 2 1,2 x 2,.... 8 8), first reduce the design space through performance evaluation for individual ablation boxes. Tables 7 g and are the results when only the first term of Eanniis used, when only the second term is used, and when both the first and second terms are used, respectively. Two characteristics can be found in these tables. First, if any of the width or height of the ablation box is 3, the performance drops significantly. (We also experimented with cases where the ablation box had a width or height of 4 or more, but we did not include it in the table because performance continued to deteriorate.) Second, when the width and height of the ablation box are both less than equal to 2, the performance of using both the first and second terms of Eqn 7 is always better than the performance of using only the first or second term. means that the first term and second term complementary to each other. Based on these results, only ablation boxes of sizes x 1, x 2, 2 x 1, and 2 x 2 are used in the following experiment to find the optimal combination.
Table I0 shows the evaluation results for all possible combinations of the four ablation boxes se- lected in the previous experiment. (We evaluated all 15 combinations, but combinations that did include an ablation box of size 1 1 were not recorded in Table because their performances were all low rank.) This result shows that the performance gets better the number of ablation boxes increases. For these reasons, we use all four ablation boxes.
C
MORE QUALITATIVE EVALUATION
Figures 16
77
18
19 and 20 contain more examples of the qualitative evaluation covered in
S.c Figure Comparison (AET), Figure 18 Explanation Map Comparison (AE2), Figure 19 Guided CAM-
Exp Explanation Map Comparison (Natural Image), Figure 17 Explanation Map
based Explanation Comparison, Figure 20 Weakly Supervised Localization Comparison)
12

---

Under review as a conference paper at ICLR 2022
Table 7: Comparison of each ablation box I ("Delction" Perspective)
Combinations of Ablation (Superscript
the Equ
DAPI
DAP2
(1x 1) (1 2) (2 1)
(2 21
(1 x 3)
(3 1)
(2 x 3)
x 2)
13 x3
PG
O1.
52.4
43.9
519
438
52.0
44.1
528
44.1
44.4
50.1
41.4
499
40.9
69.
40.9
49.7
41.2
Table 8: Comparison of each ablation box II ("Preservation" Perspective)
Combinations of Ablation Boxes (Suporseript + means the second torm of Egn 7
DAPI
DAP2
x1 x (2 1)+ (2 2) (1
3)+
1y+
ay+
x 2)
x3x
OL
PG
$2.9
44.0
52.2
44.1
$1.7
44.3
51.9
43.9
52.5
43.8
51.3
44.3
48.9
41.4
50.1
41.
49.5
40.3
49.7
41.0
48.5
41.
47.8
41.
48.5
40.9
48.3
414
Table Comparison of each ablation box III ("Deletion Preservation" Perspective)
Comhinations of Ablation Boxes (Superscript T means the first and second terms of Eqp
DAPI
DAP2
(1 1)
(1x 2
x2x 1) x 2
(1 3) (3 1y
(2 x 3)
(3 x 2)
x 3)
PG
OL
PG
OL
538
47.1
53.3
46.6
52.5
46.5
52.3
45.9
53.5
48.4
41.1
5011
407
50.2
41.3
49.1
40.5
47.
418
48.1
40.6
Table 10: Find the optimal combination of ablation boxes
Combinations Ablation
(1x 1) (1 2)
(2 FF (2 x 2)
PG
Ot.
53.8
47.5
52.9
47.7
54.1
47.5
53.3
48.0
54.4
48.
55.1
J9,4
34,5
31,0
54.9
49.9
54,2
50.6
13

---

Under review as a conference paper at ICLR 2022
Input Image
C1
C4
C6
Grad-CAM
Grad-CAM-+
Scare-CAM
Ablation-CAM
XGrad-CMM
Input Imuge
Grad-CAM
Grad-CAM++
ScorsCAM
AblationCCAM
XGmié-CM
Figure 13: Adversarial Example
14

---

Under review as a conference paper at ICLR 2022
Input Image
C1
C2
C4
C5
C6
Grad-CAM
Grad-CAM-
1
20
244
Scare-CAM
18
4
Ablation-CAM
R8
XGaddCCM
Input Imuaee
C6
Grad-CAM
Grad-CAM++
93
Scors-CAM
46
Ablation-CAM
4
2
XGmi--CM
Figure 14: Adversarial Example
15

---

Under review as a conference paper at ICLR 2022
Input Image
C1
C4
C6
Grad-CAM
Grad-CAM-+
Scare-CAM
Ablation-CAM
XGrad-CAM
Input Imuge
C6
Grad-CAM
Grad-CAM++
Scor--CM
Ablation-CAM
XGmié-CM
Figure 15: Natural Image
16

---

Under review as a conference paper at ICLR 2022
Input Image
Activation CAM
Grad CAM
Grad CAM++
Score CAM
Ablation-CAM
XGrai CAM
Pixab CAM
Figure 16: Explanation Map Comparison (Natural Image)
17

---

Under review as a conference paper at ICLR 2022
Input Image
Activation CAM
Grad CAM
Grad CAM++
Score CAM
Ablation-CAM
XGrai CAM
Pixab CAM
No
SANTo Figuns
Fighiss
FIGINS
Fighticc
FIGuNC
Figure 17: Explanation Map Comparison (AE1)
18

---

Under review as a conference paper at ICLR 2022
Inpul Image
Activation-AAN
Grad CAM
Grad CAM++
Score CAM
Ablation-CAM
XGrai CAM
Pixab CAM
Figure 18: Explanation Map Comparison (AE2)
19

---

Under review as a conference paper at ICLR 2022
Input Image
Guided BP
Grad CAM
Grad CAM++
Score CAM
Ablation-CAM
XGral-CAM
Pixab CAM
Figure 19: Guided CAMM-assed Explanation Comparison
20

---

Under review as a conference paper at ICLR 2022
Input Image
Grad-CAM
Grad CAM++
Score CAM
Ablation-Cue CAM
XGrad-CAM
Pixab CAM
Figure 20: Weakly Supervised Localization Comparison
21


Reviews:
[{"review_id": "iW0mMxoGDv1", "reviewer": "Reviewer_8WRU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In this paper, the author proposes a new variant of class activation map (CAM), i.e., Pixab-CAM with a novel evaluation metric to show the effectiveness of the proposed Pixab-CAM.\n\nIn general, the Pixab-CAM provides a new view of CAM-related methods, especially on pixel-level attention. However, I have the following serious concerns:\n\n1. The writing should be improved. I see some serious grammar and punctuation mistakes. The author should improve their writing skills.\n\n2. The pixel-level idea is not a brand new idea in the computer vision community. It has been used in the field of image recognition [1], visual explanation [2], and many other areas including adversarial attack. The author should at least cite this paper and discuss the differences between their modules and the proposed Pixab-CAM. Especially, [2] proposed an attention branch, which can also perform visual explanation with spatial attention and boost the performance. The author should definitely discuss with that paper.\n\n3. More recent results and methods should be included in this paper and fewer figures should be included. First, compared to existing works like Grad-CAM, Grad-CAM++ and ScoreCAM, the proposed Pixab-CAM only lists some regular metrics in Table 2 and Table 3. I would suggest the author list all metrics in previous papers in the appendix. Also, there are some recent CAM related methods like [3], I think the author should also include them for comparison, not just for visualization methods.\n\nBased on these three drawbacks, I would like to recommend the author revise this paper for the next conference.\n\nReferences:\n\n1. Woo S, Park J, Lee J Y, et al. CBAM: Convolutional block attention module[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 3-19.\n2. Fukui H, Hirakawa T, Yamashita T, et al. Attention branch network: Learning of attention mechanism for visual explanation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 10705-10714.\n3. Bae W, Noh J, Kim G. Rethinking class activation mapping for weakly supervised object localization[C]//European Conference on Computer Vision. Springer, Cham, 2020: 618-634.", "summary_of_the_paper": "In this paper, the author proposes a new variant of class activation map (CAM), which is a gradient-free and pixel-wise CAM variant and named Pixab-CAM. The author wants to claim that the proposed Pixab-CAM is superior to previous CAM-based methods on normal metrics, and propose a novel evaluation metric using an adversarial attack.", "summary_of_the_review": "In this paper, the author proposes a new variant of class activation map (CAM), i.e., Pixab-CAM with a novel evaluation metric to show the effectiveness of the proposed Pixab-CAM.\n\nHowever, several drawbacks including writing, novelty and fair comparison suggests me to reject this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "erTlIqe6sgC", "reviewer": "Reviewer_u8GW", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n+ Easy to read and understand.\n+ The motivation is clear and well-illustrated.\n+ It is interesting to connect adversarial attack and interpretability.\n\nWeaknesses/Concerns:\n- The adversarial example (AE1) is kind of confusing. In Figure 1, (b) is an adversarial patch that is classified to a targeted class, and then the author directly put this patch into the input image (c). It is misleading to call (e) as an adversarial example, as there is no evidence showing that (e) is misclassified by the model (should it be?). Or the adversarial patch just works as disturbance?\n- What is Figure (4)? Why does the activation map look like this? Intuitively, should it be a coarse heatmap?\n- In Figure (8) rows3-4, why it is expected to highlight the adversarial patches? What is the label class / predicted class of the input image?\n- In sec 3.2, to avoid the activation tensor/map properly distributed on a single target, is it also practical to use images with two or more objects instead of adversarial patches? In this case, the activation tensor should also be randomly distributed.", "summary_of_the_paper": "This paper aims to solve two common limitations in CAM-based methods, and proposes a new type of CAM (Pixab-CAM) that utilize pixel-wise weights instead of channel-wise weight. Meanwhile, the author also proposes to use adversarial attack as a novel evaluation metric. This work demonstrates to be superior to previous CAMs on corresponding metrics.", "summary_of_the_review": "Overall, I vote for accepting, even with some concerns unsolved.\n\nThe pipeline of Pixab-CAM is simple and easy to adopt. This paper also provide comprehensive analysis of its motivation in section 3.2. The adversarial patch is novel and contributes to this filed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DEyHycrS5hy", "reviewer": "Reviewer_roRV", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n- The method is simple and intuitive.\n- Visualized examples help authors understand.\n- Code is provided, so it is likely to be reproducible.\n\n&nbsp;\n### Weaknesses\nOverall, the experiments were conducted only on a small number of images, so it is questionable whether they can be reliable.\n\n- **Quantitative results**: For all quantitative results, only 1,000 images were used for each dataset. This can be a very small weight depending on the entire dataset, and it can cause a significant selection bias. In other words, depending on how 1,000 images are chosen, the results can be different. The authors argue that the time complexity of the proposed method is not high, then why not use the entire dataset? In particular, this topic does not require a training process, so the experiment would not take much time. At least, it seems necessary to repeat the experiment multiple times and get the confidence interval of the performance.\n- **Sanity check & Applications**: Results for two sections only exist for one image. I know it is hard to add them to the body text, but I cannot find them in the supplementary file either.\n\n&nbsp;\n### Other Comments\nIn addition to Weaknesses, there are concerns about the following items.\n\n- **Readability**: The location of tables, formulas, and figures is not suitable for readability. For example, it seems better to arrange Tables 2, 4, and Eq.(7) as close as possible to the position described in the text. And, although it is the contents of the supplementary file, Tables 7~9 are also inefficiently organized. If \"Combination of Ablation boxes\" are grouped into one column and placed each item in each row, it seems to be better in terms of space utilization and readability.\n- **Qualitative results**: When showing an example like Figure 8, please indicate which class the CAM is for. Also, if there are multi-class objects in one image, it would be better to show all CAMs for each class.\n- **Time complexity**: There is only mention that the prediction process is 1/4 of Ablation-CAM, but there is no detailed comparison of time complexity. What is the time complexity (FPS) for each CAM?\n\n&nbsp;", "summary_of_the_paper": "This paper proposes a method for generating an explanation map (Pixab-CAM) showing which part the CNN model for classification refers to in making a decision. Pixab-CAM utilizes how the classification results are changed by focusing or removing a specific region of the convolutional feature map. By using this, the authors claim that various problems (gradient saturation, single scalar value for a specific channel, dependency on the activation tensor) of existing CAM-based methods are solved. Experiments were conducted on various datasets and metrics, and through this, the effectiveness of the proposed method was verified.", "summary_of_the_review": "The proposed method has the advantage of being intuitive and simple, but it is suspicious whether the experiments to support its effectiveness are reliable. Therefore, my initial rating is 5, and I expect the authors to address my concerns through a rebuttal.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0uQweSYxzsF", "reviewer": "Reviewer_Kcad", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\nMany of the ideas in the paper sound interesting.\n\nWeaknesses:\n\nThe paper is not written or structured clearly. The authors make many claims that are not adequately justified. It would be very helpful if they could define with equations the objects they are referring to (for example, they never define the activation tensor). I am not even sure I understood what the method actually does because I am unsure what it means for a pixel to be \"the only remaining pixel at a location\" or that a zero tensor is input to the model. In general the paper feels incomplete. For example, the sanity check in Section 5.3 consists of a single example. \n\nRegarding the proposed tasks based on adversarial examples, I think it is intriguing, but I again do not think that they are adequately justified. The authors state: \"If a targeted adversarial attack is made on a specific patch in the zero image, it is self-evident that CNN sees that patch and determines that it is a target, so the performance can be evaluated by whether the explanation map correctly captures that part.\" I find this explanation very confusing. It would be great if the authors explain more clearly what the advantage is with respect to using images where we know the location of the relevant object.\n\nAdditional comments:\n\nThe proposed method is built upon the strategy of checking how the classifier degrades under input ablation. It combines the intuition of Ablation-CAM and Score-CAM and operates at the pixel level, which is an interesting idea. However, as mentioned in [1], such a strategy has an important drawback -- samples where a subset of the features are removed come from a different distribution, which violates one of the key assumptions in machine learning: the training and evaluation data come from the same distribution. The empirical result in [1] implies that strong performance degradation without re-training might be caused by a shift in distribution instead of removal of information. The authors should at least comment on these issues. In addition, I would suggest using benchmark metrics such as PxAP [2]. \n\n[1] Hooker, Sara, et al. “A Benchmark for Interpretability Methods in Deep Neural Networks.” Advances in Neural Information Processing Systems 32 (2019): 9737-9748.\n[2]Choe, Junsuk, et al. “Evaluating weakly supervised object localization methods right.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.", "summary_of_the_paper": "This paper proposes an ablation-based method to produce class activation maps, as well as two new evaluation procedures based on adversarial examples. ", "summary_of_the_review": "The paper contains some intriguing ideas, but I cannot recommend acceptance due to the lack of clarity and justifications.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work received borderline rates with slight preference to rejection. The main concerns range from writing, novelty to empirical evaluations. Given that no authors’ responses are submitted, we have decided to reject this work.
Decision: Reject