Pixab-CAM: Attend Pixel, not Channel

1 INTRODUCTION . As the performance of convolution neural networks ( CNNs ) converges to some extent ( Szegedy et al. , 2015 ; Simonyan & Zisserman , 2014 ; Huang et al. , 2017 ; Howard et al. , 2017 ; Chollet , 2017 ) , researchers have become interested in interpreting their predictions ( Springenberg et al. , 2014 ; Smilkov et al. , 2017 ; Fong & Vedaldi , 2017 ; Bach et al. , 2015 ; Lundberg & Lee , 2017 ; Ribeiro et al. , 2016 ; Petsiuk et al. , 2018 ; Shrikumar et al. , 2017 ) . One of them is class activation mapping ( CAM ) based approach ( Zhou et al. , 2016 ) , which generates an explanation map by a linear combination of channels in the activation tensor and corresponding weights . The explanation map Lcij for the target class c is calculated as Eqn.1 . Akij represents activation of pixel at spatial location ( i , j ) for k-th channel of the activation tensor A , and wck denotes the coefficient of the k-th channel for class c. Lcij = ∑ k wck ·Akij ( 1 ) The activation tensor A is fixed since it is determined by the given input image and pretrained model ( LeCun et al. , 1998 ) . Therefore , Lc depends on how wck is defined . Thus , different w c k have been proposed in many CAM-based methods to generate high-quality explanation maps . The previous CAM-based methods can be divided into two categories , one is gradient-based CAMs ( Selvaraju et al. , 2017 ; Chattopadhay et al. , 2018 ; Fu et al. , 2020 ) , and the other is gradient-free CAMs ( Wang et al. , 2020 ; Ramaswamy et al. , 2020 ) . Gradient-based CAMs and gradient-free CAMs can be distinguished by whether or not gradients are used in the process of deriving weights . 1.1 GRADIENT-BASED CAMS . Grad-CAM ( Selvaraju et al. , 2017 ) obtains the weight of k-th channel by averaging their gradients . However , it has a problem in that the weight is crushed due to the offset of positive and negative gradients . ( Z is the number of pixels in a channel and Y c is the classification score for class c ) wck = 1 Z ∑ i ∑ j ∂Y c ∂Akij ( 2 ) Grad-CAM++ ( Chattopadhay et al. , 2018 ) introduced higher-order derivatives αkcij in Grad-CAM equation to solve the problem that channels with fewer footprints fade away in the explanation map . wck = ∑ i ∑ j αkcij ·ReLU ( ∂Y c ∂Akij ) ( 3 ) XGrad-CAM ( Fu et al. , 2020 ) points out that previous CAM-based methods lack sufficient theoretical support and propose weight that satisfies two basic axioms : Sensitivity and Conservation . wck = ∑ i ∑ j ( Akij∑ i ∑ j A k ij ∂Y c ∂Akij ) ( 4 ) However , Gradient-based CAMs suffer from gradient saturation problems , leading to localization failures for relevant regions . ( Wang et al. , 2020 ; Ramaswamy et al. , 2020 ; Jung & Oh , 2021 ) 1.2 GRADIENT-FREE CAMS . Score-CAM ( Wang et al. , 2020 ) defines “ increase of confidence ” as the coefficient of the channel . For input X and baseline input Xb , the weight of the k-th channel for class c is defined as wck = f ( X ◦ s ( u ( Ak ) ) ) ) − f ( Xb ) ( 5 ) where u ( · ) indicates the upsampling operation into the input size , s ( · ) is a function that normalizes all elements to [ 0 , 1 ] and ◦ is the “ Hadamard product ” . Ablation-CAM ( Ramaswamy et al. , 2020 ) , which inspired our research , defines the weight of a specific channel as the fraction of drop in activation score of target class when that channel is removed . In the equation below , Y ck is the score of class c when the k-th channel has been replaced by zero . wck = Y c − Y ck Y c ( 6 ) Gradient-free CAMs remove the dependence of gradients , but they are time-comsuming because the prediction process must be repeated as many times as the number of channels to acquire the weights . Regardless of whether gradient is used or not , all CAM-based methods so far have two things in common . First , they all use channel-wise weights wck ( see Eqn.1 ) . However , since the channel-wise weights allow all pixels in a specific channel to share a single weight , it is an unfair weighting method from the pixel perspective . ( Although αkcij in Grad-CAM++ ( Eqn.3 ) modifies the gradients at the pixel level , but these gradients are eventually crushed into a single scalar value , so it doesn ’ t solve the problem of pixels being tied together . ) Furthermore , channel-wise weight-based methods tend to assign high weight if there are pixels that have a high contribution to the target class among the active pixels in the channel . Therefore , within a specific channel , other pixels that are active with the pixels related to the target class are also given a high weight . If channels containing these overestimated pixels are accumulated , overestimated pixels also occur in the explanation map . ( see Sec.3.1 ) Second , when generating the explanation map Lc , all CAM-based methods follow Eqn.1 , which contains the activation tensor A . Therefore , the explanation map has no choice but to depend on the activation tensor , and eventually may capture too much meaningless information since the channels in the activation tensor are not necessarily related to the target category ( Zhang et al. , 2021 ) . ( see Sec.3.2 ) 2 PROPOSED EVALUATION METRICS OF THE EXPLANATION MAP . Since it is hard to know where CNN looks and makes a specific prediction , the existing evaluation metrics of the explanation map have limitations . AD & IIC ( Chattopadhay et al. , 2018 ) and Deletion & Insertion ( Petsiuk et al. , 2018 ) are coarse proxy evaluations because removing pixels by zero-ing/graying is unnatural in real photos ( Dabkowski & Gal , 2017 ) . Also , in Localization Evaluation ( Wang et al. , 2020 ) , there may not be a correlation between localization ability and the quality of the explanation map since the human-annotated bounding box may not match where CNN is actually focusing . However , if we could know where CNN is looking , we could make a more accurate assessment . We use an adversarial attack for this . ( we use FGSM attack ( Goodfellow et al. , 2014 ) . ) If a targeted adversarial attack is made on a specific patch in the zero image , it is self-evident that CNN sees that patch and determines that it is a target , so the performance can be evaluated by whether the explanation map correctly captures that part . We propose 2 evaluation tasks ( DAP1 and DAP2 ) from this idea . Before introducing them , we will explain 2 adversarial examples ( AE1 and AE2 ) first . 2.1 ADVERSARIAL EXAMPLE 1 ( AE1 ) . It is simple to attach an adversarial patch classified as a given target class to an input image . First , we apply a targeted adversarial attack so that the patch at a specific location in the zero image is classified into the class we want . Since the attack is applied only to the patch , the rest of the area is still zero as in ( b ) . Then , the input image with only the patch area of zero ( ( d ) ) is added to the image ( b ) . The finally obtained adversarial example AE1 ( ( e ) ) is used for our evaluation task DAP1 . 2.2 ADVERSARIAL EXAMPLE 2 ( AE2 ) . For generating a second adversarial example AE2 , only adversarial patches are used without input images . This can also be easily created . After generating four adversarial patches in the same way as in Sec.2.1 ( b ) , combine them to make the final adversarial example AE2 . ( Each adversarial patch is attacked to become a different class . ) AE2 created in this way is used for evaluation task DAP2 . 2.3 DETECTING ADVERSARIAL PATCH . In this paper , “ Pointing Game ” and “ Object Localization ” are used as metrics in “ Detecting Adversarial Patch 1 ( DAP1 ) ” and “ Detecting Adversarial Patch 2 ( DAP2 ) ” tasks . Pointing Game ( PG ) ( Petsiuk et al. , 2018 ) : The Pointing Game ( PG ) is a metric that measures whether the maximum point of the explanation map is inside the human-annotated object bounding box . As in columns 2 and 5 of Figure 3 , if the point lies inside the target bounding box , it is counted as a hit . Pointing Game accuracy is calculated as # Hits # Hits+ # Misses . Object Localization ( OL ) ( Wang et al. , 2020 ) : Object Localization ( OL ) is a metric that measures how much the energy of the explanation map falls into the target bounding box instead of using only the maximum point . This metric can be formulated as ∑ Lcij∈bbox∑ Lcij∈bbox+ ∑ Lc ij /∈bbox . However , in both DAP1 and DAP2 , the ground truth bounding box of the target object is the patch , not the human-annotated bounding box . So , the ground truth bounding box in Sec.5.2.2 is the human-annotated bounding box , and the ground truth bounding box in Sec.5.2.3 is the patch . 3 MOTIVATION . 3.1 CHANNELS WITH OVERESTIMATED PIXELS ( COP ) . Figure 4 : Target : “ chihuahua ” , non-Target : “ tusker ” , “ soccer ball ” , “ broccoli ” In Figure 4 , ( b ) - ( d ) indicate which part of the input image the specific channels of the activation tensor focus on ( Masking input image with upsampled channels. ) . Channel ( b ) is focusing only on the target part , channel ( c ) is focusing on both the target and non-target part and channel ( d ) is focusing only on the non-target part . We investigated the weights for channels ( b ) - ( d ) , and it was confirmed that Grad-CAM , Grad-CAM++ , Score-CAM , Ablation-CAM , and XGrad-CAM all gave high weights to channels ( b ) and ( c ) and low weights to channel ( d ) . In other words , regardless of whether a specific channel contains a non-target part or not , as long as it contains a target part , that channel tends to be given a high weight . However , since channel-wise weight-based methods assign one weight per channel , all pixels in the channel share a single weight . As a result , for a channel like ( c ) , pixels in the non-target part are also given a high weight . As shown in Appendix A , these channels are commonly found in the activation tensor . Since the explanation map of previous methods is generated by a linear combination of channels , if the number of channels with overestimated pixels ( COP ) increases , the overestimated pixels also appear in the explanation map . These overestimated pixels suppress other pixels , so the explanation map may highlight the wrong places or only part of the target . Table 1 shows the average number of channels focusing on both the target and non-target part among the channels with the top 100 weights ( Inception V3 has 2048 channels ) . ( If the channel contains more than 80 % of the patch , the channel is considered to be focusing on that patch . ) The values in Table 1 are averaged for 500 randomly generated AE2s . Among the five methods , Score-CAM best distinguishes the channel focusing only on the target part and the channel focusing on both the target and non-target part . However , this is also not a satisfactory value .

Summary: This paper proposes a method for generating an explanation map (Pixab-CAM) showing which part the CNN model for classification refers to in making a decision. Pixab-CAM utilizes how the classification results are changed by focusing or removing a specific region of the convolutional feature map. By using this, the authors claim that various problems (gradient saturation, single scalar value for a specific channel, dependency on the activation tensor) of existing CAM-based methods are solved. Experiments were conducted on various datasets and metrics, and through this, the effectiveness of the proposed method was verified.