Evaluating Weakly Supervised Object Localization Methods Right? A Study on Heatmap-based XAI and Neural Backed Decision Tree
Authors: Erico Tjoa
Cuntai Guan
Venue: Submitted to ICLR 2023 (2023)
Keywords: object localization
computer vision
deep learning
deep neural network

Abstract: Choe et al have investigated several aspects of Weakly Supervised Object Localization (WSOL) with only image label. They addressed the ill-posed nature of the problem and showed that WSOL has not significantly improved beyond the baseline method class activation mapping (CAM). We report the results of similar experiments on ResNet50 with some crucial differences: (1) we perform WSOL using heatmap-based eXplanaible AI (XAI) methods (2) our model is not class agnostic since we are interested in the XAI aspect as well. Under similar protocol, we find that XAI methods perform WSOL with very sub-standard MaxBoxAcc scores. The experiment is then repeated for the same model trained with Neural Backed Decision Tree (NBDT) and we found that vanilla CAM yields significantly better WSOL performance after NBDT training.

Under review as a conference paper at ICLR 2023
EVALUATING WEAKLY SUPERVISED OBJECT LOCAL- IZATION METHODS RIGHT? A STUDY ON HEATMAP- BASED XAI AND NEURAL BACKED DECISION TREE
Anonymous authors Paper under double-blind review
ABSTRACT
Choe et al have investigated several aspects of Weakly Supervised Object Lo- calization (WSOL) with only image label. They addressed the ill-posed nature of the problem and showed that WSOL has not significantly improved beyond the baseline method class activation mapping (CAM). We report the results of similar experiments on ResNet50 with some crucial differences: (1) we perform WSOL using heatmap- based eXplanaible AI (XAI) methods (2) we consider the XAI aspect of WSOL in which localization is used as the explanation of a class prediction task. Under similar protocol, we find that XAI methods perform WSOI, with very sub-standard MaxBoxAce scores. The experiment is then repeated for the same model trained with Neural Backed Decision Tree (NBDT) and we found that vanilla CAM yields significantly better WSOL performance after NBDT training.
INTRODUCTION
Weakly-supervised object localization (WSOL) aims to use only image-level labels (class labels) to perform localization. Compared to methods that require full annotations, WSOL can be much more resource efficient; it has therefore been widely studied (Choe & Shim, 2019; Singh & Lee, 2017; Zhang al., 201 Sa;b; Zhou et al., 2016; Guo et al., 2021: Wei et al., 2021; Babar & Das, 2021; Gao et al., 2021; Xie et al., 2021).
Class Activation Mapping (CAM) (Zhou et al., 2016) is heatmap-based explainable artificial intelligence (XAI) method that enables Convolutional Neural Network (CNN) to perform WSOL. Other heatmap-based XAI methods have been designed to compute relevance/atriibttion maps, some of which have been treated as localization maps after some processing e.g. Saliency (Simonyan et al., 2014) has been used for WSOL using only gradient (obtained from backpropagation) and minimal post-processing In this paper, besides Saliency, will also investigate the WSOL capability of several heatmap- Guided Backpropagation (GBP) (Springenberg et al., 2015) and DeepLift (Shrikumar et al., 2017). Admittedly, there are many other methods that are not included in this paper e.g.. Layerwise Relevance Propagation (also its derivatives (Bach al., 2015; Montavon et al., 2017; Kohlbrenner al., 2020)) and modifications of CAM (Muhammad & Yeasin, 2020; Wang et al., 2020; Jalwana et al., 2021: Kindermans et al., 2018).
XAI methods: GradCAM (Selvaraju et al., 2016) (generalization of CAM),
Main objective of this paper: measure the WSOL capability of existing heatmap-based XAI method applied on ResNet50 and improve them. Fig. shows how existing XAI methods can be very unsuitable for WSOL (e.g. high granular heatmaps and uneven edge detection). This paper shows that it possible modify the aforementioned methods and improve their localization ability beyond baseline CAM. Imporiant clarifications:
is not our intention to invent yet another XAI method. Instead, we add intermediate steps (based on CAM-like concept) on existing techniques to improve WSOL performance.
2. We do not claim attain the best localization. We are in fact testing the metric MaxBoxAcc presented in CVPR 2020 (Choe et al., 2020). that paper, a dedicated training process is performed to optimize the said metrie; in their github, this training is simply called the WSOL training. While their training improved WSOL, as a trade-off, their classification performance has degraded, and we quote them "There are in general great fluctuations in the classification results

---

Under review as a conference paper at ICLR 2023
(A)
Saliency/Gradient
CAM
Guilded Backpropagation
CAM
04
42
(B)
image original
633
conv5 conve. comv2.1 convi.
Image original
Ic3
comv5_3 convt_ 1 conv2_ I conv1
Gu
LRP a281
RectGrad
PatternAttr.
DTD
DIspliFT Rosc.
LRP a180
Gradient
Figure I: Localization capabilities of XAI methods differ, some may not suitable for WSOL (A) When naively normalized to [0. 1]. saliency method shows sparse and granular heatmaps compared to CAM, while guided BP shows an attribution map with unclear localization. (B) Heatmaps from various methods during sanity checks. We can infer that different localization information may (but not necessarily) exist within different layers. Figure (B) is used with permission obtained from the authors of (Sixt al., 2020).
(26.8% to 74.5% on CUB)". This accuracy degradation can be found in their appendix, section C2. By contrast, we prioritize interpretability, hence our baseline CAM without WSOL training. Instead of WSOL training, we use NBDT training (see later). Other XAI methods are tested on same metric and compared to CAM.
Summary of our contributions and results:
Vanilla CAM after Neural Backed Decision Tree (NBDT) training yields the highest performance, besting most other methods by significant margin. Heatmaps derived from Saliency method applied to the input layer obtains high scores as well but the method requires a peculiarly low operating threshold.
With the proper application of CAM-like concepts, heatmaps obtained from the inner layers of existing XAI methods can perform WSOL that beats the original CAM without NBDT.
3. The NBDT original paper (Wan al., 2021) trains larger models from scratch. However, we suc- cessfully fine-tuned pre-trained ResNet50 without causing the collapse of predictive performance.
RELATED WORKS AND LITERATURE REVIEW
Our main references are Choe's paper (Choe et al., 2020) (CVPR 2020), the NBDT paper (Wan et al., 2021) (ICLR 2021) and the following heatmap- heated XAI methods: CAM (as the baseline), GradCAM, Guided BP, Saliency and Deepl.fft.
CAM: weighted sum of feature maps. To use CNN for prediction, usually the last convolu-
tional feature maps f E ((C,HWW) is average-pooled and fed into fully-connected (FC) layer, i.e. FC(AugPool where C is the color channel, // height and W width of the images. Suppose our CNN classifies images into K different categories. Denote the i-th feature map before pooling by ft where i 1....,c. C, is the number of output channels before FC. The FC weight then given by E R K, To obtain CAM for class A {1,..., K). compute weighted-sum across channel e of f. so that CAM DE this paper, we tested various weighting schemes i.e. different wis and other empirical modifications aimed to yield better WSOL performance.
WSOL Metric. Popular metrics to evaluate WSOL are Top-1, Top-5 localization (accuracy or error) and GT-known localization accuracy (Choe & Shim, 2019; Singh & Lee, 2017: Zhang et al., 2018a;b; Zhou et al., 2016; Guo et al., 2021; Wei et al., 2021; Babar Das, 2021; Gao et al., well described in (Choe et al., 2020). Firstly, WSOL with only image-level labels can be an iol-posed problem. Secondly, the dependence operating threshold may lead to misleading comparison. Thus they introduced
2021; Xie al., 2021). The problems with these simple metrics
2

---

Under review as a conference paper at ICLR 2023
MazBorAAc(()) == maz,BorAAc(fn 8) where:
BoxAco(T, N E1 Iot' (bor(s(X(())) (n))) +)), 26
(1)
where t; 6 are the score map (operating) threshold and loU threshold respectively. B is the ground- truth box and boorss(X (x)(), ),)) r) is the tightest box surrounding the connected component with max
area of mask ((i,)) sXX ) 2 T). Note: Openev has been used to obtain bor(.) (contour algorithm,
RoundingRect etc).
Notice that ò is chosen as the threshold that yields the most number of "passing" IOU (> 8) across all N data: in the original paper's is python code, the values are "accumulated" over all samples before computing the final score. The consequence is clear: the best T is chosen in a discrete hit and miss manner so that a heatmap that is perfectly good (high IoU) at a non-chosen threshold might be considered miss in the chosen threshold. Remark: after publication, (Choe et al., 2020) introduced the improved version Max Bax such that (1) multiple thresholds 5 = (0.3, 0.5, 0.7) are used and their results averaged (2) max area assumption is removed so that the "best match" with ground totnts are found.
In (Choe et al., 2020), models are fine-tuned specifically to improve WSOL performance through the WSOL training. However, fine-tuning is done at the expense of predictive performance i.e. their classification accuracy drops. We avoid this. Since we are interested the XAI aspect of a model, we strive to achieve localization-based explanation for all class label predictions. As such, we attempt to simultaneously maintain the model's 's predictive power as we perform WSOL. The NBDT architecture appears to be a promising choice to achieve this.
NBDT is originally devised to improve not only the predictive power of a CNN, but also its inter- pretability through the inherent structure of a decision tree. Although it does not strictly improve the predictive power (sometimes the accuracy drops slightly, as shown NBDT paper and here), we bring NBDT one step further by demonstrating its additional benefit: NBDT can improve the WSOL capability of model through the application of heatmap-basse XAL, particularly (perhaps surprisingly) through the vanilla CAM.
NBDT inference proceeds in three steps. Step 1. DNN FC weights w E Rx is used to seed decision tree nodes, where K is the no. of classes: for leaf nodes i, n; we where i k {1, 2, (1.2...., K} i.e. node weight is seeded with row vectors of W; for all inner nodes i E {K + 1, NJ n; = ExEL(1) w2/ [ii) where & E L(i) are leaves in the subtree of i. Step 2. For each sample I, compute probability of the child j E C(i) of node i as p(ili) = softmax (ñ,, [}][]] where (..-) is the usual dot product. Step 3. We only use soft inference (which is shown to be the best in NBDT paper), so the final class prediction is & argmarrpp( where p(k) I, P.P' (C& and p (i)()) is the probability of each node in path Ps traversing the next node in path Ps (C
To obtain the tree, induced hierarchy is needed. In NBDT, the hierarchy is constructed using FC weights w of pre-trained model plus agglomerative clustering algorithm on w " ll; as before Wk is also the row vector of w. The weights are paired, and then constructed into binary trees whose leaves correspond to the classes, i.e. k = [1....,K),
Finally, fine-tuning is performed by optimizing a standard cross entropy loss plus the soft tree- supervision loss over class distribution of path probabilities D (p(k) so
L E (Dss(Dppss)
(Dyred: pred: Diabel) + C'E'sssssS ('ELoss(Prddts Diabet)
(2)
where CELoss denotes cross-entropy loss, 3r, we are supervision weights and decay factor respectively. Details are in the code (supp. material).
METHODS
As our starting point, we perform WSOL evaluations on GradCAM using MaxBoxAce on 4 different layers within ResNet50. GradCAM on layer 4, or vanilla CAM, is used as the baseline. We then repeat the same process for 3 other different XAI methods, each also on 4 different layers. Trial and errors are necessary during our experiments because some heatmaps are not visually sensible. In the next stage, repeat the entire process on ResNet50 after NBDT training. The entire pipeline is
3

---

Under review as a conference paper at ICLR 2023
Cany, RMA, HisL
Irput
+/
Layer
Sx
Laysr
Lover
-
Bottleenec (Cone (S9I..))
map
<kip connection
ResNe::0 layer 2 split
Laper
aryer
Loyer
++
Layer
FC
Xx
(mmgee)
Front moduln Ino graden)))
Iiput
Back Module propaeation
Figure 2: Layer splitting on ResNet50. Unlike GradCAM, some XAI methods are applicable only to the input layers in pytorch Captum implementation. We perform layer splitting allow the application of other XAI methods on different layers.
resource intensive as we experiment on multiple combinations of methods and layers, thus, this paper, we report only the result for ResNet50 and leave other models for future work. The ImageNet dataset used for training and evaluation (same setup as original NBDT code).
CAM-inspired sum of weighted feature maps. In the literature review, saw that CAM can be easily applied to the latest convolutional layer since there are e feature maps and also c rows w, the weight of the final FC layer. However, different localization information might also be present in different layers within a deep CNN (see fig. I(B),, hence we explore the effect weighted sum of feature maps inner layers.
GradCAM has been introduced as a generalization of CAM which extends the computation of CAM to layers other than the final convolutional layer. With the LayerGradCAM module of pytorch Captum, layerwise heatmaps can be extracted directly. For other methods, to obtain similar heatmaps, perform layerwise splitting (see below). Once layerwise heatmaps are generated, heatmap reformat (see below) is performed. This includes the normalization of heatmaps and the application of affremmntioned CAM-like sum of weighted feature maps with empirical choices of weights. Additional modifications are applied based on empirical observations and trial and error. For example, when we observe granular heatmaps, we apply average pooling to "patch up" the empty spaces.
Layer splitting. Fig. 2 illustrates splitting done at layer / = 2. Splitting at other layer is done similarly as the following. Step 1. Propagate input æ through the front module without any gradient, where front module is the successive layers in the deep NN up to and including layer 7. The output denoted as Is frona here NN is ResNet50. Step Perform heatmap'attribution computation as though back module is the entire neural network with us as the input, i.e. heatmap attr(M (N Noock, pred) where pred is the class prediction and attr =DeepLIFT, GBP or Saliency.
Healmap reformat is the post-processing phase with empirical settings. All heatmaps are resized to input size (H, W) and normalized to [o, 1]. By recentre we mean x - min(r).
1. For GradCAM, the resulting heatmaps have only 1 channel, hence they only need to be resized, recentred and then normalized.
For Saliency applied to the input layer, we follow the original paper: for each pixel, take max over all heatmap channels and then take absolute values. We then normalize and resize it accordingly. For Saliency applied to inner layers, modificd CAM concept is applied with pooling i.c. heatmaps obtained through the application of 2D average pooling weighted feature maps (F where weights are given by the total channel magnitude wiA = En fi regardless of class k. We found that these steps yield positive effects WSOL performance, allowing them to beat the baseline.
3. For DeepL.FF,, sum across channels (i.e. weighted sum with was = for all channels) yield positive performance. Several choices of variable weights DeepL.FFT did not yield satisfactory performance during our trial and error.
GBP on input layer is similar to DeepLIFT, but GBP on layer 1,2 and 3 yield interesting inverted heatmaps before processing. Hence, we perform the following: (1) invert heatmap h h, (2)
4

---

Under review as a conference paper at ICLR 2023
Table WSOL Evaluation relative to CAM without NBDT. Vanilla CAM after NBDT yields the best score but other methods beat the CAM baseline too, More significant improvemensvdeeraattio in MaxBoxAce are highlighted green/red. Heammapsaatributions are generated w.r.t layer I, see layer splitring. Note: 1 1= 0 means input. These scores are averaged across different values (including the
low scoring 5 = 70).
ResNet50
CAM (baseline) Max 2.144
No NBDT (MaxBoxAceV2.1)
NBDT (MaxBoxAccV2.1)
1
GradCAM
Saliency
DeepLIFT
GBP
GradCAM
Saliency
DeepLIFT
GBP
0
+0.390
+1.388
-0.828
-0.856
-0.040
+2.073
-0.841
-0.860
1
+0.602
-0.001
-0.352
+0.490
+0.446
+0.144
-0.421
+0.355
2
+0.921
+0.545
+0.742
+1.054
+0.716
+0.636
+0.640
+1.097
3
+0,911
+0.152
+0.764
+0.252
+0.625
+0.134
0.571
+0.294
4
(is CAM)
N.A.
+0.002
N.A.
+2,417
N.A.
+0,216
N.A.
apply sum across channels, (3) apply 2D maxpool before we finally recentre, normalize and resize the heatmaps. With these, GBP in these layers performs better than the baseline.
We also report N.A. results table 1. This means the heatmaps obtained are not making sense, for example, heatmaps with all zeros.
WSOL Metric. Once we have determined how to modify the XAI methods for a more visually sensi- ble localization, we will quantitatively measure their WSOL performances. We use MaxBoxAccV2... which is simply MaxBoxAccV2 The assumption was: the largest connected component is chosen for box estimation. Our rationale for using the assumption: a good XAI method is expected to yield accurate heatmaps/atriiutions in which large noisy patches should be suppressed or not exist altogether i.e. there is no large artifact. The most salient box for an accurate heatmap is thus already the largest component area. Furthermore, when deployed for application, ground-truth box is not necessarily available, hence there may not exist "best match" for MaxBoxAccV2 to use in practice; by contrast, there is always largest component as long as the heatmap is not too uniform.
(Choe et al., 2020) but with the original assumption (2) reapplied.
NBDT training and then WSOL evaluation. As previously mentioned, we will repeat the entire WSOL evaluation process, but this time with NBDT training process. Our experiments did not require retraining of ResNet50 from scratch. Our settings are nearly identical to the original NBDT paper with some differences as the following. We use 256 batch size (split across NVIDIA Tesla V100 GPUs) and disable learning rate scheduling, using instead a constant 0.001 learning rate. No mode collapse occurred throughout all training epochs. Each epoch run on the entire ImageNet training dataset. After NBDT training, the entire MaxBoxAceV2 V2.1 measurements aforementioned XAI methods and layers. Note that heatmaps are generated without the embedded decision rules for soft inferences, i.e reuse the FC layer (there has yet to be way propagate signals for XAI methods through the decision tree).
repeated for all
We collect the result (out of 14 epochs) from the best performing checkpoint. Results are only taken after at least 10 epochs of training (see appendix table 3) since we want to make sure that the effect of soft tree-supervision loss WSOL is present.
EXPERIMENTS AND RESULTS
MaxBoxAce stringent metric. Let the operating thresholds be Term = { (0, k 1, ..... 100}; this is also called CAM interval in the original code of (Choe et al., 2020), implemented as aranpe,a 1,deam) where deam = 0.01 is the cam_eurve
interval. _ntveeentervall....
Based equation (1), MaxBoxAce considers a bounding-box "correct" if the IoU exceeds threshold This is a very stringent condition since a single "best" operating threshold is used for all images. To score high on this metric, the collection of heatmaps need to fall into a particular form of distributions, as the following. Let the subset of operating thresholds used to extract the correct tight bounding box from heatmap be Tn {Th E Team (s(.....)) Tk) 2 5)n where 1,.-., N indexes
5

---

Under review as a conference paper at ICLR 2023
particular image. Then, to obtain high score, qualitatively, many images need share the same interval thresholds.
N such that [T] approaches and the intersection of clements in 7 is strictly non-empty. This appears to be a very stringent condition to achieve: as gauge, throughout the paper, the best score obtained is 12.8 (which is + +10.688 from baseline average), achieved by CAM after NBDT at ō = 30. In other words, only 12.8% samples are localized correctly. The following are the advantages and disadvantages of using MaxBoxAce for further considerations.
More precisely, there has to exist element T in the power set P n .,,
Advantages. MaxBox
be very useful:
1. Methods that score high on this metric are useful in practice mainly because there is a consistent window of operating threshold that will yield good bounding-boxes boxes. We do not need worry about selecting different "correct threshold" each time during application.
2. Anti axtreme-threshooding artifact, see next sub-section.
Disadvantages. Vanilla CAM scores very low on this metric although it has been considered successful in WSOL.. We obtain a score of only 2.143 for CAM baseline (see table 1) averaged across 8 = 30, 50, 70. This means that, on average, only 2% of the entire dataset tested is considered correct: given the best T particularly chosen through MaxBoxAce, most IoU does NOT exceed 6. From fig, 4 last row, the scores for = 70 show that WSOL break down severely at high threshold. Hence, for XAI, the averaging of thresholds recommended by (Choe et al., 2020) may not be suitable. For further considerations:
1. Before the WSOL training in (Choe et al., 2020), it is well-known that existing pre-trained CNN models score well on class-labeling accuracy (typically after finetuning). However, their performances on MaxBoxAce have not been reported. In this work, we fill the gap in the literature and show that the scores are very low.
2. After WSOL training. WSOL performance improves significantly: we must mention that the MaxBoxAce scores for their trained models are very high, reaching around 60 in (Choe et al., 2020). However, final class label prediction accuracy might have degraded as shown in section C2 of their appendix. To again quote one of their results, "there are in general great fluctuations the classification results (26.8% to 74.5% on CUB)."
While we list them as disadvantages, we hope that the result might instead encourage more efforts to improve WSOL based on this metric.
Improving WSOL beyond baseline. Here, XAI methods are used to extract localization maps from the inner layers of ResNet50. With our empirical adjustments, they are able to score better than CAM on MaxBoxAcee2.1 metric. Table left (No NBDT) shows the improved results of localization that we have obtained with only few steps of post-processing. The original Saliency method appeared to score the highest (amongst others with no NBDT), attaining + +1.388 relative to baseline and high spike (with peak near 10) at 8 = 30 in fig. 4. Other methods are able to achieve above baseline scores as well, especially when XAI methods are applied to / = 2 layer.
Comparison with visual perception. Higher scoring heatmaps might not appear correspond well with human perception of localization, as seen in fig. 3(B) and (C). To understand this, readers should revisit MaxBoxAcc and equation (1) in details. Note that for image, each bounding box is obtained using a "best" threshold T i.e. MaxBoxAcc selects only one of all the cascading boxes (see fig. 3(B.C) last row) whose 7 gives the best score after taking into account every image in the dataset. This could work well with many other images but is not necessarily good on the particular image being inspected, hence the localization might appear visually poor.
Tighter cascading bounding boxes better? Fig. 3(B) also seems to indicate that a set of cascading bounding boxes that are tighter around the ground-truth bounding boxes score better than the haseline; see fig. 3(A) for a simpler illustration. For example DeepLIFT'ss bounding boxes score better than CAM even though CAM's heatmaps appear visually better. Why? Given a "best" т, there is naturally a higher probability that a good bounding-box being eventually selected comes from one of the tighter bounding box if most bounding boxes are concentrated around the ground-truth CAM heatmaps appear to localize very well, but there is a lack of precise, tight bounding. Likely this is the reason why other XAI methods score better than the baseline CAM under MaxBoxAce metric. The above suggestions are based on the observations of multiple images; we show only few images here.

---

Under review as a conference paper at ICLR 2023
(A) Wiith
(0)
commeilyy precccee
(C)
lbrvester 5951 cornectly predeced
according Mactectze?
(SraoAAM
40.55
DeepURT2
bowelir
CAM
caP!-2
No memberle PuatTrpp
Predution
Figure 3: Heatmaps and cascading bounding boxes (CBBs). (A) Top: CAM-like CBB. Bottom: tighter CBBs appears score better MaxBoxAce. (B) Row and 2 show heatmaps at normal and extreme colouring thresholds. Row are CBBs obtained from row 4 with each rectangle filled with colour (with high alpha transparency). Row 4 shows CBBs: blue bounding boxes obtained from lower threshold т, yellow from higher threshold. MaxBoxAccV2.. scores of each method is listed on the top. Recall: the score computed for each individual image, but an average over all images. (C) Similar to (B).
Readers must understand that the heatmap of single image is representative of the final score, especially due to the fact that the metric is computed over all N in equation (1).
Anfi thresoiitrrssolddngg artifact. A special benefit from the way MaxBoxAce is defined is that we have an anti extreme-thresholding artifact, best illustrated by fig. 3(B) GBP / = 2. At very high thresholds, artifacts might be mistakenly selected by contour algorithm, and the results are small yellow boxes in seemingly random places. However, if the large majority of bounding boxes are tightly overlapping around the object's ground-truth bounding boxes, MaxBoxAce's selection has a higher chance of picking up more sensible bounding box that corresponds to the ground-truth as suggested the previous paragraph.
Performances at different 5 thresholds. We conducted multiple experiments across different XAI methods and layers. The results are also presented as BoxAcc vs operating threshold plots, similar to the fig. 5 of (Choc et al., 2020) for ImageNet. Our results shown in fig. (dotted lines for non-NBDT results). As before, the baseline is CAM i.c. the green dotted line GradCAM at Results at 8 30 without NBDT are shown below:
1. the original saliency attains a very high score at peculiarly low r threshold (blue high narrow spikes). From fig. 1(A), Saliency heatmaps appear very granular, although correspond relatively well to the shape of the object. Fig. 4 shows that within a small range of low thresholds, Saliency bounding boxes localizing more accurately than other XAI methods.
that they
GradCAM appears to perform better the right compared to layer 3, i.e. there is smaller window of slightly higher thresholds for layer 2 perform WSOL better.
2,3. Layer 2 curve narrower and slightly leaned to
3. DeepLIFT curves are less predictable. At input layer and early layers, DeepLIFT performs very poorly. However, similar to GradCAM, a relatively better localization power is found in the inner layers = 2, 3, consistent the averaged results shown in table (red highlight, / 0, 1).
4. GBP is equally unpredictable. GBP at deeper layers show wider curves well. The performance at lower thresholds in = 2 appears to be the best.
In general, 1 = 2 seems perform relatively better across all methods. As previously mentioned, the metrie very stringent, and nearly all the scores are below 10.
At loU threshold 5 = 50, the patterns similar. Naturally. lower scores attained since the condition for the correctness of a bounding box is more stringent. At the highest = 70, the results appear to have broken down into irregular hits and misses, resulting in jagged curves and very low
7

---

Under review as a conference paper at ICLR 2023
10 Wrrhe Exckc
nomoiggtssevdl (?)
Figure 4: Performance (BoxAce) at varying thresholds (r) plotted against the same vertical axis per IoU threshold 8. XAI method. layers and post-NBDT result are indicated in the legends; when no layer is indicated, the XAI method applied on the input. Top/MicddleBBottom are results for 5 30, 50, 70 respectively. Dotted-lines are plots for non-NBDT WSOL, solid lines for WSOL after NBDT training. The overall highest score is attained by CAM (GradCAM 4) after NBDT training. This image is best viewed electronically, zoomed in.
accuracies. The recommendation to use high 5 for averaging seems unsuitable for most of these methods.
After NBDT: training with Soft Tree-Supervision loss. The entire experiment above repeated after NBDT-based training is applied. Recall that our objective is to improve WSOL (better MaxBox- Acc) while maintaining predictive accuracy. NBDT has the benefit of improved interpretability from its decision tree structure.
The baseline method CAM improved significantly after NBDT training with an average of +2.417 improvement compared to pre-NBDT model (see table 1). The best improvement is achieved on CAM at 30, with a remarkable +10.688 increase. Furthermore, for all thresholds, post-NBDT CAM show the widest and tallest BoxAcc curves, shown in fig. 4 green solid line (GradCAM 1 == 4); it is in fact the best performing method throughout our experiments.
The overall result is mixed when include other XAI methods into consideration. Most results beat the Pre-NBDT baseline CAM although some WSOL performances only marginally improve e.g. for GBP / 2, the score 1.097 1.054. The scores generally remain low, and their improvements are sometimes lower than pre-NBDT models e.g. DeepLIFT for / = 0, 1, 2, with NBDT are all worse than pre-NBDT results. In general, other than CAM, the only other visible change after NBDT is Saliency at = 0 (original version). Across all IoU thresholds = 30, 50, 70, its MaxBoxAcee2.1 performance increases even higher compared to other XAI methods. Its best threshold appears to shift to an even lower value, hence taller blue spikes in fig. 4 with peaks that are nearer to 0 along the horizontal axis.
In this section, ResNet50 has undergone 13 epochs of training and the 12-th epoch is used. We only consider the model after 10 epochs of training just to ensure that NBDT effect there (previous epochs may have better scores). The model that use attains accuracy of 72. 16% which is 2.49% drop from the original class-labeling accuracy; compared to other results in table 2, there is nothing remarkable with the change in accuracy. The original NBDT paper also presents result where NBDT causes a small accuracy drop in one of their models (and a small increase another), so believe this is not crippling issue. However, this is significant when compared the classification
8

---

Under review as a conference paper at ICLR 2023
Table 2: Class prediction accuracy. Apart from NBDT (ours), the other results are quoted from the original NBDT paper (Wan et al., 2021), including XOC (Alaniz & Akata, 2019), NofE (Ahmed al., 2016).
Method
NBDT (ours)
NBDT
NBDT
XOC
NofE
Backbone
ResNet50
EfficientNet
ResNet18
ResNet152
AlexNet
Original Acc
74.65%
77.23%
60.76%
78.31%
56.55%
Delta Ace
-2.49%
-0.63%
+0.50%
-17.5%
+4.7%
Explainable Ace
72.16%
76.60%
61.26%
60.77%
61.29%
accuracy obtained from WSOL training performed by (Choe et al., 2020). As shown in their section C2 appendix, WSOL training might cause classification accuracy to suffer e.g. prediction on CUB dataset may go low 21% in accuracy. Their ResNet50 classification performance is generally in the vicinity of 63%. Furthermore, we have shown that fine-tuning ResNet50 with NBDT is possible, which is much more efficient than the original NBDT paper that initialized weights from scratch and performed a very long 200-epoch training for ImageNet.
CONCLUSION
Best values. Naturally, it is easier to attain correct localization with lower 5. It might be debatable whether the average over different 5 values is necessary or the set values should be lowered e.g. 10, 15, 30 (at the risk of overestimating IoU performance). Regardless, we present best score before NBDT is attained by Saliency method with the score +8.428 above CAM baseline 30, while the best score post-NBDT +0..888 above baseline, also
best findings:
30.
Shortcomings, future directions and suggestions. Repeating the experiments by sweeping through every possible variation of XAI methods for WSOL over different layers is not feasible. To prevent blind search of WSOL information hidden within many possible layers, future neural network archi- tectures can be designed with special layers that are specifically taylored to perform WSOL, aimed at optimizing MaxBoxAce metric. As we previously mentioned, a method with good MaxBoxAce score can very useful terms of practicality (less maintain the accuracy of post-NBDT models that interpretability. WSOL and predictive powers of the future models are all improved. The integration of XAI methods with the NBDT might be useful too: more research can be done to find meaningful way to propagate signals in the XAI methods through the decision tree.
T). More efforts can spent to
Summary. We have presented investigations of (1) existing WSOL metric that handles ill-posed localization problem (2) the different WSOL capability of various XAI methods at different layers. We have also repeated the entire experiment for the same model after NBDT training and compared their results. Caveat. Finally, we should mention that the technical and subtler details in the appendix worth perusing. We have used the codes from main papers (NBDT and Choe's s) mostly as they are presented in their respective github repositories. There might be some concerning details that readers would like pay attention regardless of the fact that they have been published in top conferences.
ACKNOWLEDGMENTS Anonymous for now
REFERENCES
Karim Ahmed, Mohammad Haris Baig, and Lorenzo Torresani. Network of experts for large-scale image categorization. In ECCV, 2016.
Stephan Alaniz and Zeynep Akata. Xoc: Explainable observer-classifeer for explainable binary decisions. ArXiv, abs/1992.01788. 2019.

---

Under review as a conference paper at ICLR 2023
Sadbhavana Babar and Sukhendu Das. Where to look?: Mining complementary image regions for weakly supervised object localization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1010-1019, January 2021.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE, 10(7);1-46, 07 2015. doi: 10.137 wow.oral..pon..0..1..
URL https::////// /tt///oi...... www.:o.......ooooooo 1371/ wow.///rr/.o/a.....ooaa.aaaaaaaa www.puaaaaa..n............eeeeee 0130140.
Junsuk Choe and Hyunjung Shim. Attention- dropout layer for weakly supervised object localization. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern
Recognition (CVPR), June 2019.
Junsuk Choe, Seong Joon Oh, Seungho Lee, Sanghyuk Chun, Zeynep Akata, and Hyunjung Shim. Evaluating weakly supervised object localization methods right. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Wei Gao, Fang Wan, Xingjia Pan, Zhiliang Peng, Qi Tian, Zhenjun Han, Bolei Zhou, and Qixiang Ye. Ts-cam: Token semantic coupled attention map for weakly supervised object localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2886- 2895, October 2021.
Guangyu Guo. Junwei Han, Fang Wan, and Dingwen Zhang. Strengthen learning tolerance for weakly supervised object localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7403-7412, June 2021.
Mohammad A. A. K. Jalwana, Naveed Akhtar, Mohammed Bennamoun, and Ajmal S. Mian. Cameras: Enhanced resolution and sanity preserving class activation mapping for image saliency. 2021 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 16322-16331, 2021.
Pieter-Jan Kindermans, Kristof T. Schütt, Maximilian Alber, Klaus-Rober Müller, D. Erhan, Been Kim, and Sven Dähne. Learning how explain neural networks: Patternnet and patternatrribttio.. In ICLR, 2018.
Maximilian Kohlbrenner, Alexander Bauer, Shinichi Nakajima, Alexander Binder, Wojciech Samek, and Sebastian Lapuschkin. Towards best practice explaining neural network decisions with 1rp. In Proceedings of the IEEE International Joint Conference on Neural Networks (ICNN), pp. 1-7, 2020. doi: 10. 109111NNN40000 202,86000..22002229775 URL http: doi http:://dxx.....or///.0 #109/IJCNN40 605 2020 9206975.
Grégoire Montavon, Sebastian Bach, Alexander Binder, Wojciech Samek, and Klaus-Robert Müller. Explaining nonlinear classification decisions with deep taylor decomposition. Pattern Recognition, 65:211-222, 2017. doi: 10.60 10.1016//.pptcoo.g.oo pal.og.201..11..oo 11.008. 1016/3 pat.oo...p.o.......o............ 2016 6.1 11. 008.
URL http://dx. dwtp//xxxxiiiiio.o/ooo//./.....t org/10.
Mohammed Bany Muhammad and Mohammed Yeasin. Eigen-cam: Class activation map using principal components. 2020 International Joint Conference Neural Networks (IJCN), pp. 1-7, 2020.
Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. Grad-cam: Why did you say that? visual explanations from deep networks via gradient- based localization. CoRR, abs/16110.0345 2016. URL hatpp://arxea org. org/ abs/ 1610. 02391.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje Learning important features through propagating activation differences. volume 70 of Proceedings of Machine Learning Research, pp. 3145-3153, International Convention Centre, Sydncy, Australia, 06-11 Aug 2017. PMLR. URL htttp:/ppooceeee
.ml.. pwwss/vvesseeeeeeee /0/ wiiiiaiaaria htmi.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. In Workshop at International Conference Learning Representations, 2014.
10

---

Under review as a conference paper at ICLR 2023
Krishna Kumar Singh and Yong Jae Lee. Hide-and-seek: Forcing network to be meticulous for weakly-superrised object and action localization. 2017 IEEE International Conference on Computer Vision (ICCV). pp. 3544-3553, 2017.
Leon Sixt, Maximilian Granz, and Tim Landgraf. When explanations lie: attributions fail. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference Machine Learning. volume 119 of Proceedings of Machine Learning Research, pp. 9046-9057. PMLR, 13-18 Jul 2020. URL hwwpt::pp///o. httttps:/prroceeddiiii wwt.:sm..........l.....mm sixt22al html.
Why many modified BP
Jost Tobias Springenberg, A. Dosovitskiy, T. Brox, and Martin A. Riedmiller. Striving for simplicity: The all convolutional net. CoRR, abs/1412.6806, 2015.
Alvin Wan, Lisa Dunlap, Daniel Ho, Jihan Yin, Scott Lee, Suzanne Petryk, Sarah Adel Bargal, and Joseph E. Gonzalez. (NBDTT): Neural-backed decision tree. In International Confer- ence on Learning Representations, 2021. URL httpt:ttp///c....tt /oppnreeeeeeeeeeeeeeeeeeeeeeeeee ew, net/ forum? d= mCLVeeppINE.
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-cam: Score-weighted visual explanations for convolutional neural networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp-
111-119, 2020.
Jun Wei, Qin Wang, Zhen Li, Sheng Wang, S. Kevin Zhou, and Shuguang Cui. Shallow feature matters for weakly supervised object localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 5993-6001, June 2021.
Jinheng Xie, Cheng Luo, Xiangping Zhu, Ziqi Jin, Weizeng Lu, and Linlin Shen. Online refinement of low-level feature based activation map for weakly supervised object localization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 132-141, October 2021.
Xiaolin Zhang, Yunchao Wei, Jiashi Feng, Yi Yang, and Thomas S. Huang. Adversarial complemen- tary learning for weakly supervised object localization. 2018 IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 1325-1334, 201 8a.
Xiaolin Zhang, Yunchao Wei, Guoliang Kang, Yi Yang, and Thomas Huang. Self-produced guidance for weakly-supervised object localization. In ECCV, 2018b.
B. Zhou, Khosla, Lapedriza, Oliva, and A. Torralba. Learning deep features for discriminative localization. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2921-2922 June 2016. doi: N0.1109/CVPR.2016.319
A APPENDIX
A.1 DATA AND SETUPS
Our codes are available in the supplementary materials and they are directly derived from the github repository of our two main references, namely Choe's paper /tttp::/githuu wtwoovaawwwooevvlllll.l.o
and NBDT paper https: ////gtttt thub. wwwm/aviinwnnnnnmmomwwwmommooaaa decision, trees. We strongly advise users to first set up everything speci- fied in BOTH repository (including the arrangement of data). Make sure that both codes work before running our code.
In our code, the code base is Choe's github repository, and NBDT is included as package (the folder is named 'nbdd'). The parts of codes that we write ourselves are clearly denoted with prefixes "x" to distinguish them from the original codes. Instructions to run the codes are listed in README.md.
A.2 TECHNICALITIE AND SUBTLER DETAILS
This paper is based on the first version of available codes. The next versions with minor fixes and clean-ups will be available after publication.
11

---

Under review as a conference paper at ICLR 2023
Table 3: Accuracy (%) by epoch. Epoch 12 (starred) is used for experiment.
epoch 0
3
4
5
6
7
8
acc (%)
72.32
72.02
72.24
72.50
72.57
71.91
72.0
72.29
72.22
72.09
epoch
10
11
12a
13
(%)
72.06
71.96
72.16
71.56
A.2.1 EVALUATION AND METRIC
Codes. As mentioned, our main references are Choe's paper (Choe et al., 2020) (CVPR 2020), the NBDT paper (Wan al., 2021) (ICLR 2021). We use their evaluation codes as they are and followed their dataset settings.
MaxBoxAce a stringent metric. Why do we talk about how stringent is the metric? In Choe's paper, the CAM baseline result for models without WSOL finetuning is not presented anywhere, and we reiterate that our paper is intended partially to fill this gap. Because of the stark difference between the baseline obtained (average 2. 143 for our bascline) and their results which are fine-tuned for WSOL (around -60), we are compelled to present readers a clearer picture regarding the MaxBoxAce metric.
MaxBoxAccV2.11 To reinstate assumption (2), we set multi_contour to False.
Top 1 NBDT accuracy. We follow the evaluation code of NBDT paper as is. The accuracy reported in the paper is computed on the validation dataset of ImageNet, so we do the same. We use their exact same code, The instructions for training and evaluation we followed are listed in their original README.md.
A.2.2 ARCHITECTURE
In our code, we use ResNet50CAM, which is technically just ResNet50, but with additional function including CAM computation and treate-split-at-laeere perform layer splitting.
A.2.3 NBDT TRAINING AND CHECKPOINTS
Pytorch DataParallel is for training across different GPUs. Despite the warning given the main Pytorch page, we follow the original NBDT paper since their superior results are obtained using the said module. Some weaknesses of DataParallel include lost in-place updates, https: wraytorrch.rg//deeo.
nn Dat DataParallelll htt.aalllhhll...................
Main result is shown in table 3. Accuracy used is top-1.
For authors' own reference: are limited by our GPU server queue. More precisely, every job can be submitted maximum of 48 hours, hence we need to split training across several jobs. To prevent repetitive, unproductive training, we save the checkpoint at each epoch and separately save the best result of that particular relabeling are shown in table 4. We notice there are minor mislahelings, e.g. if the latest epoch saved is epoch n, in the next run, each epoch is started while it should have been n + For now, we rectify and present them in the "relabeled epoch" column. Most importantly, the NBDT-trained checkpoint we used checkpoint before relabeling, or overall epoch 12 (after relabeling) with accuracy 72.16%.
rather than loading from the latest best checkpoint. Results with
12

---

Under review as a conference paper at ICLR 2023
Table 4: Please refer to table for the main result. This only for authors' reference: this table is included only for reference to minor mislabelling issue (see explanation in appendix section A.2.3). Results as stored in pbsarxiv folder (left column). Starred epochs denote epochs where checkpoints are separately saved. epoch=12 taken for experiment, i.e. the model after 13-th epochs of training.
Run no.
epoch aco(%)
relabeled epoch acc(%)
0": 72.32. 1:72.02 2: 72.24
Same
2
2*; 72.50, 3*:72.57, 4:71.91 5:72.0
3*; 72.50 4*:72.57 5:71.91 6:72.0
5: 72.29*, 6: 72.22, 72.09, 8: 72.06
7#; 72.29, 72.22, 9: 72.09, 10: 72.06
4
71.96, 9*: 72.16 (used), 10: 71.56
11: 71.96, 12*: 72.16 (used), 13: 71.56
13


Reviews:
[{"review_id": "omWERAVEiW", "reviewer": "Reviewer_GA9G", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper measures the WSOL capability of existing heatmap-based XAI method applied on ResNet50 and improves them.", "summary_of_the_review": "The overall contributions of this paper are not significant.", "strength_and_weaknesses": "This paper finds that XAI methods perform WSOL with very sub-standard MaxBoxAcc scores.\n\nThe experiment is then repeated for the same model trained with Neural Backed Decision Tree (NBDT) and finds that vanilla CAM yields significantly better WSOL performance after NBDT training.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "mF5Z92nsMkH", "reviewer": "Reviewer_8A57", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This manuscript aims to improve the localization performance of XAI models. The authors claim that incorporating NBDT into CAM can result in improved BoxAcc metric.", "summary_of_the_review": "This manuscript resembles more a poorly-written school report than a research paper.", "strength_and_weaknesses": "In summary, this manuscript is of very bad quality. \n\nIntroduction:\n- The research question is insufficiently motivated. What makes it worthwhile to improve the localization performance of XAI methods?\n- What's the difference in definition between XAI methods and WSOD methods?\n- \"The NBDT original paper (Wan et al., 2021) trains larger models from scratch. However, we successfully fine-tuned pre-trained ResNet50 without causing the collapse of predictive performance.\" I don't see why this is a contribution.\n- \"Firstly, WSOL with only image-level labels can be an ill-posed problem.\" I don't see why.\n\nMethod & Experiment:\n- Why choose MaxBoxAcc over mAP (detection) and Dice (segmentation)?\n- Table 1, please report absolute performance.\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "Yy_Bgaf87dM", "reviewer": "Reviewer_53je", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers the problem of weakly supervised object localization (WSOL), in which the goal is to accurately predict bounding boxes using only image-level labels at training time. The goal of this paper is to assess the performance of different explainable AI (XAI) methods (GradCAM, Saliency, DeepLIFT, GBP) for WSOL. The paper also asses the utility of neural backed decision tree (NBDT) training for the backbone and studies the utility of using different weighted sums of activation maps from different layers to produce localization heatmaps. All experiments are performed on ImageNet. ", "summary_of_the_review": "This paper addresses and interesting problem and undertakes a worthy benchmarking project. However, this work seems to be built on significant misunderstandings of prior work, and makes serious negative claims about prior work that are insufficiently supported. Of course it is possible that I am the one with a profound misunderstanding, in which case I hope to be corrected. ", "strength_and_weaknesses": "# Strengths\n* WSOL is an interesting problem that may lead to label-efficient methods for predicting bounding boxes.\n* Benchmarking new methods on WSOL is a worthwhile service to the community. \n* I agree that MaxBoxAcc is a somewhat odd metric, and I appreciate critical thinking about its pros and cons. The paper's related discussion of thresholds is interesting. \n* The paper considers a number of different XAI methods, and the idea to train using NBDT is interesting. \n* It is interesting to see the qualitative results in Figure 1 showing that XAI methods can produce heatmaps that are unsuitable for WSOL. \n* The visualizations of \"cascading bounding boxes\" in Figure 3 are interesting. \n* The per-layer experiments in Figure 4 are interesting. \n\n# Weaknesses\n\n* I suspect there may be some fundamental misunderstandings about WSOL and in particular [Choe]. The paper makes reference to \"WSOL training\" and says that [choe] performs \"a class agnostic training to optimize\" for WSOL performance. I have read [choe] and this claim does not seem right to me. The typical approach in WSOL (and in [choe]) is to (i) train a CNN end-to-end on a multi-class classification task, with various bells and whistles (which is the main differentiator between WSOL methods) and (ii) generate boxes by using CAM, thresholding, and then picking a connected component to draw a box around. Nothing about the training process is \"class agnostic\" - the different WSOL methods in [choe] differ in their method for training the backbone, but all of them train multi-class classifiers. This seeming confusion is repeated on Page 6 (\"However, their performance on MaxBoxAcc have not been reported.\") where it is used to insinuate that [choe] did not report MaxBoxAcc results for methods \"without WSOL training\" because performance was low. I find this all very strange, and I am eager to hear a clarification. \n* Related to the previous point, the paper says that in [choe] \"models are fine-tuned specifically to improve WSOL performance\" - this is not true to the best of my knowledge. \n* The numbers reported in Table 1 are extremely low compared to [choe]. The paper mentions that \"Vanilla CAM is equivalent to GradCAM applied to layer 4\" - if so, then that result in Table 1 disagrees with the exact same result in [choe] by a large margin. Unless an excellent explanation is provided, implementation issues seem likely. \n* The paper uses only one dataset, which is not standard in the WSOL literature. Most papers use CUB and ImageNet. [choe] introduces a third. [cole] introduces a fourth.\n* The paper says that it \"attempts to simultaneously maintain the model's predictive power as we perform WSOL\" unlike the methods in [choe] which improve localization \"at the expense of predictive performance\". However, there is no comparison provided between the predictive performance of the proposed methods and the methods in [choe]. On Page 6, there is a claim that \"final class label prediction accuracy results have not been reported\" by [choe] after \"WSOL training\" because it \"might have degraded significantly\" - the numbers in question can be found at [choe-results], which is linked on the GitHub page for [choe]. In fact, the results from [choe] for ResNet-50 on ImageNet are similar to those in Table 2. \n* Related to the previous point, I would caution against publicly suggesting that other authors deliberately omitted unflattering results without providing any evidence. One instance is described in the previous bullet point, and another instance is where the paper concludes: \"Finally, we should mention that the technical and subtler details in the appendix are worth perusing. We have used the codes from two main papers (NBDT and Choe’s) mostly as they are presented in their respective github repositories. There might be some concerning details that readers would like to pay attention to regardless of the fact that they have been published in top conferences.\" This seems to ominously imply that there are significant flaws in those papers. When we turn to the appendix, there seems to be just a repeated allegations against [choe] based on what I perceive to be a deep misunderstanding of that paper, which I have addressed above. Identifying flaws in published work is absolutely crucial, but it should be done professionally, and with great care and attention to detail. \n\n[choe]:\n\n@inproceedings{choe2020evaluating,\n  title={Evaluating weakly supervised object localization methods right},\n  author={Choe, Junsuk and Oh, Seong Joon and Lee, Seungho and Chun, Sanghyuk and Akata, Zeynep and Shim, Hyunjung},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  pages={3133--3142},\n  year={2020}\n}\n\n[choe-results]:\n\nhttps://docs.google.com/spreadsheets/d/1O4gu69FOOooPoTTtAEmFdfjs2K0EtFneYWQFk8rNqzw/edit#gid=0\n\n[cole]:\n\n@article{cole2022label,\n  title={On Label Granularity and Object Localization},\n  author={Cole, Elijah and Wilber, Kimberly and Van Horn, Grant and Yang, Xuan and Fornoni, Marco and Perona, Pietro and Belongie, Serge and Howard, Andrew and Mac Aodha, Oisin},\n  booktitle={European Conference on Computer Vision},\n  year={2022}\n}", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9_wr8VcrHL", "reviewer": "Reviewer_zRUD", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors discussed the work of (Choe et al., 2020) for the evaluation of WSOL methods. and the work of (Wan et al., 2021)  for the neural-backed decision tree (NBDT) method. The authors describe both works, and also evaluate some heatmap-based explainable methods such as GradCAM, saliency, Deeplift, and Guided Backpropagation, and compare them to the CAM method. They also compare with and without NBDT training. Evaluation is done on the Imagenet dataset.", "summary_of_the_review": "+ This paper reads more like a tutorial rather than a research paper. It lacks any innovative contribution. In addition, it is not clear why authors refer to WSOL methods (Choe et al 2020) as class-agnostic - they are class-aware methods. ", "strength_and_weaknesses": "Strength: \n+ The paper concerns WSOL tasks and explainability, which are very important and relevant.\n+ The empirical results show some benefits of the method, and some ablation studies are provided.\n\nWeakness:\n+ There is a lack of contribution and novelty.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject