Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose a gradient-weighted visualized explanation technique for object detectors called ODAM. The technique produces instance-specific heat maps indicating the regions that have an impact on the object prediction. It combines attributions from the class and object localization prediction to produce a single heatmap that is used during the training.

To address the problem of crowded scenes and avoid potential heatmap “leaks” onto neighboring objects, the authors propose a method called ODAM-Train to better identify and discriminate the detected object from its neighbors. Two terms are added in the original learning loss function to constraint the consistency and separations of the detected objects.

To mitigate the NMS post-processing problems that still occur in crowded scenes with ODAM-Train, the authors propose ODAM-NMS. ODAM-NMS uses both the IoU between two predictions, and the heatmap correlations to evaluate the probability that they correspond to the same object.

The approach is evaluated on MS COCO dataset using various fidelity and accuracy metrics for explanation and shows good results compared to D-RISE.


strength_and_weaknesses: == Strengths ==
- One of the few white-box  instance specific explanation generation for object detection
- The explanation combines class-specific and object localization information in a single heat-map
- ODAM-Train generally improves the object localization
- Exploits explanation to improve the performance of NMS (but slightly) for crowded scenes


== Weaknesses == 
- The explanation is not neutral since the model is modified by an extra loss term during learning.
- Mixing localization and class-specific information in a single representation may lead to confusion about the explanandum (what is explained). 
- The computation of the explanation requires the object bounding box ground-truth.



clarity,_quality,_novelty_and_reproducibility: - Novelty

The paper proposes an approach for generating instance specific white box explanations of object detection that can be applied to one-stage and two-stage architectures. As far as I know, this is the first study about this kind of problem. However, the approach is a simple restriction to the detected instance of well known feature weighting explanation generation such as GradCAM: the main innovation compared to this family of explanation is to have introduced a specific training loss that is expected to improve explanation.


- Clarity and quality

The paper is clearly written and easy to follow. The experiments are rather convincing.

There are however a few points that need to be clarified.

** Miscellaneous questions and comments

Does the model learned with the ODAM-Train loss have the same performance as the original algorithm, what is the influence of adding the new loss ?

Did you compare the results using $H_{IoU}$ or $H_{class}$ only? Or without using the ground truth bounding box to compute the IoU term in $H_{comb}$ ? As a complementary comment, I found it quite strange to use the ground-truth to build the explanation for $H_{IoU}$ and not for $H_{class}$.

In the same vein, does ODAM-NMS exploit the explanation that has been computed using the bounding box ground truth for $H_{IoU}$?

I didn’t understand how are computed the gradients $\partial Y_c / \partial A_k$ used to build the GradCAM and GradCAM++ explanations. What is $Y_c$, is it the predicted class of a given instance?

When using attention-based architecture, [1]  proposes another  white-box method to generate instance-level explanations for object detector predictions? Can the approach be applied to transformer architectures such as DETR [2], and what would be the difference ?

** Several issues in the writing

No appendix provided, although it was claimed to contain useful information (pseudo-code for ODAM-NMS for instance).

In Tab. 5, does the indicated computing time correspond to Odam-NMS + heatmap computation, or only NMS once heatmap are already computed ? Did you measure the computing time of heatmap only ?


- Reproducibility

The appendix is not provided and does not contain the pseudo-code for Odam-NMS as announced in the main paper.

The learning parameters of Odam-Train are not provided (hyperparameters, weighting of losses)

- References

[1] Chefer, H., Gur, S., & Wolf, L. (2021). Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 397-406) 

[2] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020). End-to-end object detection with transformers. In European conference on computer vision (pp. 213-229). Springer, Cham.

summary_of_the_review: The paper proposes a novel approach for generating instance specific white box explanations of object detection that can be applied to many current architectures. The evaluation on  MS-COCO gives good results on several metrics. There are however a few points that need to be clarified to recommend acceptance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an interpretation method for object detector. The authors slightly modify the existing Grad-CAM method to make it location-sensitive. In addition, ODAM-Train is also proposed for improved interpretation for overlapping objects. The experiments show that the proposed method provides better interpretation of object detector than previous methods, and can be applied to better NMS method.
strength_and_weaknesses: Strength
1. The proposed method is sound and intuitive. 

2. The experiments are thorough and show the effectiveness of the proposed method.

3. The generated explanation can be utilized to improve NMS

Weakness
1. My major concern is a technical novelty. The proposed method is simple and effective, but naive. The modification of Grad-CAM seems straightforward.

2. I am not sure if ODAM-Train can provide "real" explanation of the neural network. It is natural for different predictions to produce different interpretations, but ODAM-Train forces the explanations of different predictions to be consistent. Although this may improve several numerical results, it is questionable whether it can provide real explanations of neural networks. 

-- In Figure 1, ODAM-Train seems focus on the center of the object, but intuitively thinking, contour of the object is more important than the center for object detection.

-- Please provide the results of deletion and insertion experiments of ODAM-Train

3. Due to the ReLU operator in Eq (2), the gradient direction is very important for Grad-CAM. Gradient direction of the classification score is quite intuitive (which feature is important to increase the classification score), but the gradient direction of bounding box regression is ambiguous. Which gradient should be considered? The gradient that increases the value of box regression? Or gradient that decreases the value of box regression? The increase of values of x1 and y1 shrinks the bounding box, and the increase of values of x2 and y2 expands the bounding box. 

4. There are some missing references. [A] also aims to interpret the object detectors. [B] also obtains a visual saliency map for object detectors. In particular, [B] can be directly compared to the proposed method. Please provide the comparison with [B].

[A] Wu, Tianfu, and Xi Song. "Towards interpretable object detection by unfolding latent structures." ICCV 2019.

[B] Lee, Jungbeom, et al. "Bbam: Bounding box attribution map for weakly supervised semantic and instance segmentation." CVPR 2021.

5. Following recent trends, I recommend the authors to apply the method to transformer-based object detectors like DETR. In addition, the transformer-based methods provide attention map, which also can be considered as an interpretation map. Please provide the detailed comparison with those methods. 
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and easy to understand. In my opinion, the proposed method can be easily reproduced. However, I think the novelty is quite limited.
summary_of_the_review: This paper is well-written and well-motivated. However, I have major concerns as mentioned above. If the authors can address my concerns, I am open to increase my score.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This submission presents a framework for visual explanations of object detection models. The visual explanations are in the form of heatmaps that indicate the importance of each pixels for the prediction of an individual instance (classification score and bounding box location). Key differences to prior works are that the proposed method is instance-based (not category-based) and the heatmaps are computed gradient-based (not perturbation-based). The former makes the visual explanations more suitable for object detection, while the latter make the computation efficient. The technical contribution over gradient-based image-level visual explanation methods is that the "importance-weights" are computed for each individual pixel and activation (feature) map, while image-level methods only computed one weight per activation map. Based on the per-instance heatmaps, the paper proposes a training objective to make these heatmaps more compact, in a sense that the contributing pixels for each prediction do not overlap with other instances in the same image. Such heatmaps are then further used to improve non-maxima-suppression of standard object detectors by not only looking a intersection-over-union (IoU) between bounding boxes, but also at the correlation between the per-instance "visual explanation" heatmaps. Experiments are conducted on the COCO and CrowdHuman datasets.
strength_and_weaknesses: Strengths:
- Explainability is an important topic by itself. Methods tailed for object detection are certainly also in high demand given the wide applicability of object detection.
- The efficacy of the proposed gradient-based method over the existing perturbation-based D-RISE is good and important from a practical point of view.
- Novel non-maxima-suppression (NMS) methods that are more intelligent than looking at bounding box overlaps are certainly interesting. (Although more recent Transformer-based detectors like [A] do not require NMS anymore.) The intuition of leveraging pixels that contribute to the prediction for NMS is good; Figure 2(c) shows that well.
- Two different types of detection architectures are evaluated (one-stage, FCOS, and two-stage, Faster-RCNN)
- The quantitative evaluations in Section 4.2 are good and thorough I think.

Weaknesses:
- General comment/question regarding visual explanations and explainability
  - Shouldn't the analysis of such methods focus a lot more on erroneous predictions (both false positives and false negatives)? I assume a big part of explainability is to figure out why a model makes certain errors.
  - The qualitative inspection of heatmaps seems arbitrary and disconnected from actual "explainability". It seems the paper hints at more compact heatmaps being better at explaining the network, which feels a lot like confirmation bias. There may be regions away from the detected object that made the network predict it (e.g., contextual information).

- The motivation for Odam-Train and the Object Discrimination Index seem flawed
  - As mentioned above, the paper seems to hint at compactness of heatmaps being an indicator for a good visual explainability tool. But I do not quite understand why these two things are correlated. What if the existence of one instance of category A makes it more likely that another instance of the same category exists in the image? Another example: If the image captures the sun, wouldn't that influence the probability (confidence scores) of predicting the category "umbrella"?
  - Odam-Train essentially has the goal to make the heatmaps more compact. The loss functions make sense for that goal. But I do not understand how this makes ODAM a better explainability tool? Obviously, if we consider compactness of heatmaps as good for explaining a model, then optimizing for that will make it better. But does this really improve explainability of the model?
  - The same argument is true for the proposed Object Discrimination Index, as a quantitative measure for how good the visual explanation is. I'm wondering whether the motivation for this metric is valid.

- Odam-NMS:
  - The related paper [B] is not discussed or compared with.
  - I'm missing a quantitative comparison of Odam-NMS without using Odam-Train first. Based on the motivation of the paper, Odam-NMS without Odam-Train should perform worse.

- Qualitative results are insufficient:
  - Unfortunately, I believe the appendix or supplemental material is missing, which would include more qualitative results. I cannot find it, though. So the only qualitative results for ODAM are given in figures 1 and 3. Irrespective of the missing appendix, I would expect more visual examples in the main paper when the topic is about visual explanations.
  - As mentioned above, I think it would be important to showcase visual explanations for wrong detections.
  - The qualitative results in Section 4.1 for models trained with and without Odam-Train (Figure 4) are only a confirmation that the proposed loss is doing what it is designed for. The claim of better explainability is somewhat flawed because the loss just optimizes for the definition of the heatmap-based visual inspection tool.

- The paper mentions that ODAM is applicable to one-stage detectors like YOLO or FCOS. I am missing a discussion about recent Transformer-based (DETR-based [C]) methods like DINO [A]?

References:
- [A] DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection. Zhang et al. arXiv 2203.03605
- [B] Learning non-maximum suppression. Hosang et al. CVPR 2017
- [C] End-to-end object detection with Transformers. Carion et al. ECCV 2020
clarity,_quality,_novelty_and_reproducibility: - The description of Odam-NMS needs improvement. While the intuition is clear (also with Figure 2(c)), the text only describes the algorithm vaguely. Are there multiple thresholds for the two metrics (IoU and correlation of heatmaps)? Are the two metrics combined and then thresholded with a single value? Besides pointing to the appendix for pseudo code, this information is valuable in the main paper too.
- Figure 2 is only referred to in the text in Section 3.3 for some visual examples. It should be referred to more often throughout Section 3 to help understand the heatmaps, the training losses and the NMS.
summary_of_the_review: Overall, I'm missing justification and for several aspects of the paper (see "Weaknesses"); might be also caused by the missing appendix.

*Update 11/28*: After author response and availability of the appendix, many of my concerns have been addressed -- raising my score.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose an Object Detector Activation Map (ODAM) to visually interpret object detectors' predictions. ODAM follows the ideas from Grad-CAM, using gradients to produce instance-specific heat maps highlighting the important pixels. ODAM can not only work with class labels but also bounding box coordinates. Additionally, a training scheme OdamTrain and an NMS mechanism, Odam-NMS are proposed to improve object detection performance. 
strength_and_weaknesses: Strength
- The proposed ODAM sounds technically correct to provide instance-specific heat maps for explaining the prediction attributes of object detectors, which is beyond the existing class-wise activation maps. 
- The paper is easy to follow.					
- OdamTrain and Odam-NMS provide insight into how to use the visual explanation to boost detection performance. 	
 
Weakness
- My main concern is that a big advantage of class-wise attention is to achieve weakly-supervised object detection or segmentation, where only image-level annotation is required. However, ODAM needs an extra bounding box to generate instance-specific maps. So can ODAM work well on a task with weak supervision? Additionally, as the authors claim that ODAM could interpret any type of prediction, I am curious how well the ODAM can perform for segmentation tasks. 
- To evaluate the visual explanation quality, though the experiments look solid and extensive, I still suggest evaluating robustness, e.g., how does ODAM perform under adversarial attack?
- As the ODAM includes extra bounding box information, it seems unfair to compare ODAM with existing class-wise activation, e.g, Grad-CAM.
 - Additionally, I suggest authors include an ablation study to discuss the performance variance when ODAM is applied to different feature layers. 

clarity,_quality,_novelty_and_reproducibility: The ODAM is supposed to be easy to reproduce. The novelty is minor as it mainly follow the ideas of Grad-CAM.
summary_of_the_review: To my knowledge, it is the first work to give a visual explanation of object detection. Though robustness evaluation is not included, an evaluation of visual explanation is still solid. The  ODAMTrain and ODAM-NMS achieve performance marginally boosting, providing insight into how to utilize the visual explanation. Overall, I think the paper reaches the weak acceptance bar.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary:  
This papers a method of instance-specific visual explanation for object detection called ODAM. ODAM follows the idea of gradient-based visualization from Grad-CAM, while the key differences are that the "importance-weights" are computed for each individual pixel and feature map to generate instance-based heatmaps. Because the heatmap may “leaks” onto neighboring objects in crowded scenes, the authors further propose a method called ODAM-Train that adds loss functions to constraint the consistency and separations of the detected objects. They also proposed ODAM-NMS which uses the correlations between instance-level heat maps and the box IoUs to remove duplicate proposals of the same object. In the experiments, the proposed approach is evaluated on COCO and CrowdHuman datasets using various fidelity and accuracy metrics for explanation and shows good results compared to D-RISE.

Strengths:
1. This paper investigates an important yet relatively unexplored topic, visual explanation for object detection.
2. The proposed method is simple but intuitive and works. 
3. The experiments are thorough and show the effectiveness of the proposed method, using many detection models such as one- and twostage, and transformer-based detectors with different types of backbones and detector heads.
4. OdamTrain and Odam-NMS provide insight into how to use the visual explanation to boost detection performance.

Weaknesses:  
1. Technical novelty is not very big since the core idea is a relatively straightforward extension of Grad-CAM.
2. Experiments on Transformer-based models (also, comparison with self-attention-based explanation methods) can be elaborated more.

justification_for_why_not_higher_score: As described in detail below, the main value of this paper lies in the combination of a few simple (but solid) ideas and their evaluation, while the fundamental novelty is limitted. Therefore, the AC supposes that it is a bit short for spotlight or oral. 
justification_for_why_not_lower_score: This paper initially received the score of 5,5,6,6. Since the supplementary material was forgot by the authors to be uploaded, many details were unclear. The reviewer raised many concerns regarding its novelty, insufficient experiments and analysis, and unclear details of the method.
During the discussion period, the authors and the reviewers had very active discussion. The authors have presented many detailed explanations of the method/experiments and additional results as well as the improved paper and supplementary. Notably, novel experiments showing the effectiveness of ODAM on Transformer-based models have been presented, which was one of the main concerns in the beginning. Although not 100% satisfied, all reviewers consider that their concerns are generally addressed and raised their scores to 6,6,6,8. 
Meanwhile, the concern regarding the fundamental novelty may still remain. Although the AC agrees that the novelty of plain ODAM is not very big, the paper also provides two extension modules, OdamTrain and Odam-NMS, which are reasonable and not trivial. Also, they provide insight into how to use the visual explanation to boost detection performance, which is interesting and will stimulate not only the XAI community but also the broader audience in visual recognition. Overall, the reviewers are unanimous for acceptance, and the AC also conclude that the pros of the paper sufficiently overweight the cons, thus recommends acceptance.
Decision: Accept: poster