Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: I Summary

The authors present a method that computes a saliency map after each scale block of a CNN and combines them according to the weights of the prior layers in a final saliency map. The paper gives two main contributions: SMOE, which captures the informativeness of the corresponding layers of the scale block and LOVI, a heatmap based on HSV, giving information of the region of interest according to the corresponding scale blocks. The method is interesting as it does not require multiple backward passes such as other gradient-based method and thus prove to be more time-efficient.

II Comments
1. Content
Overall the paper is very interesting, but it is not always clear what the contributions are.The authors refer to "scale" in CNNs, this could be described a little as to explain what are scale blocks in a CNN, before introducing the terminology.
The proposed method "LOVI" is promising and I like the use of HSV for the different components. However, it is hard to read, especially when alpha-blended with a grayscale version of the original image.
- The introduction doesn't clearly state what are the contributions of the method, an example of possible applications would be appreciated (the following about robotic for example but with more details)
- 3.1 I really enjoy the fact that different datasets are used for their different properties (foreground, background, sparsity), this is a nice touch
Figure 4 results are a little underwhelming, SMOE's scores are very close to the other methods
- 3.2 The authors refer to Smoothgrad squared method, it is indeed a good process to refine saliency maps, however why it is used could be detailed, just as the parameters chosen for its implementation.
- 4. The authors claim their implementation is "approximately x times faster" but there is no quantitative proof of it, which seems to be one of the selling-point of the paper (or at least one of the best results)

2. Writing
Those typos do not impact the review score, I hope it can help the authors to gain more clarity in their writing.
- Abstract: "it is also quantitatively similar or better in accuracy" -> shouldn't it be "and" instead of "or"?
- Intro
"a gradient saliency map by trying to back-propagate a signal from one end of the network and project it onto the image plane" not well articulated, "by trying to back-propagate" -> "by back-propagating" (the claims seems weak otherwise)
"is running a fishing expedition post hoc" ;)
"An XAI tool which is too expensive will slow down training" Computationally expensive?
- 2.1
"The resemblance to conditional entropy should be apparent" -> is apparent 
"we might say it is" -> it is (don't weaken your claims)
"we simple apply" -> we simply
- 3.1
"if the results appeared terrible" -> terrible is too strong/not adapted. What is a bad result and why?
after eq 7 " the second method is in information" -> an information
 -3.2
"which locations are most salient correctly" -> word missing?
- Figure 5: "Higher accuracy values are better results for it" -> yield better results

The phrasing with"one" as in "if one would like to etc" is used a lot through the paper, it can be a little redundant at times.

III Conclusion
The paper is interesting, however, one of the major contributions seems to be the speed of the method but no quantitative results have been reported. I would really appreciate seeing some experiments over it. Overall the results of the obtained maps are not very convincing compared to existing methods. I believe the writing of the paper could be wrapped around the speed of the method and in which context it would be important (robotic, medical?).  The conclusion and discussion are short and could be filled a little more (some captions could be shortened in order to give more space).

Edit: The authors have answered most of my concerns and I am happy to re-evaluate my score to weak accept.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The paper presents a new approach, SMOE scale, to extract saliency maps from a neural network. The approach is deemed as efficient, because it does not require one or multiple backward passes, as opposed to other approaches based on gradients. The main idea is to process the output activation tensors of a few selected layers in a deep network using an operator combining mean and std.dev. called SMOE scale. The result of this operator can be combined through different scales to obtain a global saliency map, or visualized in such a way that shows the consistency of saliency maps at different scales. Experiments show some improvement against traditional gradient-based approaches.

This paper is interesting in that it provides a different way to extract saliency maps from a network. The visualization at multiple scales is also nice and while I do not perfectly agree with the HSV encoding in Fig.2, I do see the potential. Being efficient is also a nice to have. There are three issues, however, which limits the novelty of the paper. First, the SMOE metric does not seem to bring much improvement compared to simple metrics. Second, the few comparisons made against other methods do not reveal a significant improvement. Third, at core, the paper suggests that the "high efficiency" of this approach is one of its main advantages, a statement I do not forcibly agree with. More details follow.

For the first element, we have to consider the paper as the combination of two things. 1) the use of activation maps as source of salient information, and 2) the way we should process these activation maps. 1) is relatively straightforward, so the core of the contribution should lie in 2). However, while the SMOE scale method definition (eq.2) is sound, it does not bring valuable improvement compared to other "trivial" metrics, like standard deviation. For instance, Fig.4 caption tells that "SMOE Scale differentiates itself the most early on in the network", but it is actually only for the very first scale layer. At every other scale, standard deviation (for instance) is at least as good. Same thing can be said about Table 4 in appendix, and also about Table 2 (and the scores of Trunc. Normal Entropy). Overall, while SMOE is indeed novel, it is not highly convincing.

On a side note about the SMOE description, I did not find the list of "conditions and assumptions" at the beginning of Sec. 2.1. It looks more like an after-thought over which the proposed method coincidentally fits. Moreover, point 3 is kind of conflicting in its formulation.

For the second element, the improvements in KAR and ROAR scores are quite minimal. It does seem to have an edge on KAR score, but not by a huge amount. Additionally, the methods compared are relatively old. To give just two examples of missing new techniques, Smooth Grad-CAM++ (Omeiza et al.) or even Grad-CAM++ (Chattopadhay et al.) would presumably obtain better performances. Moreover, Smooth Grad-CAM++ allows to target a particular feature map or even a specific neuron, which makes it even more relevant to this work.

Finally, a note about efficiency. Generally speaking, I agree that it is always good to be more efficient. However, I fail to see the high importance given to efficiency for this particular problem. Sure, gradient-based approaches are probably not suitable to online, in-network applications, but is it an important requisite? Computing saliency maps on a subset of the dataset once a few epochs already gives a good idea of what the network is doing. In any case, since these saliancy maps are intended for human use, I am not convinced about the importance of computing them for each training example at each epoch. Overall, in my opinion, being efficient at generating saliency maps is a nice to have, but not much more.

Some general comments:
- In Sec. 2.2, the last "con" seems a bit out of place. This could be applied to pretty much anything.
- Sec. 2.3 is interesting, but the explanations are convoluted. In essence, what should be said is 1) value encompasses the importance of a pixel, 2) saturation presents the max-min of the distribution and 3) hue shows the position in the network. Also, superimposing these HSV maps over gray scale version of the image like is Fig.2 is difficult to analyze because the "gray" of the image can be confused with the saturation channel. 
- On p.2, "this is proceeded" -> "this is preceded"?

In summary, this paper presents a nice way of generating saliency maps from activations inside a network. However, the comparison to other approaches does not show a clear advantage, and the related work (and experiments) lacks recent techniques. I would thus ask this general question: what is the "selling point" of this method? The current focus on efficiency does not convince me. That being said, there are no clear flaws in the paper, so I am open to reconsider my assessment if improvements are made to the paper (better descriptions, comparison with more recent techniques, experimental justification for SMOE instead of simpler approaches, etc.).

On a final note, there is also the Score-CAM approach (Wang et al.) that looks similar (in the idea of using activation maps). I did not consider it in this review since it was published a few weeks ago on Arxiv, but it could be interesting to discuss it nevertheless.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper presents a method for creating saliency maps reflecting what a network deems important while also proposing an interesting method for visualizing this. The central premise for the method of characterizing relative importance of information represented by the network is based on an information theoretic measure. A variety of results are presented to examine the impact of keeping or removing information deemed important by this measure and a comparison is made to existing approaches as a justification for the proposed methods.
I find that the proposed method appears to be theoretically sound and is interesting in revealing differences to other information theoretic methods especially in the early layers. The relative similarity in later layers is also an interesting observation and one that is relatively hard to characterize. 
Perhaps the strongest case for the proposed method comes from the keep accuracy subject to SMOE scale. The removal accuracy is a bit less convincing but appears to be a fair and honest evaluation. Moreover, the LOVI scheme for visualization is interesting in itself and I found this aspect of the work to be thought provoking with respect to how such methods are examined from a qualitative perspective.
review_title: Official Blind Review #3

Meta-review: The paper presents an efficient approach to computer saliency measures by exploiting saliency map order equivalence (SMOE), and visualization of individual layer contribution by a layer ordered visualization of information. 

The authors did a good job at addressing most issues raised in the reviews. In the end, two major concerns remained not fully addressed: one is the motivation of efficiency, and the other is how much better SMOE is compared with existing statistics. I think these two issue also determines how significance the work is. 

After discussion, we agree that while the revised draft pans out to be a much more improved one, the work itself is nothing groundbreaking. Given many other excellent papers on related topics, the paper cannot make the cut for ICLR. 
Decision: Reject