Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper summarizes and compares some of the current explanation techniques for deep neural networks that rely on the redistribution of relevance / contribution values from the output to the input space.

The main contributions are the introduction of a unified framework that expresses 4 common attribution techniques (Gradient * Input, Integrated Gradient, eps-LRP and DeepLIFT) in a similar way as modified gradient functions and the definition of a new evaluation measure ('sensitivity n') that generalizes the earlier defined properties of 'completeness' and 'summation to delta'.

The unified framework is very helpful since it points out equivalences between the methods and makes the implementation of eps-LRP and DeepLIFT substantially more easy on modern frameworks. However, as correctly stated by the authors some of the unification (e.g. relation between LRP and Gradient*Input) has been already mentioned in prior work.

Sensitivity-n as a measure tries to tackle the difficulty of estimating the importance of features that can be seen either separately or in combination. While the measure shows interesting trends towards a linear behaviour for simpler methods, it does not persuade me as a measure of how well the relevance attribution method mimics the decision making process and does not really point out substantial differences between the different methods. Furthermore, The authors could comment on the relation between sensitivity-n and region perturbation techniques (Samek et al., IEEE TNNLS, 2017). Sensitivtiy-n seems to be an extension of the region perturbation idea to me.

It would be interesting to see the relation between the "unified" gradient-based explanation methods and approaches (e.g. Saliency maps, alpha-beta LRP, Deep Taylor, Deconvolution Networks, Grad-CAM, Guided Backprop ...) which do not fit into the unification framework. It's good that the author mention these works, still it would be great to see more discussion on the advantages/disadvantages, because these methods may have some nice theoretically properties (see e.g. the discussion on gradient vs. decompositiion techniques in Montavon et al., Digital Signal Processing, 2017) which can not be incorporated into the unified framework.
review_title: Good paper, but discussion on methods which do not fit into the proposed framework could be extended.

final_score: rating: 7: Good paper, accept
review_content: The paper shows that several recently proposed interpretation techniques for neural network are performing similar processing and yield similar results. The authors show that these techniques can all be seen as a product of input activations and a modified gradient, where the local derivative of the activation function at each neuron is replaced by some fixed function.

A second part of the paper looks at whether explanations are global or local. The authors propose a metric called sensitivity-n for that purpose, and make some observations about the optimality of some interpretation techniques with respect to this metric in the linear case. The behavior of each explanation w.r.t. these properties is then tested on multiple DNN models tested on real-world datasets. Results further outline the resemblance between the compared methods.

In the appendix, the last step of the proof below Eq. 7 is unclear. As far as I can see, the variable g_i^LRP wasn’t defined, and the use of Eq. 5 to achieve this last could be better explained. There also seems to be some issues with the ordering i,j, where these indices alternatively describe the lower/higher layers, or the higher/lower layers.
review_title: Useful work showing the similarity between different neural network interpretation techniques

final_score: rating: 7: Good paper, accept
review_content: This paper discusses several gradient based attribution methods, which have been popular for the fast computation of saliency maps for interpreting deep neural networks. The paper provides several advances:
- \epsilon-LRP and DeepLIFT are formulated in a way that can be calculated using the same back-propagation as training.
- This gives a more unified way of understanding, and implementing the methods.
- The paper points out situations when the methods are equivalent
- The paper analyses the methods' sensitivity to identifying single and joint regions of sensitivity
- The paper proposes a new objective function to measure joint sensitivity

Overall, I believe this paper to be a useful contribution to the literature. It both solidifies understanding of existing methods and provides new insight into quantitate ways of analysing methods. Especially the latter will be appreciated.
review_title: Ties together attribution methods in a unifying and systematic way

Meta-review: With scores of 7-7-6  and the justification below the AC recommends acceptance.

One of the reviewers summarizes why this is a good paper as follows:

"This paper discusses several gradient based attribution methods, which have been popular for the fast computation of saliency maps for interpreting deep neural networks. The paper provides several advances:
- This gives a more unified way of understanding, and implementing the methods.
- The paper points out situations when the methods are equivalent
- The paper analyses the methods' sensitivity to identifying single and joint regions of sensitivity
- The paper proposes a new objective function to measure joint sensitivity"

Decision: Accept (Poster)