Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The scope of the paper is interesting i.e. taking a closer look at saliency methods in view of explaining deep learning neural networks. The authors state that saliency methods that do not satisfy an input invariance property can be misleading.

On the other hand the paper can be improved in my opinion in different aspects:
- it would be good to be more precise on the type of invariance (e.g. translation invariance, rotation invariance etc.) or is the paper only about invariance to mean shifts? I suggest to explain in the introduction which type of invariances have been considered in the area of deep learning and then position the paper relative to it.
- in the introduction the authors talk about an "invariance axiom": it was difficult to see where in the paper this axiom is precisely stated.
- While in section 2.1 specifies a 3-layer MLP as the considered deep learning network. It is not clear why CNN haven't been used here (especially because the examples are on MNIST images), while in section 3.1 this is mentioned. 
- I think that the conclusion with respect to invariances could also depend on the choice of the activation function. Therefore the authors should from the beginning make more clear to which class of deep learning networks the study and conclusions apply.
- From section 3.1 it becomes rather unclear which parts of the paper relate to the literature and which parts relate to section 2.1. Also the new findings or recommendations are not clear.






review_title: The (Un)reliability of saliency methods

final_score: rating: 4: Ok but not good enough - rejection
review_content: Saliency methods are effective tools for interpreting the computation performed by DNNs, but evaluating the quality of interpretations given by saliency methods are often largely heuristic. Previous work has tried to address this shortcoming by proposing that saliency methods should satisfy "implementation invariance", which says that models that compute the same function should be assigned the same interpretation. This paper builds on this work by proposing and studying "input invariance", a specific kind of implementation invariance between two DNNs that compute identical functions but where the input is preprocessed in different ways. Then, they examine whether a number of existing saliency methods satisfy this property.

The property of "implementation invariance" proposed in prior work seems poorly motivated, since the entire point of interpretations is that they should explain the computation performed by a specific network. Even if two DNNs compute the same function, they may do so using very different computations, in which case it seems natural that their interpretations should be different. Nevertheless, I can believe that the narrower property of input invariance should hold for saliency methods.

A much more important concern I have is that the proposed input invariance property is not well motivated. A standard preprocessing step for DNNs is to normalize the training data, for example, by subtracting the mean and dividing by the standard deviation. Similarly, for image data, pixel values are typically normalized to [0,1]. Assuming inputs are transformed in this way, the input invariance property (for mean shift) is always trivially satisfied. The paper does not justify why we should consider networks where the training data is not normalized is such a way.

Even if the input is not normalized, the failures they find in existing saliency methods are typically rather trivial. For example, for the gradient times input method, they are simply noting that the interpretation is translated by the gradient times the mean shift. The paper does not discuss why this shift matters. It is not at all clear to me that the quality of the interpretation is adversely affected by these shifts.

I believe the notion that saliency methods should be invariant to input transformations may be promising, but more interesting transformations must be considered -- as far as I can tell, the property of invariance to linear transformations to the input does not provide any interesting insight into the correctness of saliency methods.

review_title: Poorly motivated invariance property

final_score: rating: 4: Ok but not good enough - rejection
review_content: The authors explore how different methods of visualizing network decisions (saliency methods) react to mean shifts of the input data by comparing them on two networks that are build to compensate for this mean shift. With the emergence of more and more saliency methods, the authors contribute an interesting idea to a very important and relevant discussion.

However, I'm missing a more general and principled discussion. The question that the authors address is how different saliency methods react to transformations of the input data. Since the authors make sure that their two models compensate for these transformation, the difference in saliency can be only due to underlying assumptions about the input data made by the saliency methods and therefore the discussion boils down to which invariance properties are justified for which kind of input -- it is not by chance that the attribution methods that work are exactly those that extract statistics from the input data and therefore compensate for the input transformation: IG with black reference point and Pattern Attribution.
The mean shift explored by the authors assumes that there is no special point in the input space (especially that zero is not a special point).
However, since images usally are considered bounded by 0 and 1 (or 255), there are in fact two special points (as a side note, in Figure 2 left column, the two inputs look very different which might be due to the fact that it is not at all obvious how to visualize "image" input that does not adhere to the common image input structure).
Would the authors argue that scaling the input with a positive factor should also lead to invariant saliency methods?
What about scaling with a negative factor?
I would argue that if the input has a certain structure, then it should be allowed for the saliency method to make use of this structure.

Minor points:

Understanding the two models in section 3 is a bit hard since the main point (both networks share the weights and biases except for the bias of the first layer) is only said in 2.1

review_title: Interesting idea but missing good motiviation and dicussion

Meta-review: This paper showcases how saliency methods are brittle and cannot be trusted to obtain robust explanations. They define a property called input invariance that they claim all reliable explanation methods must possess. The reviewers have concerns regarding the motivation of this property in terms of why is it needed. This is not clear from the exposition. Moreover, even after having the opportunity to update the manuscript they seem to have not touched upon this issue other than providing a generic response.
Decision: Reject