Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: SUMMARY

The authors propose an intuitive new measure (definition 2.1) of feature importance based on a robustness criterion (with two variants of equations 3 and 4). Two optimisers are proposed for finding the most important features according to this measure, in order to explain why a classifier is making a certain prediction.

The experiments are both qualitative (showing pixel-wise importance maps for image problems) and quantitative (showing how well the various feature importance scoring algorithms do albeit on the measure explicitly optimised by the new proposed algorithms).

COMMENTS

The paper organisation is clear enough though the English needs a little work to make it read really nicely.

The proposed measures are natural and intuitive. Especially the Robustness-$\hat{S_r}$ is an interesting twist on the more obvious Robustness-$S_r$.

While the measures are interesting, the justification for them is somewhat weak. This amounts to 

1) Quantitative experiments which seem to only test how well each method works in relation to the very metric which only your proposed method directly optimises - this is nice but not surprising. Also, the way you define the AUC of your measure seems a little strange. I don't know why different baselines appear in different comparisons as in e.g. tables 1 and 2. Finally your curves in appendix A don't seem to have the same number of points for each method in all cases? 

2) Qualitative examples with images and text. These are nice, but alas only qualitative. Also, it seems as though not all baselines are included in all examples (even figures 3 and 5, which are analogous, include different baselines). 

It seems like the paper needs either 1) a quantitative evaluation that is not subjective. Surely, the evaluation metric should not match the (novel) objective of the proposed method? Or, 2) a theoretical result in support of the new measures.

The Reg-Greedy algorithm is a major contribution of this paper, but receives very little explanation. Indeed, perhaps the clearest quantitative statement of the paper is that Reg-Greedy beats Greedy. Is this a common method for optimising w.r.t. a subset? Is it similar to other methods? I felt that Reg-Greedy is a really nice idea but the paper did not do it justice.

DETAILS

Perhaps unifying (3) and (4) by defining a single g that subsumes both cases would be neater.

Please define g in equation (1) rather than in words after equation (4).

It should be S_r in the subscripts of (3) and (4)

"Crutial" spelling

FINALLY

I'm open to be swayed on any of the above points, pending the author feedback.

review_title: Official Blind Review #4

final_score: rating: 6: Weak Accept
review_content: 

The manuscript proposes a method for model explanation and two metrics for the evaluation of methods for model explanation based on robustness analysis. More specifically, two complementary, yet very related, criteria are proposed: i) robustness to perturbations on irrelevant features and ii) robustness to perturbations in relevant features. Moreover, different from existing works which defined the perturbation values following different somewhat-fixed procedures, the proposed method aims at allowing perturbations in any directions.

A greedy algorithm optimizing these criteria is proposed in order to produce a method able to highlight important features of the input and justify/explain model predictions.
In addition, the proposed robustness criteria are used as metrics to assess the performance of methods for model explanation.

Experiments on models addressing image classification and text classification, shows the performance of the proposed method w.r.t. existing work.

The manuscript has a good flow, and its content is easy to follow. The proposed method is sound, well motivated, and very well founded. The formal presentation of the proposed method is good. I appreciate the fact that evaluation covers different modalities of data, i.e. text and images.

My main criticism over the manuscript is the following:

In Sec. 3.3, when describing the first criterion, it is stated that the size |S_r|, i.e, the amount of anchors, could be defined by the used. In my opinion, this may not be applicable since in theory the amount of relevant/irrelevant features is unknown before hand. In that case the proposed AUC-based method seems more adequate. Could you comment on this?


In  Sec. 3, a pre-defined size K is introduced. Later in Sec. 3.1 it is stated that the greedy algorithm uses this size as a stopping criterion for the optimization of the proposed robustness criteria. Could you indicate how this size K is defined in practice? Is there a principled way to define it? What is the effect of this parameter on the performance of the proposed method? An ablation study focused on this parameter would provide further insights into the inner workings of the proposed method and would improve the manuscript.


In Sec.4 it is stated that only 50 random examples are considered when reporting results. When comparing the performance across the different methods, are these random 50 examples fixed or always re-sampled? Also, given the size of the considered datasets, where the number of images in their test sets is in the order of the thousands, it is hard to grasp how representative are the reported results?

In the same paragraph discussed above, it is mentioned that the GRAD method performs competitively on the proposed criteria. It might be interesting to further positioning the proposed method w.r.t. GRAD. Given the comparable performance achieved by GRAD and its relative simplicity, it would be hard to motivate why not choose GRAD instead of the proposed method? Could you provide some discussion on this?

In Sec.4 (pag. 6) it is stated the the proposed regression-greedy method outperforms other methods in these criteria. In my opinion this trend shouldn't be surprising given the fact that the proposed method is specifically optimized on such criteria as it is clearly stated by the title of Sec.3.

Fig.3 and Fig.4 display binary images indicating the top-n features selected by different methods. Perhaps it would be more informative to have a heatmap highlighting/grading the entire input space. This may throw more light on the performance of the compared methods.

In Adebayo et al., NIPS'18 (and very related efforts), there are presented a set of sanity checks to be applied to explanation methods to ensure their predictions are relate to the class and model being predicted. Could you provide any indication on whether the proposed method passes these checks?

In Sec.4 (visualization) it is stated that the proposed method effectively highlights crucial positive pixels as well as pertinent negative pixels. A similar capability has also being reported earlier in Samek et al., Trans NNLS'16 and Oramas et al. ICLR'19. Since the visualization analysis (discussed in pag.6) focuses exclusively on this capability. There should be a comparison between the proposed method and the two mentioned works.


review_title: Official Blind Review #1

Meta-review: The paper proposes an approach for finding an explainable subset of features by choosing features that simultaneously are: most important for the prediction task, and robust against adversarial perturbation. The paper provides quantitative and qualitative evidence that the proposed method works.

The paper had two reviews (both borderline), and the while the authors responded enthusiastically, the reviewers did not further engage during the discussion period.

The paper has a promising idea, but the presentation and execution in its current form have been found to be not convincing by the reviewers. Unfortunately, the submission as it stands is not yet suitable for ICLR.
Decision: Reject