Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The author focuses on exploring the XAI methods that none of the methods discussed, highlighting specific parts in the classification to enhance the explanation. The author proposes a method, Critical Classification Regions (CCRs), to do this. CCRs use a nearest-neighbor example to highlight similar important parts in the image explanation. The author performed a user study on a subset of the Imagenet to show improvement of the CCR method.


main_review: Strengths:

The main reason to accept this paper is empirical results, showing performance on the variants of the proposed method.  The author has evaluated performance on a standard dataset. The problem statement is exciting but not the solution.

Weaknesses: 

1. The author has proposed the critical classification regions (CCR) method, but it is not discussed in the paper. How does the proposed method help to find critical regions? Does the region help in explanation? 

2. The core idea is not Novel. Nearest-neighbor classification[1,2] already appears as an explanation for visual object recognition. Nearest-neighbor-based explanations are already presented in the literature. So the main contribution is not novel and seems to be very limited. 
In the past, Patro et al. have proposed an attention-based nearest neighbor technique to highlight specific parts of the image and try to improve attention and classification score. 

3. What would be the motivation for the Nearest-neighbor example for the proposed task? Can it highlight specific parts in the classification to enhance the explanation? What is its requirement? Is this the only method to solve the task? The author should explain this.

4. The core idea is not Novel. Nearest-neighbor classification[1,2] already appears as an explanation for visual object recognition. Nearest-neighbor-based explanations are already presented in the literature. So the main contribution is not novel and seems to be very limited.

5. What would be the motivation for the Nearest-neighbor example for the proposed task? Can it highlight specific parts in the classification to enhance the explanation? What is its requirement? Is this the only method to solve the task? The author should explain this.

6. Selection NN for test samples: This part is not discussed with a detailed analysis. The method section is not clearly discussed. The author should rewrite these sections.

7. How does Speeding-up Twin-Systems help improve explanation? If not, please remove that from the method section; it creates confusion.

8. What happens when the method gets a different pattern/object/class in the nearest neighbor? Is there any way to tackle this issue? Have you tried multiple nearest neighbors like K=1,2,3,4…

9. Patro et al.[3] have tried different values of K and used a clustering technique to select the optimal nearest neighbor. They also tried contrastive techniques using positive examples, negative examples to anchor to improve the critical regions. 

10. The proposed method has not been compared with the latest explanation methods. The author should provide both quantitative and qualitative comparison results with the latest explanation methods and also visualize the proposed explanation method with the latest methods. 

11. The author should explain figure-6 and its attributes; it is not clear from the caption. What do you mean by those plots and the significance of those bars? 

12. CUB and Imagenet datasets are mainly focused on single instances. What happens to your algorithm for multiple instance data like MS-COCO images.


13. However, the paper misses one of the core aspects of machine learning practice: readability and reproducibility of results. The author should provide an algorithm or pseudocode to reproduce the results, missing in this paper.

Ref:
[1] Chen, George H., and Devavrat Shah. "Explaining the Success of Nearest Neighbor Methods in Prediction." Foundations and Trends® in Machine Learning 10, no. 5-6 (2018): 337-588.

[2] Ma, Wei, Kendall Nowocin, Niraj Marathe, and George H. Chen. "An interpretable produce price forecasting system for small and marginal farmers in India using collaborative filtering and adaptive nearest neighbors." In Proceedings of the Tenth International Conference on Information and Communication Technologies and Development, pp. 1-11. 2019.

[3] Patro, Badri, and Vinay P. Namboodiri. "Differential attention for visual question answering." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7680-7688. 2018.


summary_of_the_review: Overall, I do not feel like this paper is above the bar for acceptance because the core idea is not terribly exciting and has a lot of technical issues. 
The author should rewrite the paper and provide all other details mention in the weakness section.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors proposes an approach to improve explanation-by-example techniques, by picking nearest neighbors (NNs)  based on fine-grained image content that are critical for the classification decisions. Four variations to select these NNs are explored and compared against each other.
The main advantage claimed is the causal role of the generated explanation. The authors support this claim via quantitative evaluation and human evaluation.

main_review: I appreciate the wide array of experiments conducted by the author. The manuscript reflects the thought process the authors had while developing their ideas, touching multiple times at the hypotheses they had and the rational behind their algorithmic choices and experiment design.

I like the fact that the authors algorithm help explore different explanations based on different CCRs (Figure 10 and Figure 11).
Does the algorithm support finding NNs based on multiple CCRs at once, so that two images can have a high match if they share multiple CCRs? If this is the case, I strongly encourage the authors to provide examples, because such ability helps make their method more generic: Many classification decisions are based on multiple cues in the input image not a single CCR.

I would be helpful if the authors can describe how their solution can be used in practice. In particular, do the users have to pick alpha and beta? The authors provide some analysis on ImageNet and CUB-200, highlighting the compromise between higher values and lower values, and concluding that 1.25 is the optimal value for ImageNet while 1.1 is the optimal value for CUB-200. I do not fully understand how the authors came to these values and whether the users of their algorithms will be able to make informed decisions about these values (especially also due to the computational overhead of scanning a range of values).

I miss a comparison with influence functions (Koh and Liang, ICML'17). The authors mentioned that such comparison would be very computationally expensive, however, there are two algorithms that allow computing influential instances more efficiently, described in:
- Estimating Training Data Influence by Tracing Gradient Descent (Garima et al., NeurIPS'20)
- Representer Point Selection for Explaining Deep Neural Networks (Yeh et al, NeurIPS'18)
I also miss a comparison with counter-factual methods (e.g. Goyal et al., ICML'19).

Language / formatting issues:
- hyperparamter
- were another => where
- the only method that (1) use […] and (2) do => (1) uses […] and (2) does not
- misclassifcation
- Section 3 determines […] => end with a full stop.
- was tested on CIFAR-10/ImageNet => on CIFAR-10 and on ImageNet
- is the regions found are contingent => is that
- it’s effect
- [similar to Hooker et al. (2019)] => avoid text within citations.
- [Kenny and Keane 2021]  add page number instead of tbd.

Wording Suggestions:
- Critical Classification Regions => Classification-Critical Regions?
- seriously explored / considered => directly? exntesively studied? [seriously might imply unserious efforts].
- Twin-Systems, Twin Systems, twin-systems, twin-system => use a consistent term, e.g. Twin-Systems Framework (what about an abbreviation?).
- we found four candidate => ... we identified four candidate ...
summary_of_the_review: The manuscript is heavy on analysis and seems to reflect work-in-progress results rather than solid and generalizable findings.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The manuscript proposes a method to extend explanation-by-example (aka. exemplar-based explanations) by highlighting the regions that links the test image with example(s) provided as part of the explanation. Towards this goal, different methods to compute Critical Classification Regions (CCR) are proposed where image regions/pixels linking the input with the explanation examples are highlighted.

The proposed method is validated on the CUB-200 and ImageNet dataset. This is complemented with a user-study.

main_review: - Overall the proposed idea is clear, I find its simplicity a good strength of the proposed method.
- I find the user-study presented in Section 5 adequate and with a proper level of depth.
- In addition, it seems that a code library related with the proposed method will be released after publication of the manuscript. This is always welcome from the reproducibility point of view.

My concerns with the manuscript are the following:

- The novelty of proposed proposed CCRs is reduced, and somewhat incremental. Moreover, it is only limited to explanation-by-example methods.

- To a good extent, the idea behind CCRs of highlighting the regions that link input images with training examples is similar to that of prototype-based explanations, especially the work from [Chen et al., 2019]. In this regard, proper positioning with respect to that family of methods would strengthen the manuscript.

- It is not clear to me what the value on the y-axis of Figure 2 (A & B) refers to. Is that the activation value of the logit related to the predicted class or the ground-truth class? how do you handle the cases when the prediction changes over the evaluation of different segments within the same image?

- Perhaps I missed something, but in Section 4, it seems that 30 versions of the original model are fine-tuned by gradually considering (adding) the superpixels of each image in the order of importance predicted by the explanation method. (Each time a segment is added, the model is fine-tuned. From this procedure, it seems that the evaluation is still more related towards assessing the relevance order in which the regions of the image are ranked by the explanation methods and not really the effect that specific training examples may have on performance.
- Also, I was wondering if these networks are not trained from scratch, then there is no guarantee that information present in the held-out segments is actually ignored since the model may still contain features related to then.

- Also, in Section 4, the manuscript explicitly links observations made in the experiment conducted therein with causality. Given the fact that the tested model is continuously modified on that experiment (none of the 30 fine-tuned resulting models is the same as the original model), I do not see how analyzing these different models can tell us something regarding causal properties of the original model.

- Proper validation of a method is not a contribution per se but a requirement in place for a method to be accepted/adopted by the community. There do not seem to be any novel protocols proposed in this regard, if that is indeed the  case, this should not be claimed as a novelty.

- Text and legends present in the plots from Figure 2 is not legible. I had difficulties reading it in digital version of the document. Please ensure the text have sufficient size to be readable when printed in standard A4 paper.

References
- Chen et al., This Looks Like That: Deep Learning for Interpretable Image Recognition
NeurIPS 2019.
summary_of_the_review: While there is novelty in the proposed method,I find it quite limited. Moreover, at this point, I have several doubts regarding the quantitative evaluation, i.e. Sections 3 and 4, of the proposed method (please see my review). Perhaps it is a problem of clarity, but is something to be addressed if the manuscript is to be published as a solid piece of scientific work.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a local model-specific post-hoc explanation method for image classifiers (CNN) returning as explanations examples suggesting the reasons for the classification together with subparts of the images named critical regions common to the test instance and the examples and responsible for the classification.
main_review: The paper presents a nice advancement of the state-of-the-art in XAI. The proposed method is partially incremental as it is based on the composition of existing approaches for the various steps, and their composition happens in a rather simple way. However, I believe that it brings innovation to the field as it seems effective from the evaluation. The weak points are listed in the following. First, the twin-system that is at the basis of the proposal is not properly presented and illustrated. It is impossible to fully understand the paper without reading the papers presenting the twin-system. Second, Experiment 1 is biased because the way adopted to test the importance of the parts of the image, i.e., by image occlusion, is exactly the same used by SP-CCRS. Thus this method has a clear advantage over the others. The authors should adopt a validation fair for both proposals or test them into separate settings. Third, the paper lacks for comparison with a method like the one of Been Kim or Cynthia Rudin. This statement is true for Experiments 1, 2 but also for Experiments 3. In this last case, in fact, I suppose that the examples are extracted with the twin-system. What about examples retrieved by the proposals of Been Kim or Cynthia Rudin? Also, a comparison against counterfactual explanations (see Yash Goyal), would have provided a better picture. Finally, the very interesting case study, unfortunately, does not bring the expected results. I suppose the way the questions are proposed to the users impacted the results as the explanation provided with CCR should be better than those having only examples. I suggest the authors repeat the experiments focusing only on images correctly classified.

Minor issues. I suggest avoiding using k for both the number of classes and the k of the NN algorithm.
The number k used in the experiments is not clearly stated as well as which are the different versions of kNN tested.
summary_of_the_review: A very nice idea probably a bit incremental.
An experimentation not sound in some aspects.
review_title: null

Meta-review: The paper proposes a method to improve explanation-by-example by identifying important parts of the image when using nearest engihbor explanations-by-example. Towards this goal, the notion of Critical Classification Regions (CCR) is proposed. The method is tested both computationally and a user study.

The reviewers felt that the paper had interesting ideas, but overall the reviewers agreed that the paper needs more work before being ready for publication: this includes improving the soundness of the empirical evaluations and clarifying the contribution of the paper.
Decision: Reject