Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a unified framework to explain the current model explaining algorithms. Under this framework, the authors discovered the current algorithms are approximating second-order Shapley-Taylor indices, where they also proposed a fast-kernel-based estimation method. Empirically, the authors benchmarked different algorithms in two large segmentation datasets. They showed the proposed algorithms achieved better faithfulness and inefficiency. 

main_review: This work has twofold contributions, which are both theoretic and empirical. The theoretic contribution is the unification of existing algorithms using Shapley-Taylor indices. The empirical contribution is their proposed fast-kernel algorithm, which has been validated through quantitative experiments. 

Few concerns: 

1. Readability. For the theoretic part, this work is built upon Shapley value. I would recommend authors to include more introduction for Shapley value (at least more intuition), so that readers do not need to read external references. 

2. Please re-order table 1 that mIoU should be the primary metrics. It is because, without good mIoU, faithfulness and inefficiency is not that meaningful metrics. 

3. Variance for Table 1? 

4. Wondering do people really care about the speed of explanation? 




summary_of_the_review: This work has solid contributions, and the writing and experiments presentation could be improved.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an approach for explaining visual similarity models.  They provide a general framework that enables them to adapt existing explanation methods to the similarity learning task, and then discuss how they apply their framework to generalize multiple methods.  The authors evaluate their work on PASVAL VOC and COCO, where they use automatic metrics to argue that their approach accurately captures model performance.
main_review: ***Strengths***

Providing a general framework grounded in some theory for adapting existing work to a new task ensures that their approach can likely continue to provide value over time (since as new classification explanation methods are developed they could adapt those too).  The paper is also pretty well written and easy to understand, and addresses a problem with wide applications and relatively little work, making the added discussion quite valuable on its own.  The experiments themselves do suggest their approach has merit, although lacking in several aspects in this version.

***Weaknesses***

1. The authors never showed their explanation method was human-interpretable or could be used in any downstream methods that leverage explanations.  This is a critical flaw in their paper that is enough to recommend rejection on its own.  The authors try to avoid this by saying people introduce biases when considering explanations so is out of scope, but I am going to flatly reject this line of reasoning.  If it is an explanation, then one factor should be human interpretability, since it is one of its primary applications.  I would be willing to accept an alternative, such as demonstrating that this kind of approach would be useful for methods that utilize explanations during training (the authors referenced a few, so I will omit citing examples here), but these experiments have yet to be conducted as well.

2. Note the authors failed to discuss and cite another paper that introduced a general framework for generalizing prior work for explaining similarity models:

Bryan A. Plummer, Mariya I. Vasileva, Vitali Petsiuk, Kate Saenko, David Forsyth. Why do These Match? Explaining the Behavior of Image Similarity Models. ECCV, 2020.

The authors should also discuss and compare to this paper.

3.  Following up on the paper in the second weakness, the authors of that paper found saliency-based methods (such as this paper) is not actually human interpretable for the image similarity setting, and could actually lead to more confusion when used in isolation.  While this could simply be due to the biases the authors argued as a reason not to do human studies, this should be confirmed as it suggests that the approach used by this paper may lead to similar results.  Some discussion by authors on this point is warranted.

4. The authors seem to mostly consider the case for explaining why two images are similar, but not what causes dissimilarity.  For example, some key image region A could be neglected by image B, producing a poor similarity score.  The authors should include a discussion and experiments that handle this case.

5. The authors perform experiments on two object detection benchmarks, which makes their claim of an image similarity explanation model a bit suspect.  They should also include experiments over standard deep metric learning benchmarks at the very least (e.g., Deepfashion, Stanford Online Products, CUB, etc), although I have a strong preference for medical images since this case was used repeatedly in the motivation for this paper.

***Post Rebuttal***
For the most part I was satisfied with the rebuttal.  The approach has not been demonstrated that it is human-interpretable, meaning that one of the major applications of explainable methods are not verified.  In addition, the authors did not compare to the prior work that I suggested.  While the authors argue that their explanations are better, and I don't necessarily doubt it, but demonstrating it empirically would make a stronger paper.  I would encourage the authors to at the very least discuss and (hopefully) compare to the paper I cited as it a published paper that addresses the same topic as this paper and currently is (still) uncited.

summary_of_the_review: My initial recommendation is to reject this paper.  The authors have not demonstrated that their approach provides useful explanations either via a human evaluation or for some downstream task.  They are missing a reference to at least one work addressing the same task that has claims that are problematic for this paper, and the experiments were not conducted on datasets that were designed for the motivating applications as well, making it unclear if the approach will generalize to the target setting.  The authors need address at least the first two weaknesses before I would consider recommending acceptance, although the 5th weakness is also high on my priority list for changes.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a explanation framework for retrieval/similarity/metric learning models. It generalizes the form of a couple of previous works, e.g. Zhou et al., 2016; Selvaraju et al., 2017; Zhu et al., 2019; Ribeiro et al., 2016. The proposed method relies on estimation of Shapley value, the authors then propose a kernel-based approximator to make it more computationally efficient. Experiments are conducted on PascalVOC and MSCoCo dataset to show the faithfulness and quality of explanations.
main_review: Strengths:
+ This work builds up a formal explanation framework for retrieval/metric learning, which can benefit future works in this field. It also generalizes other explanation methods, like CAM, GradCAM, VEDML (generalized as SAM in this paper). 
+ The idea of using Shapley value to explain retrieval model is very interesting.
+ The definitions and propositions are reasonable, I did not find any errors there.
+ The experiments provide a good comparison between all existing and generalized methods on faithfulness, quality as well as the efficiency.
+ The writing is easy to follow and well organized. 

Concerns:
- The claim of "Axiomatic" is very strong. Shapley value may be a fair metric to measure the contribution of each feature, but the value in this work is estimated with approximator. Is the estimated value guaranteed to be "Axiomatic" here?
- There are many principles to follow for explanation, and Shapley value is not the only reasonable one. For example, perturbation [1], and Deep Taylor Decomposition principle [2]. Why is Shapley value called "Axiomatic" while others not? 
1. Fong, Ruth C., and Andrea Vedaldi. "Interpretable explanations of black boxes by meaningful perturbation." Proceedings of the IEEE international conference on computer vision. 2017.
2. Chefer, Hila, Shir Gur, and Lior Wolf. "Transformer interpretability beyond attention visualization." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.

- The performance is not improved. In table 1, it seems that the proposed kernel-SAM only improves efficiency over SAM, the quality and faithfulness are both on par with second-order SAM. Since the experiment is conduct with GAP architecture, the SAM is equivalent to VEDML (in related work), it this correct? If yes, then the performance is not improved over VEDML. In scenarios where efficiency is not important, we still don't know how to get a better explanation.
- The proposed method is claimed to have advantage over other methods when the architecture is not based on GAP. But the experiment is only conducted with GAP-based architecture. Why not try other architectures where the proposed method could have better performance?
- The explanation framework is for retrieval/metric learning/similarity models, while this paper only conducts experiments on object detection datasets (VOC, COCO). It would be more convincing if results on widely used retrieval/metric learning datasets are provided, e.g. CUB, Deepfashion.
- The authors use Moco, a contrastive learning model for self-supervised learning. It uses augmented images as positive and all the other images as negative, which is different from normal retrieval model. The results on retrieval model may be different. Why not train with metric learning setting, i.e. using the same class as positive while other classes as negative (contrastive loss, triplet loss, proxy). 
-  Proposition 4.2 raises a limitation about GradCAM. It seems that VEDML also mentioned the limitation of GradCAM about the GAP operation. Although their definition is not as formal as this work, it somehow aligns with the observation of this work.

summary_of_the_review: Overall, I think the idea is very interesting and the proposed explanation method is good contribution. But the experiment is not strong due to the concerns, and the evaluation could be improved in the future. My rating is borderline leaning to accept.
review_title: null

Meta-review: The initial reviews for this paper were 6,6,6, the authors have provided a rebuttal and after the rebuttal the recommendation stayed the same. The reviewers have reached the consensus that the paper is borderline but they have all recommended keeping it above the acceptance threshold. Following the recommendation of the reviewers, the meta reviewer recommends acceptance.
Decision: Accept (Poster)