Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a method for post-hoc explanations of unsupervised methods. At the heart of this proposed idea is the *contrastive corpus similarity* which employs a corpus set of samples (e.g. abnormal cases in an illustrative task of classifying bone abnormality) and a foil set of samples (e.g. normal cases for the same task). The proposed method uses the notion of contrastive corpus similarity to generate Contrastive Corpus Attributions (COCOA). COCOA works by identifying the features that, when removed, make the representation of an explicand (a sample to be explained) similar to the foil set and dissimilar to the corpus set (hence the term `contrastive' in the name). 

The authors evaluate COCOA to establish whether the identified features are indeed related to corpus and foil samples using different models (SimCLR, SimSiam etc.), datasets, and feature attribution methods. They also demonstrate the utility towards understanding image data augmentation and mixed modality object localization. 
strength_and_weaknesses: ### Strengths

1. Writing and notational discipline: The idea is clearly described, and the notations introduced are well-explained. I cannot find any errors in the notations. 

2. Simplicity: The idea of using contrastive similarity towards a foil set and away from a corpus set is simple and elegant. 

3. Extensive experiments: The experiments are comprehensive, or as extensive as can be expected within one paper. The sanity checking in the Appendix is also appreciated. 

### Weaknesses

1. Conceptual Concerns and situating within the broader concerns surrounding XAI (Explainable AI)

The paper early-on cites the paper on "The Mythos of Model Interpretability" by Lipton to effectively state that deep learning methods can be black boxes. However, I believe it fails to contextualize the proposed method within the larger discourse on XAI methods established in that paper. Specifically, the use of attribution maps (or saliency maps or heat maps as alternatively known) has been shown to suffer from several issues. As Barredo Arrieta et al. point out, “there is absolutely no consistency behind what is known as saliency maps, salient masks, heatmaps, neuron activations, attribution, and other approaches alike” [R1, Sec. 5.3]. Also see [R2, R3] for a discussion on the issues with saliency/attribution maps. 

The experiments still uses these and other SHAP based methods known to have issues to support their argument. (See [R4] for links to other papers discussing how SHAP can produce inconsistent and unstable explanations). 

I'd like to hear the authors' thoughts on why, broadly, their empirical argument which is built upon these methods will not suffer from similar issues. I don't see in the experiments how the proposed method would implicitly be robust to these issues surrounding instability since the method is inherently dependent on having a well-separated concept of foil and corpus samples (or at least that these samples need to already have a human understandable semantic meaning which is subject to biases in the human perception). 

One of the core issues of the current state of XAI is how the methods largely depend on biases and interpretation of the observer, and it would be good to establish how this method is not adding to the same issue. 


[R1] Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. - Barredo Arrieta et al. 
[R2]  Sanity checks for saliency maps. - Adebayo et al. 
[R3] Exploratory Not Explanatory: Counterfactual Analysis of Saliency Maps for Deep Reinforcement Learning - Atrey et al. 
[R4] The Disagreement Problem in Explainable Machine Learning: A Practitioner’s Perspective - Krishna et al. 
clarity,_quality,_novelty_and_reproducibility: 1. Clarity: The paper is clear, the notations clean and well-explained.
2. Quality: The quality of the paper is good. The proofs are provided, the experiments are comprehensive.
3. Novelty: The notion of adding a contrastive notion to the representation space (as opposed to methods that only see if the representation changes, but not away or towards other clusters) is interesting. It builds upon and extends these previous ideas, but I don't consider this to be a bad thing. 
4. Reproducibility: The code in the Supplementary is useful in making this work reproducible, although I have not had time to run the code. 
summary_of_the_review: The proposed idea of generating explanations for unsupervised methods is simple and elegant. Incorporating the notion of contrastive changes to the representation (towards a foil cluster and away from a corpus cluster) in response to changes in the feature space is interesting. The writing is clear and demonstrates notational discipline. 

However, my issues with this paper is that it fails in contextualizing how the proposed idea and empirical evaluation do not suffer from well-established issues with the underlying attribution methods used in buildding the empirical evidence in support of the proposed method. (Especially because the paper seems aware of the discourse in the "The Mythos of Model Interpretability" paper). I have added other references to help the authors address this issue. Nevertheless, this prevents me from recommending a higher pre-rebuttal score.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This study indicates contrastive document similarity as a scalar explanation output for unsupervised methods. A corpora of samples collected and foil are used in the process (negative samples or distribution). It calculates the average cosine similarity between an input's encoded representation and those in the corpus, subtracting the average similarity with foil representations, for the given input. The suggested method can find pertinent input features when used in conjunction with current feature attribution methods.
strength_and_weaknesses: Strength
1. The approach is simple, and appears to work well, as demonstrated by the excellent results.
2. The figures hopefully demonstrate how the method might be employed and what sort of impacts it could have.

Limitation
1. More often than not, the author to labeled the data to determine what the corpus and foil are.
2. If the feature representations of the dataset are heterogeneous in nature (high commonality with some corpus examples and low similarity with others), the proposed model might start giving a scoring rate that is excessively limited.
clarity,_quality,_novelty_and_reproducibility: The paper has been mostly straightforward to understand, although some of the equations are really quite complex and difficult and could be challenging to figure out speedily. According to what I understand, the research is innovative, and yet I may not be intimately acquainted with all of the related literature works. There should README file in the supplementary material that could easy for readers to follows.
summary_of_the_review: The article proposes a simple, efficacious, and potentially useful method for generating attribute inferences for unsupervised models that really are different from each other.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes contrastive corpus similarity as a scalar explanation output for unsupervised models. The method relies on a corpus (positive samples) and foil (negative samples or distribution). For a given input, it computes the average cosine similarity between its encoded representation and those of the corpus and subtracts the average similarity with the foil representations. Combined with existing feature attribution methods, the proposed approach can identify relevant input features.
strength_and_weaknesses: Strengths

The approach is quite simple, yet effective as demonstrated by the strong results.

The figures clearly help demonstrate the potential impact and applications of the approach.

Weaknesses

To identify the corpus and foil, it is sometimes necessary to have some labeled data.

It is unclear why the results do not include the performance of RELAX.

If the feature representations of the corpus are multimodal (i.e. you get high similarity with some corpus examples, but low similarity with others), the proposed method might arguably assign too low of a score.
clarity,_quality,_novelty_and_reproducibility: The authors appear to have been rigorous, and I didn't check all the proofs in the appendix. The paper is mostly clear, although some of the formulae are quite complex and (arguably) hard to parse quickly. To my knowledge, the work is novel, although I may be unfamiliar with some of the existing literature. The supplementary material includes code, but the documentation (e.g. the README file) is lacking.
summary_of_the_review: The paper proposes a simple, effective and potentially impactful method to generate contrastive feature attributions for unsupervised models.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Strengths (supported by all reviewers, myself included):

* clarity
* simple and elegant
* effective
* extensive experiments

Weaknesses

* Method depends on one's ability to assess similarity. This was mentioned by two reviewers, and I agree this is a limitation, but it doesn't seem to be a reason to keep the paper out of ICLR. I recommend the authors to discuss the limitations on the manuscript, as they do in the rebuttal. 
* Method requires some labelled data. Also mentioned by two reviewers. I am choosing to see this as a limitation (i.e., it limits applicability) but not a weakness. 
* The limitations of saliency maps should be better discussed with a more transparent contextualisation of the method in this landscape. I agree with the reviewer who brought this up and also think it's an important point for discussion in the manuscript. I appreciate that the authors conducted some additional analysis in the rebuttal phase.

I'd like to recommend acceptance, while highly encouraging the authors to be a bit more transparent about the last issue above, and clearly discussing all three points above in the manuscript.
justification_for_why_not_higher_score: Had the limitations I listed above been discussed fully and openly in the submitted version, I think we would have pushed the score up.
justification_for_why_not_lower_score: The idea is simple yet elegant, the paper is executed well and the method is shown to be effective. The limitations require some open discussion in the text but are not necessarily motivating much lower scores.
Decision: Accept: poster