Kati8/ICLR_2023_2025 huggingface dataset:
authors: Michal Moshkovitz
authors: Zhili Feng
authors: Dotan Di Castro
authors: J Zico Kolter
reviews: 
Comments: In this paper, authors propose a method for concept explanations that satisfy three axioms. Such explanation is verified to provide insights into model selection, optimizer selection, and model improvement.
Strengths: + Concept explanation is an important direction in explainable AI research.
+ It is interesting to connect the proposed method with existing methods including TCAV.
Weaknesses: - It seems that the proposed method is only applicable to datasets with pre-defined concept annotations c(x).
- Though authors show some connections between the proposed method and two existing methods, there still lacks baseline comparisons in the experiment part. For example, why not compare the proposed method with existing methods (such as [cite 1,2]) for model improvement.
- As typical concept explanations usually provide visualizations of extracted concepts. It will help a lot if authors also provide such visualizations, and compare their method with existing concept explanations from this perspective.


[cite 1] This Looks Like That: Deep Learning for Interpretable Image Recognition (NeurIPS 2019)

[cite 2] Concept-level Debugging of Part-Prototype Networks (ICLR 2023)
Recommendation: 3: reject, not good enough
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 2 fair
Clarity: 2 fair
reviews: 
Comments: Their objective is to design a model-agnostic concept explainer; Existing concept explanation methods assume the model-to-be-explained has a representation layer limiting their application to DNNs.
They begin with a presentation of three axioms, which must hold for any concept explainer, and draw connections to previous estimators.
The paper claims that their framework enables efficient implementation and helps interpret concept scores estimated by existing estimators. 

The paper does bring some structure to the problem but I have strong concerns regarding faithfulness of their overly-simplified estimator and reservations about their compute efficiency claims.
Strengths: - The paper unifies and reinterprets two popular concept explanation techniques (Kim et.al. and Yeh et.al.), which is neat.
- The motivation for a model-agnostic explainer is a valid concern. Concept explanations are biased towards DNNs.
Weaknesses: - **Presentation issues**. The notation is confusing. Concept is denoted with c(x) and is used to denote either set of concepts (i.e. a concept vector) or a scalar score interchangeably causing much confusion, see (1), (2). Yeh et.al. 2020 should be explained in more details in Sec 4.1. h is overloaded to map from example or representations to the label. (2) uses y to enumerate over concept values, which can be confusing.
- **Faithfulness is not established**. (This is less a weakness and more a show-stopper). The premise of an explanation is its faithfulness, which surprisingly is not one of the axioms. All their estimators at best measure the correlation in activation of concept and the model-to-be-explained. They need to establish the faithfulness of their estimator with the evaluation suite of TCAV or Yeh et.al. to first establish theirs as an explanation. Imagine a model trained on a dataset with two co-occurring concepts but the model may only exploit one of the concepts for prediction (say because one concept is way easier than the other), however none of their estimators can bring out the asymmetric concept importance.
- **Unconvincing evaluation**. The paper evaluates on a gamble of tasks. The prediction tasks considered are to predict *cat*, *chair*, *motorbike*, *bottle* in Figure 2, 3, 4 respectively. Some rationale on choice of tasks is expected, and some evaluation using established benchmarks is also expected. 
- **Computational efficiency not presented.** They claim computational efficiency but do not present any empirical evidence supporting the claim.
- **Sufficiency vs necessary based score**. The paper discusses concept scores based on sufficiency or necessity, but do not discuss their practical implications. When is one better than the other?
Recommendation: 3: reject, not good enough
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 2 fair
Clarity: 2 fair
reviews: 
Comments: This paper addresses the problem of obtaining model-agnostic and human-understandable concept explanations of deep networks, specifically in the context of image classification. It first defines a set of general axioms that a concept explanation should follow, and shows that some of the existing approaches to concept explanations (e.g. TCAV) can be viewed as a part of this framework. It then defines measures that satisfy the axioms and explores their use for model selection (for models that use appropriate features), model optimizer selection (SGD vs AdamW), and prompt editing (to improve models).
Strengths: 1. The paper addresses the important problem of obtaining human-understandable faithful explanations of the decisions of deep networks, which is crucial for trusting model decisions.
1. It proposes a systematic, model-agnostic approach using a set of axioms that a concept explanation should satisfy. It shows that some of the prior work on concept explanations can be placed within this general framework.
Weaknesses: 1. The experimental evaluation appears to be limited. The analysis in Section 5.2 is limited to a small set of classes, and it is unclear how generally the trends observed hold. The evaluation for model selection also uses very simple models (linear regression and random forest). For broader applicability, it would be useful to evaluate whether this generalizes to deep networks, and whether one can select between different networks and network architectures with this approach.
1. The evaluations in Sectionn 5.2 appear to focus on images that were predicted to be a given class. For different models and different optimizers, this set of images could be different. Given that the ground truth labels of the images do not seem to have been taken into account, it is unclear if the presence of "spurious" concepts observed is due to the model relying on spurious correlations for images where the ground truth class is the same as the predicted class being considered, or due to the model simply misclassifying images of other classes as the predicted class being considered. This would be important to properly evaluate the proposed approach.
1. Given the efficiency and equivalence arguments between the proposed approach and prior work (completeness-aware and TCAV, Section 4), it would be helpful to have an experimental comparison between these approaches in the application settings in Section 5, both in terms of performance and computational efficiency.
1. The formulation of TCAV as an instantiation of the proposed framework seems to only apply to explanations at the last layer, while the TCAV formulation as originally proposed is more general. If this is the case, then this should be made clear.
Recommendation: 3: reject, not good enough
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 2 fair
Clarity: 2 fair