Kati8/ICLR_2023_2025 huggingface dataset:
authors: Yan Cui
authors: Shuhong Liu
authors: Liuzhuozheng Li
authors: Zhiyuan Yuan
reviews: 
Comments: The authors propose to build an image representation as follows:
- build a set of (textual) concepts with GPT4
- learn a projection that aligns the output of a backbone with the similarity matrix between the CLIP representation of an image and the CLIP encodings of the concepts.
- train a VAE to reduce this representation
- use the reduce representation for some clustering tasks
- interpret the compressed concepts with Crabbe & van der Schaar (2022)
Strengths: Representation learning is a topic that remains relevant within the computer vision community.
Weaknesses: It seems to me that the main weakness of this paper lies in a form of misunderstanding. The authors pretend that their work is an unsupervised method (representation learning), and the whole motivation of the paper comes from this, , all the way down to the clustering task used to validate the method.

The reality is in fact quite different, in that the core of the method is based on CLIP, which has been learned from a gigantic database of annotated (captioned) images. CLIP's zero-shot classification performance is very good, as numerous papers have shown. The CLIP representation is likely to contain precise definitions of the database labels used to evaluate the proposed method. Not surprisingly, a representation derived from CLIP is good at doing clustering. The problem is that evaluating a method on a clustering task when the method uses labels is not fair at all.
Recommendation: 3: reject, not good enough
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 1 poor
Clarity: 2 fair
reviews: 
Comments: The paper presents a method for concept-based image representation, where the set of concepts is obtained by using GPT-4. As far as I understand, the concept extractor is trained by distilling CLIP through aligning the linear projection of an input image to the concept space with the similarities between the input image and textual representation of the concept. The dimensionality of the concept vectors (an input image after the linear projection) is then reduced using VAE. The experimental results provide some comparison.
Strengths: (1) The idea may be interesting but I don’t understand the method and I’m not very sure about this.
Weaknesses: (1) I cannot understand what the method is doing. For example, I don’t see what the superscript “3” in Eq. (1) means. Does it mean cubic? Why it’s necessary? $\mathcal{Q}$, $\mathcal{H}$, $\mathcal{A}(\mathbb{R}^\mathcal{Q})$, and $\mathbb{R}$ are not defined (at least the paper is not self-contained). All these missing details make it almost impossible to see what’s going on in the method.

(2) My major question is the reason why the backbone and the projection layer are necessary. If $P_{i,:}$ is the target when training the projection layer and $q_i$ is expected to be closer to $P_{i,:}$, one can use $P_{i,:}$ as a concept vector. 

(3) If I understand correctly, VAE compresses the concept vectors to a latent vector. I don’t see why this is necessary. What is the typical number of concepts for each dataset (are they like 182 or 1401 as in Table 10)? As all the information provided in the latent vector is in the concept vector, I’m not sure applying dimensionality reduction really benefits some aspects of the method. An ablation study may help understand.

(4) I cannot get what is evaluated in the experiment section. I think the paper should evaluate how well the method can find the designated concepts in the concept vectors. The method has two concept representations (concept vectors and concept importance), so their consistency should be evaluated. I understand that this is not straightforward, as there are no annotations on the concepts, but I think evaluation over a small subset is helpful. Also, the reference set for each class should be evaluated in some way (I’m sorry, I didn’t come up with a good way except for human evaluation) to show the validity of the approach. I also think the representation should be evaluated for downstream tasks, like classification.
Recommendation: 3: reject, not good enough
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 1 poor
Clarity: 1 poor
reviews: 
Comments: This paper proposes a method to generate concept-based explanations for image classifiers. The method uses GPT and CLIP to identify a set of nameable visual concepts, then projects the image embeddings from the classifier into the CLIP concept space to get a concept-based representation of the images. This representation is further simplified by using an autoencoder to project the rather large set of concepts into a smaller set that represents only the most important concepts. The method is evaluated as an unsupervised learning method and produces clustering results comparable to or better than state-of-the-art on ImageNet, CIFAR, and STL-10 datasets.
Strengths: The method produces impressive clustering results on ImageNet, CIFAR, and STL-10 datasets (comparable to or above state of the art).

Using CLIP to find nameable concepts for XAI is a good idea, and the paper demonstrates how this makes it easier to access and interact with the concepts (e.g., find more images from another class that contain the same concept as a given class).
Weaknesses: The writing is frequently unclear, which makes many parts of the paper hard to understand.

The evaluation is limited to everyday object/scene datasets (ImageNet, CIFAR, STL-10) which are the datasets where this approach should work best due to the high overlap with CLIP’s training set. It would be nice to see evaluation on a broader range of datasets.

There's no user experiment, so it's unclear if these explanations would be useful for humans or how they compare to other XAI approaches.

The explanations don't seem to be entirely correct, given the examples in Figure 3. It seems like the model just lists concepts that are related to the predicted class, regardless of whether they are present in the image (e.g., “lion-like mane” for a female lion, or “sun lotion” for an image mistaken for a swimsuit photo, or “rotational movement” for an image containing a ball).
Recommendation: 3: reject, not good enough
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 1 poor