Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduces a way to create a concept bottleneck paper without having to provide the concept datasets (which is usually user-specified) (this can be seen as a bug or a feature in the community). The way they achieve this is to first use GPT-3 to query related concepts given an output class. Using these GPT-created concepts they then use CLIP embeddings to match the concerted to images in the training data and hence create their concept dataset without labels. Once this dataset is created they can then train sparse layers i.e. sparse matrices using elastic net regularization to obtain their final predictions.

They then show that their method is able to perform well on datasets like imagenet while still being interpretable.
strength_and_weaknesses: Strengths:
- I have not seen people utilize GPT and CLIP in this manner to construct concept datasets (not expert in this field though)
- The idea to create your label-free concept bottleneck paper is very interesting and a step forward to interpretable ML models
- the paper is mostly well and clearly written.
- Given that they use spare matrices they are also able to investigate the model on a sample level as well as edit the model accordingly. i.e. only changing a few weights in the sparse layers.
- This opens up a new avenue for interpretable Deep models.

Weakness:
- I might be wrong but there are some typos in the paper that make it hard for me to understand what steps 2 and 3 are doing. 
 - in 3.2 you have $W_c \in d \times N$ shouldn't that we M given figure 2? if f(x) \in R^d what is the dimension for f_c(x)? and therefore what is q_k? I am very confused with the notation.
 - I am pretty sure eq(1) cannot be right. the index of t only goes from 1-M and the index for q (I don't understand). Should this be M?
 - EXPERIMENTS: Please add the 2 standard deviations in the table of results and how many times the algorithm has been run (I might have missed this detail. please point me to the part in the paper)
- EXPERIMENTS: Could the authors please clarify what "Standard (sparse)" means?
- EXPERIMENTS: I would like to investigate the code and see if the authors have cherry-picked any of the examples in the paper. Could the authors please confirm that the examples were not cherry-picked?
clarity,_quality,_novelty_and_reproducibility: Overall very interesting new idea. I am not an expert using GPT or CLIP but if the idea of creating concept data is new then I consider this to be an interesting paper !

In terms of clarity, I fail to understand what is happening due to I believe notational issues which hopefully can be clarified during the rebuttal.

In terms of reproducibility, I would be curious to see if there has been any cherry-picking done to obtain the results. The authors seem to only show very few examples and hence I would like the authors to confirm that this in fact works in general
summary_of_the_review: I have mentioned all my concerns as well as thoughts on the paper in the review above.

I am more than happy to increase my score if the authors are able to answer the concerns above.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper presents a framework to transform a neural network model into an interpretable CBM model that does not require a domain expert to provide concepts and does not require labeled concept data. The authors leveraged GPT3 model by prompting it in a certain way to get concepts related to each output class. They later perform a set of heuristics to clean the concept list. In order to train a concept bottleneck layer and to learn a projected layer, authors exploited the CLIP-dissect. Finally, they introduced a sparse final layer in order to keep it more interpretable by using the ElasticNet regularization in the loss function. The experiments are conducted using 5 different datasets. The results showed that their method is able to maintain decent performance. Later in the paper, the authors perform preliminary experiments on changing the model's prediction by manipulating the neurons.
strength_and_weaknesses: Strength: the method provides a framework to build concept bottleneck models that do not require domain experts and labeled concept data.
Weakness: The evaluation is rather weak. The downsides or the limitations of using a model to create concepts are not discussed.
clarity,_quality,_novelty_and_reproducibility: I had issues understanding how CLIP-dissect is effectively used during training. 
summary_of_the_review: - The idea of using GPT3 is smart. I am wondering how the effectiveness of the proposed approach changes, with the change in the model, used to extract concepts.
- The concept filtering is a bit ad-hoc. e.g remove concepts we can't project accurately, remove concepts too similar to each other, concept length longer than 30 characters. I am wondering how much noisy were the initial concepts and how much careful cleaning is required.
- Authors mentioned that the number of concepts is related to the number of classes. Does it mean that it is good to have a one-to-one mapping between concepts and classes? Restricting the neurons to learn a single concept extracted using GPT3, would it not affect the generalization and robustness of the model? 
- The number of concepts mentioned in section 4.1. Are these the remaining concepts after cleaning or some further restriction is applied on the number of concept to consider based on the number of classes?
- The evaluation can be improved. The paper focuses a lot on qualitative evaluation. The main quantitative result is on maintaining the overall performance of the model. Authors may add a correlation between the selected concepts and the predicted class. They may also consider ablating concepts and present their effect on the output class.
- Figure 4, I am wondering if authors found false negatives and false positive cases. If a correct concept is ranked the lowest for a certain prediction, this raises the question of the faithfulness of the interpretation.

After rebuttal:
Thank you for the detailed reply and for making substantial improvements to the paper. 
- It would be great to add a point on how did you select the number of concepts for a dataset e.g. how did you come up with the number 128 for CIFAR10?
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Citing two limitations of Concept bottleneck model (CBM),  (1) the time consuming and labor intensive need to collect labeled data for each of the predefined concepts and (2) their accuracy s often significantly lower than that of a standard neural network, the authors propose Label-free CBM; a framework to transform any neural network into an interpretable CBM without labeled concept data, while retaining nearly the same accuracy as the original model.  They prompt CLIP based on templates filled in with task class names to generate an initial concept set (ie, label free) and then filter that down based on many heuristics.  They then learn a concept matrix ( the dotproduct of task image encodings with each concept text embedding ) and subsequent concept bottleneck layer of concept weights for a given image and then finally learn a sparse final layer using elastic net penalty and solved via GLM-SAGA ( from Wong et al 21 ).  They compare there results with CBMs and PostHoc CBMs (a similar-ish prior method).  They additionally show how to explain individual instances via concepts ( like CBMS ), how to generate global explainability rules from the concept weights and final layer weights and discuss model editing, namely the final sparse layer weights.
strength_and_weaknesses: **Strengths:**   
The use of CLIP to generate task specific concepts sets seems like a useful method and the use of CLIP-Dissect (CD) and the GLM-SAGA solver also seem novel and well motivated. 

**Weaknesses**  
The filtering methods felt like heuristics that should have included ablation studies to justify certain decisions ( concepts not being near classes for instances, and all the  thresholds that are included in section 3.1 B ).

The paper makes a few unsupported claims such as   
1) in general we found GPT-3 concepts to be of higher quality (than ConceptNet), but you never actually do a qualitative comparison or study in the paper to back that ( and a concept quality analysis is one of the major things missing from the paper).   Using a human curated ontology would seem to have some advantages over task specific concept sets ( namely using the same concepts over different data and tasks) though the paper shows accuracy wise they get better result, but this would have been made stronger had they included an experiment showing the results they would have obtained by using Concept Net to generate their concept set ( ie, how much is it the concepts vs the architecture which gives this paper a performance bump.  
2) criticizing the use of CLIP in Posthoc CBMs when its a substantial part of the architecture here  ( I honestly could be missing something, but it doesn’t seem like Post CBMs tie them to CLIP anymore than the Label Free model is? )  
3) In Table 2 you show Label-Free CBM  outperforms Post-hoc CBM in 3 of 5 and 2 of 5 tasks for the CLIP variant . You can show values for other datasets if you have them, but without that you can’t say your work clearly outperforms PCBMs “on all datasets”.  
4) Section 5 starts, “this is the first example of manually editing a large neural network with no obvious flaws”.  This is a vague assertation. What methods of manually editing are you referring to that have obvious flaws? PCBMs propose model editing in their paper so if you are commenting on their proposal or another papers proposal you should specify why they have obvious flaws.

The paper under explains important elements of the paper, in particular with regards to having human validation of the filtered concepts.
Ablations and some under explained areas include:  
1) How important are each of the filtering techniques really?  An ablation on these, in particular points 1 + 2, would be useful.  The number of cutoffs and thresholds makes this feel not very “automated”.   
2) What would have happened without the sparseness  constraint on the last layer in terms of accuracy?    
3)  How are the decision rules  in Fig 3 learned? Is this showing the average Wf layer for training examples of each class and then showing relative weight.  This vizs are nice, but you should explain this better ( maybe swap 4.4 and 4.3 in order and then add additional explanation )  
4) Its possible I’m missing something, but the algorithm in section 5.2 is quite vague and not terribly convincing because of it.  It seems like its really flipping model weights and hoping more examples are corrected than made incorrect by the new model and is not a systemic, principled way to do things.  

**Nitpicks:**
1) Why do you spend time introducing IBDs as interpretable CBMs only to not compare against them because you say they have non-interpretable components?    
2) I personally think its ok to have a residual layer to improve CBM performance ( PCBM-h ) and it’d be better to include those results with that caveat of the model not being fully interpretable   
3) For Fig 2: you should show  
- there are M concepts in concept set ( which is implied by numbers in Concept Matrix ) AND  
- f_c(x) = Wc * f(x) AND  
- the output of step 4 are classes and not concepts ( so add that each x has an associated y in R^p for instance ) 
   so the architecture its not misunderstood to be completely unsupervised.    
   you then need to write that W_f is M by p   
clarity,_quality,_novelty_and_reproducibility: The paper is mostly clearly written, though with some unsupported claims.  Prompting an LM for concepts followed by filtering is not entirely novel (Yejin Choi’s group has quite a bit of work in the area ), but the combination of the methods is novel.
Reproducing would probably not be possible since prompting a LLM isn’t entirely deterministic so its unclear I’d get the same concept set as the authors ( and they aren’t releasing code or the concepts they generate ) , but its likely I’d get approximately similar results ( though not guaranteed ).  

summary_of_the_review: The use of CLIP to generate task specific concepts sets seems like a useful method, though the filtering methods felt like heuristics that should have included ablation studies to justify certain decisions ( concepts not being near classes for instances, and all the  thresholds that are included in section 3.1 B.   The use of CLIP-Dissect (CD) and the GLM-SAGA solver also seem novel and well motivated though I’m not sure of what the exact difference between CD and what they propose in 3.2 is.   The paper makes many unsubstantiated claims, under explains important elements of the paper, in particular with regards to having human validation of the filtered concepts and I don’t find section 5.2  terribly convincing.  The paper is a useful contribution, but could be made much stronger by addressing some of the issues and performing certain ablations to justify decisions/claims made..
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper introduces a methodology to train concept bottleneck models, which automates the process of concept definition and it is empirically shown to outperform existing baselines making use of user-defined concepts. The methodology consists of four stages: 
In stage 1, concepts are defined by generating lists of words conditioned on information related to each class in the downstream task using GPT-3. Additionally, a set of heuristics is applied to filter out generated concepts and to produce a curated concept list which is used in the subsequent stages. In stage 2, CLIP embeddings for the image and the concepts are used to produce a similarity matrix. In stage 3, the concept bottleneck layer is trained to maximise a similarity score between the weights of the layer and the similarity matrix. Finally, In stage 4, a linear layer is learnt to solve the downstream task.

Experiments are conducted on 5 common vision benchmarks, including CIFAR10, CIFAR100, CUB200, Places365 and ImageNet, by comparing the proposed strategy against post-hoc concept bottleneck models [1] and a black-bock baseline. Additionally, the work proposes a new strategy intervening on the weights of the final prediction layer as demonstrated with a use case on ImageNet.

[1] Post-hoc Concept Bottleneck Models. ICLR 2022 PAIR2Struct Workshop
strength_and_weaknesses: **Strenghts**
- The paper is clear and easy to follow.
- The idea is simple, intuitive and original. I appreciated the idea of reusing existing technology and to apply it to concept bottleneck models.
- The visualisations in the experiments are compelling and informative.

**Weaknesses**
- The proposed strategy includes several heuristics and an ablation study is missing.
- It’s unclear what is the scope of application of the proposed strategy, as it seems to be limited to traditional vision benchmarks. For instance, what happens if classes are imbalanced or if data come from specific domains?
- The experimental evaluation (Section 4.2) is non-standard and the authors should consider the same experiments performed by post-hoc networks. Additionally, the authors should report the set of whole results in the original paper.
- From a computational point of view, the proposed strategy is quite complex.
- Code is not available. However, the authors have stated that they will release it prior to publication.

**Detailed comments and constructive feedbacks**
1. Several heuristics are used throughout the different stages of training. For instance, in stage 1 several rules are used to filter out the generated concepts.  How are these rules and hyperparameters affecting the final performance? Are these configurations data/task dependent?
2. At which stage is the main backbone trained?
3. The experimental analysis focuses on 5 traditional vision datasets, namely CIFAR10, CIFAR100, CUB200, Places365 and ImageNet. I can imagine that the concept generated by GPT-3 can transfer to these benchmarks and therefore help in the automation process. However, what does it happen when the downstream tasks include rare classes (e.g. iNaturalists) or it requires concepts which requires domain specific knowledge (e.g. medical data)?
4. Why aren’t the experiments on accuracy following the ones in the paper of post-hoc concept bottleneck models (P-CBM)? The author should start from the results in Table 1 of the paper of P-CBM, rather than running different experiments.
5. In Section 5.2, only 5 interventions are found to be useful. Are there any examples of interventions in Section 5 which hamper the predictive performance? And how hard is to find such beneficial interventions?
6. What are the limitations of the proposed methodology?

**Additional missing references**

[2] Concept Whitening for Interpretable Image Recognition. Nature Machine Intelligence 2020
[3] Concept Embedding Models. NeurIPS 2022
clarity,_quality,_novelty_and_reproducibility: **Clarity**

The paper is well-written, clear and easy to follow.

**Quality**

The scope of validity of the proposed solution is unclear to me. Additionally, an ablation study is required in order to gain insights on the impact of the different training stages on the final predictive performance. Finally the quantitative experiments on accuracy should follow the original work of P-CBM.

The qualitative analysis with the visualisations in Section 4.4 and the use case in Section 5 are original and informative.

**Novelty**

The problem of automating the process of concept definition is to my knowledge new and relevant. Also, the idea of reusing existing technology (i.e. GPT-3) for tackling such problem is novel and original. 

**Reproducibility**

Code is not available. However, the authors are willing to release it prior to publication.
summary_of_the_review: I recommend for an initial score of 5, as the weaknesses are currently outweighing the strengths.

----- POST REBUTTAL ------

I went through the rebuttal. The authors have addressed many of my concerns, (i) by including an ablation analysis on concept filtering, (ii) discussing the limitations of the proposed approach and (iii) providing substantial clarifications to all my questions.
The quality of the paper has consequently increased thanks to the ablation analysis. The scope of the proposed solution has been made clearer thanks to the discussion on the limitations.
Overall, I'm quite positive about the work. The idea is novel and original, pushing forward research on explainability for concept bottleneck models. The only drawback is related to the reproducibility. Certainly, building solutions around proprietary software (like GPT-3) is not a good practice towards open science. However, the authors plan to release their code, all generated concepts and also make the necessary steps to ensure that all experiments are reproducible.

Based on these considerations, I increase my score.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a new way to learn a "concept bottleneck model", which maps activations in the penultimate layer of a classifer to human-understandable concepts. Compared to prior methods for learning concept bottleneck models, the proposed method does not require hand-specifying the concepts and also retains strong accuracy even on large-scale tasks. The paper includes experiments on a few vision datasets, but also includes an interesting study on manually editing the classification layer in a model to improve its performance based on insights gained from the concept bottleneck model. While reviewers had various suggestions for improved clarity in the initial review, all reviewers ultimately recommended acceptance.
justification_for_why_not_higher_score: From my perspective this is an interesting interpretability paper - it seems to innovate over prior work and also includes a nice study of manual editing of a classifier. The specific focus (concept bottleneck models) is pretty niche and the method itself is pretty complicated, so I think it may have limited impact. But it's worth publishing.
justification_for_why_not_lower_score: All reviewers recommended acceptance.
Decision: Accept: poster