Concept Bottleneck Large Language Models
Authors: Chung-En Sun, Tuomas Oikarinen, Tsui-Wei Weng


Reviewer paper summaries: The paper proposes a way to implement adding a concept bottleneck to large language models for both text classification and text generation. The concepts are generated by an LLM. For text classification, the concepts are aligned with examples by using sentence embedding similarities. The final classification is taken by adding a linear layer over the concept scores. In the case of text generation, the paper proposes concatenating the concept layer to the final layer of an LLM before the token prediction to have a steerable generation. The experiments on several text classification tasks show the proposed method can achieve comparable performance as black-box models.
Summary of strengths: The approach in this paper improves upon the scalability and efficiency of previous work, requiring fewer resources to label concepts. It also extends the idea of concept bottlenecks for language models to cover both classification and generation.
Summary of weaknesses: 1. **Interpretability of the generation case.** For the generation model, the input to the linear layer is a combination of concept scores and black-box features. There is a chance that the linear layer will mainly rely on the black-box signals rather than concept scores, thus hurting the interpretability (a similar issue as the Post-hoc CBM [1] model).

2. **Evaluation.** There are a lot of issues in the evaluations. Here are some of them:
* *Lack of baselines.* There is only one baseline for each task: the fine-tuned LM. However, this is an unfair comparison as the proposed model has additional supervision from the ChatGPT for concept annotations. Fine-tuning LM on those annotated example-concept pairs is a reasonable baseline in this case. In addition, compared with zero-shot ChatGPT and existing CBMs, the evaluation could also be made more solid. For the human evaluation, it is compared to a random baseline, which is a low bar, making the results less convincing.

* *Evaluation metrics on generation.* It is unclear how accuracy is defined in the generation tasks. It seems models are saturated on this metric; on almost all tasks, models are close to 100%, making me skeptical about the usefulness of this metric. In addition, the CB-LLM has a much higher Perplexity than the black-box baselines.

* *Lack of ablations and missing technical details.* Eq (5) contains four terms while only one is ablated. Many hyperparameters are not specified, and unsure how they are decided, such as $\alpha$ in regularization, $\lambda$ in Eq 4, 5, etc. This makes the reproduction of the proposed method difficult.

3. **Presentation.** There is a lot of room to improve the presentation of this paper. All the figures are not carefully designed and are below the quality of ACL papers. Cramming too many results in the main paper without clearly organizing them. All tables are overly squeezed. The method section doesn't read like a method, more like some random implementation details with some unnecessary equations that can be thrown into an appendix. You need to have a problem formulation section to abstract your method instead of *step1, 2, ...," which sounds like a pipeline.

4. **Missing References** It is hard to believe this paper only has 15 references, which is unreasonably small. The authors did not carefully review the background literature and missed a line of work on interpretable methods, especially some recent work on LLM-based CBMs [1, 2, 3, 4]. The authors should rewrite the related work section to credit prior works.

[1] What does a platypus look like? Generating customized prompts for zero-shot image classification. ICCV 2023.
[2] Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification. CVPR 2023.
[3] Visual Classification via Description from Large Language Models. ICLR 2023.
[4] Learning concise and descriptive attributes for visual recognition. ICCV 2023.
Comments, suggestions and typos: Mainly see the weaknesses. Here are two major parts for improving the paper:
1. Improve the presentations.

2. Make the evaluation solid.
Overall assessment scores: 2.5
Metareview score: 2
Metareview: This paper presents the Concept Bottleneck Large Language Model (CB-LLM), a novel approach to enhance the interpretability of large language models (LLMs). Unlike traditional black-box LLMs, CB-LLM integrates a concept bottleneck layer that allows for inherent interpretability while maintaining high performance in both text classification and text generation tasks. Experiments demonstrate its effectiveness in providing explanations and facilitating human interaction with LLMs.

Reviewers acknowledge the proposed method's contribution to enhancing the interpretability of LLMs and its applicability to both classification and generation tasks. However, they have concerns regarding the clarity and soundness of the methodology, the lack of rigorous evaluation, missing related works, and the overall quality of the writing. While the authors have provided additional experiments, clarifications, and explanations during the discussion period, the paper may need another round of significant revision.
Summary of reasons to publish: -	CB-LLM improves model transparency by integrating a concept bottleneck layer, enabling built-in interpretability that is absent in traditional LLMs.
-	This paper not only focuses on text classification but also extends the concept bottleneck approach to text generation.
-	CB-LLM achieves performance comparable to black-box models while providing interpretability.
-	CB-LLM shows promise for enhancing interaction between humans and LLMs.