Towards Achieving Concept Completeness for Textual Concept Bottleneck Models (2025)

Abstract: This paper proposes Complete Textual Concept Bottleneck Model (CT-CBM), a novel TCBM generator building concept labels in a fully unsupervised manner using a small language model, eliminating both the need for predefined human labeled concepts and LLM annotations. CT-CBM iteratively targets and adds important and identifiable concepts in the bottleneck layer to create a complete concept basis. CT-CBM achieves striking results against competitors in terms of concept basis completeness and concept detection accuracy, offering a promising solution to reliably enhance interpretability of NLP classifiers.

## introduction the striking level of performance in natural language processing (nlp) achieved by blackbox neural language models @xcite @xcite comes along with a lack of interpretability @xcite . the field of explainable artificial intelligence (xai) @xcite intends to make the behavior of such models more interpretable. a common distinction of xai is to define interpretability methods either (1) by applying post hoc explanation methods to interpret black box models, or (2) by constructing interpretable models by-design @xcite . one promising approach in the second category is concept bottleneck models (cbm) @xcite . cbm are models that first map the input representations to a set of humaninterpretable high-level attributes, called concepts, in a concept bottleneck layer (cbl). these in this paper, we propose complete textual concept bottleneck model (ct-cbm), a novel approach to transform any fine-tuned nlp classifier into an interpretable-by-design tcbm accurately detecting concept from a complete concept basis. as summarized in this way, the ct-cbm method we propose offers both (1) an affordable concept annotation method to construct a concept bank and (2) a method to generate a tcbm derived from any initial concept bank, with an accurate cbl constructed upon a complete concept base. the paper is organized as follows: section 2 recalls some basic principles of xai and related work. section 3 describes the proposed ct-cbm. section 4 presents the conducted experiments, that show that ct-cbm systematically succeeds in reaching the performance of its competitors in terms of downstream task accuracy and detects significantly more precisely the concepts present in its concept layer, while containing fewer concepts than its competitors. ## background and related work this section first recalls some principles of xai methods used later in the papers and presents existing methods generating concept bottleneck model in general, and for nlp. ## proposed approach: ct-cbm this section describes our proposed complete textual concept bottleneck model (ct-cbm). as shown in ## experimental settings this section presents the experimental study conducted across 5 datasets and 3 nlp classifiers of different sizes. we compare ct-cbm to several competitors, and we run an ablation study to assess the impact of (1) the concept clustering during the cbl initialization, (2) the method used to compute concept importance and (3) the concept identifiability score. then we illustrate tcbm applications, namely concept intervention during inference, better understanding of counterfactual explanations and adversarial attacks @xcite and providing global explanations. ## conclusion we introduced ct-cbm, a novel approach to transform a fine-tuned nlp classifier into a textual concept bottleneck model. ct-cbm automatically generates, scores and targets concepts to build a complete concept bottleneck layer. ct-cbm replicates the same downstream classification performance than its competitors in normal and critical domains dataset, while generating a complete concept base with drastically less concepts, leading to significantly more accurate concept detection. moreover, we highlighted several advantages of tcbm, such as intervening on concepts to improve performance, increasing the intelligibility of adversarial attacks and counterfactuals and producing global explanations.