Adaptive Concept Discovery for Interpretable Few-Shot Text Classification
Authors: ZHENG Lifang
Hanmo Liu
Kani Chen
Venue: ICLR 2026 Poster (2026)
Keywords: concept bottleneck models; few-shot text classification;
Primary area: interpretability and explainable AI
TL;DR: We propose a novel CBM paradigm for few-shot text classification. Our method significantly outperforms prior CBMs and is competitive with LLMs using only 10 samples, while offering superior interpretability and efficiency.

Abstract: Few-shot text classification is a critical real-world task for which Large Language Models (LLMs) have shown great promise. However, their high inference costs and lack of interpretability limit their practical use. While Concept Bottleneck Models (CBMs) offer an efficient and interpretable alternative, their reliance on training surrogate models makes them incompatible with few-shot scenarios. 
To bridge this gap, we introduce a novel CBM paradigm that relies solely on sample-concept similarity to make predictions.
We ensure the effectiveness of our concepts through a prototypical-discriminative dual-level architecture and a dynamic concept refinement mechanism. Extensive experiments show that with as few as 10 training samples, our method surpasses prior CBMs and even achieves performance comparable to LLMs.
The code is available at \url{https://github.com/alexiszlf/StructCBM}.

Published as a conference paper at ICLR 2026
ADAPTIVE CONCEPT DISCOVERY FOR INTERPRETABLE FEW-SHOT TEXT CLASSIFICATION
Lifang ZHENG, Hanmo LIU, Kani CHEN"
The Hong Kong University of Science and Technology
(Izhengah, h1 i 1llub è connect. usk makchen&ust.ce
ABSTRACT
Few-shot text classification is a critical real-world task for which Large Lan- guage Models (LLMs) have shown great promise. However, their high infer- ence costs and lack of interpretability limit their practical use. While Concept Bottleneck Models (CBMs) offer an efficient and interpretable alternative, their reliance on training surrogate models makes them incompatible with few-shot scenarios. To bridge this gap, we introduce novel CBM paradigm that relies solely on sample-concept similarity to make predictions. We ensure the effec- tiveness of our concepts through tecture and a dynamic concept refinement mechanism. Extensive experiments show that with few as 10 training samples, method surpasses prior CBMs and even achieves performance comparable to LLMs. The code available at
mrolotpical-discriinaat dual-level archi-
INTRODUCTION
While Large Language Models (LL.Ms) have set new performance standards for few-shot and zero- shot text classification (Brown et al. 2020 Wei al. 2022, OpenAl 2023 Gao et al.) 2202 Min iet al. 2022). their practical deployment is constrained by sigmificant trade-offs. On one hand, their substantial computational requirements present a major barrier to scalable, cost-effective inference on large datasets (Zhou et al. 2024). On the other hand, the "black-box" nature of their reasoning process raises critical concerns about trustworthiness, particularly in regulated fields such as finance, healthcare, and legal services, where decision transparency is non-negotiable (Bender et al., 2021 Doshi-Velez Kim 2017).
To address the challenges of high inference cost and lack of interpretability in LLMss Concept Bottleneck Models (CBMs) (Koh et al.) (020) offer a promising alternative. As lightweight and inherently interpretable "white-box" models, CBMs ground their predictions in set of human- understandable concepts, thereby providing both efficiency and transparency. Recognizing this po- tential, some pioneering studies have explored using the extensive world knowledge of LLMs to help CBMs automatically extract these concepts (Luyten & van der Schaar 2024 Ludan et al,
2024
Feng et al., 2024 Sun et al. 2025).
However, the application of CBMs, particularly in few-shot scenarios, introduces its own set of challenges. While theoretical work suggests that CBMs are well-suited for few-shot learning by improving sample efficiency (Luyten & van der Schaar 2024), practical implementations face sig- nificant hurdles. Specifically, the limited few-shot data are insufficient train reliable prediction layer for mapping concepts to labels. Furthermore, existing LLM-augmented CBMs face a critical trade-off: iterative approaches like TBM and BC-LLM (Ludan et al., [0224] Feng et al. 2024) remain dependent on costly LLMs during inference, while models designed for efficient inference, such as CB-LLMS (Sun et al. 2025), require large datasets for training and are thus ill-suited for few-shot learning. A final limitation is that these models often frame the text-concept similarity as super- vised learning objective, precluding the possibility of iteratively refining the concepts themselves.
To address this challenge, introduce StructCBM, novel framework centered around proooypiical-disrimiaatve concept architecture. We first employ a large language model to gen-
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Table 1: Summary on current text concept bottleneck models.
Method
Few-shot
Inference w/o LLM Classifier Free
TBM (Ludan et al.
2024
No
No
No
M (Tan
2024b)
No
Yes
No
SparseCBM Tan al. 2024a
No
Yes
CB-LLM el al. 2025
No
Yes
No
CT-CBM Bhan et al 2025
No
Yes
No
StruciCBM (Ours)
Yes
Yes
Yes
crate two distinct sets of concepts from small number of samples: general prototypical concepts to identify candidate set of labels, and fineggraieed discriminative concepts to distinguish between them. The prediction is then made via a lightweight, two-stage process that relies on efficient text embedding similarity, removing the need for the LLM during inference. To ensure the quality of our concepts, the framework is designed as a closed-loop system where misclassifications are fed back to the LLM, which then performs semantic and logical refinements iteratively tune the concept library. This preeratt-peedie-reffiie workflow allows our model to remain highly efficient and in- terpretable while effectively handling the constraints of few-shot learning environment. Our main contributions are summarized as follows:
We are the first to apply LLM-augmented concept bottleneck models to few-shot learning, estab- lishing novel direction that leverages the synergy between the interpretability of CBM and the few-shot reasoning ability of LLM.
We propose dual-level, prototypical prototypicllddiccrmmnniit discriminative concept architecture that handles few-shot learning directly through sample-conces matching. We further design a senerate-predicc-reffe loop to guarantee the concept efffcctiveness.
We demonstrate through extensive experiments that our method achieves performance comparable to black-box baselines and surpasses prior LLM-based CBMs on multiple benchmarks.
2 RELATED WORK
Few-shot Text Classification. Few-shot text classification aims to build gegg-performance clas- sifiers using only small number of labeled samples. Mainstream approaches can be broadly di- vided into three categories. The first involves fine-tuning pretrained language models (PLMs) al. 2022). The second category consists prompt-based learning methods, which leverage task- specific templates to elicit the prior knowledge of language models. The third is the in-context learning paradigm of large language models (LLMs)(rrww)) et al.) 2020 Min et al. 2022), such as directly using GPT-4 for few-shot prediction. Although these methods have achieved remark- able performance, their decision- making processes are opaque, and the high inference costs of large models limit their application in domains requiring decision auditing or handling large-scale data.
Concept Bottleneck Models. To address the black box nature of modern NLP models, research has pursued two primary paths: post hoc and inherent interpretability. Post hoc methods, like feature attribution (e.g.. LIME, SHAP)Ribbeir et al.] [016)] Lundberg & Lee (017), aim to explain pre- trained models, but studies have shown their explanations may be faithful the model's actual reasoning. In contrast, inherently interpretable models like the Concept Bottleneck Model (CBM) transparent by design, forcing predictions through layer of human-understandabb. concepts and thus providing a faithful explanation for each decision.
Building on this framework, a recent line of work utilizes LLMs define concepts for CBM, while a smaller model performs the final prediction. These methods differ in their concept generation and inference paradigms. Some approaches generate static concept library for given task, such as CB- LLMs (Sun et al.) 2025) and SparseCBM (Tan et al.) 2024a), or expand upon human-provided seed concepts, as in CM (Tan et al.) (0024b). Others introduce dynamic concept discovery. For example, Ludan et al. 2024) introduced TBM, which uses classification errors to iteratively prompt an LLM for new concepts, while Feng et al. (2024) framed this process in a Bayesian framework with BC- LLM, exploring the concept space via MCMC sampling. A critical drawback of these models is their reliance on the LLM during inference. In contrast, methods like CB-LLMs and Sparse-Clr CBM achieve efficient prediction by training an embedding model to match samples concepts (Sun al. 2025
2

---

Published as a conference paper at ICLR 2026
Concept Generation
Two-stage Prediction
x
Eies
(z)
Arnotation and Salectio
C
4,(ar)
Ca.
Top-k (x)
ax()))
C
(x)
((())
Prediction
Concepts
Madessifed
Somantical
Tuning
C2
Large Language
Logical
CH
Tuning
Concept Tuning
The framework of StructCBM. We first prompt an LLM to generate an initial library of prototypical concepts C, and discriminative concepts Cą- Next, predict with the concept library in a two-stage process: we first select candidate labels by ranking sample similarities with Cp. and then finalize the label through contrastive comparison among these candidates using Ca- For misclassified samples, we semantically or logically refine the concept library based on their feedback in the concept tuning stage.
Figure 1:
Semenov al. 2024). Other works introduce parallel residual paths improve performance, as seen in TBM (Ludan et al. (024) and CT-CBM (Bhan et al., 2025). but this hybrid architecture risks information leakage potentially compromising the faithfulness of the explanations. A summary of related CBMs for text classification is shown in Tab.
3 METHODS
In this section, we first introduce the motivation of our poototypical-dicriminnne concept structure in Sec.3.11 Then, exploit the advantages of our concept structure, our workflow is designed in generate-predict-reeee pattern, where are introduced in Sec. 3.2 to Sec. 3n4 illustrated in Fig.
3.1 PRELIMINARY
To establish an interpretable and efficient model, recent CBMs (Sun et al. 2025 Ludan et al. 2024) often decomposes the prediction procedure into two sequential functions go f()): concept matching function f(-) X >> [Cl that maps the input data X' prediction model g(.) I R Dil that predicts the label set based on the concept relatedness. This framework is good substitute for black-bor models when there is abundant labeled data, but it would fail to handle few-shot scenarios due to the inability optimize g(-).
of prepared concepts C and a white-box
Motivated by this dilemma, we leverage extensive prior knowledge of LLMs to establish an explicit one-to-one correspondence between concepts and labels. This allows us simplify the prediction function g(-) into a static mapping, thus avoiding classifier training and shifting the entire burden of prediction optimization the concept representation function f(-).
From a probabilistic perspective, predicting on sample x with a concept library C can be expressed as ý arg maxgey P(y|z,C) C). With few-shot samples and possibly numerous concepts, using parametric methods to directly optimize this distribution is still impossible. Inspired by coarse-to- fine strategies, we re-frame this problem follows:
Plyæ,,C) = Σ C.DDE,. J.CY
(I)
where Y& y contains & candidate labels and P(y C) = when y & V/. Following this extension, the prediction transformed into 1) finding top-k candidate labels from y that contain the ground-truth and 2) finalizing the top-1 label from & candidates. This decomposition reveals two
3

---

Published as a conference paper at ICLR 2026
distinct sub problems: crriine task of generating high- nigh-quality candidate set (P()) C)), and a precision- precision-oriented task of ranking labels within that set (P(y))k, C)). Our key insight is that single, flat concept library is ill-suited to optimally solve both. We therefore propose two- stage architecture that employs specialized concept sets for each sub-problem: general Prototypical Concepts Cp ensure high recall in the first stage, and specific Discriminative Concepts Ca to achieve high precision in the second:
Prototypical Concept Set C,: Structured union of per-class subsets Cp = Uyrey Cj where each subset ç₁ is one-to-one correspondent to a label y" from the label set V. Each concept op E c, captures common and representative concept that defines the core identity of class y².
Discriminative Concept Set Ca: Structured as union of pairwise subsets Ca = Up C2.. Each concept c' E Ch provides a fine-grained, contrastive evidence to distinguish class y' from a confusable class y²
In this way, the strong LLM generalization ability amends the data shortage problem by optimizing concept matching effectiveness, which further improves the prediction accuracy. From causal perspective, architecture forms sufficient justification for each prediction by jointly answering "why is it class y " via Cp and "why it not the confusable class y ?d via Ca.
3.2 CONCEPT GENERATION
Concept Proposal C, and Ca are extracted by distinct strategies to guarantee their effectiveness. Cp are designed ensure high recall of correct candidate labels. Thus, we prompt the LLM with multiple samples from the same target class and instruct it to induce their shared, common features. Ca, contrast, must exhibit high inter-class separability to distinguish between similar candidates. To achieve this, we prompt the LLM with samples from two classes, y² and y², and instruct it to identify concepts that are characteristic of class y" but are rarely or never present in class y². Details of the concept proposal are shown in Appendix. A.3 The pair-wisc C. proposal process causes quadratic cost ((D)))) To mitigate such cost for large-scale scenarios (3) >> 10), one can cluster classes based Cp similarity and restrict Ca generation to the top-K semantic neighbors, thereby reducing complexity from to K), < D+
In detail, all concepts have name for identification and a human- understandable description for semantic matching. description crucial The will be converted into a vector embedding for similarity calculation. Therefore, the prompt explicitly instructs the LLM to write de- scription that is comprehensive and semantically rich, ensuring an accurate representation for the sub- sequent matching process. Concrete examples shown in Sec. [477
x11
z1
Class y²
x²
x
-1
Class y
22
X Reject
X Reject
1: Approce
Accept
Accept
0: Irrelevant
Coverage on
Average on yf
1: Opposite
y 50%
Average y
Concept Annotation and Selection Following the generation of candi- date concepts, we employ a logical annotation and selection process to prune low-quality concepts that fail their functionalities within the lim- ited training data. This process starts with an annotation procedure. For each candidate prototypical concept of and a training sample a from y we require an LLM to annotate their logical relationship with a score a(x", .e,,)))) E (-1, 0, 1), which means contradiction, irrelevance, logical consistency, respectively (see Figure 2.. And for
Figure 2: Illustrative example of LLM-based Concept An- notation. Rows represent text samples (æx), columns repre- sent candidate concepts (cz). Values {-1, 0, 1} denote se- mantic consistency.

---

Published as a conference paper at ICLR 2026
f.) each discriminative concept cd" the same annotation procedure will be applied to samples from both related labels y and y².
After annotation, we apply the following rules to select high-quality concepts:
For each cp. is only selected into C₁ when a certain threshold of samples from y' are consistent.
For each od acceptance into pairwise set C demands satisfaction of possitive scores for samples from target class y and non-positive scores samples from contrastive class y²
concept generation halts when all support samples satisty the activation criteria (i.e., a(x',c) for at least one of and a(x", c) = for all <'') ensure the reliability of this automated process, we rigorously validated both the internal consistency of the L.M'' scoring and its alignment with human judgment (see Sec. 4.6 Appendix. A.7
3.3 TwO-STARE PREDICTION
Once the concept library is generated, StructCBM operates as lightweight and LLM-free prediction engine. The prediction process follows a hierarchical decision-making procedure that is transparent and mirrors human reasoning: it first employs the prototypical concepts (Cp) to identify high- potential candidate set labels (D)), and then uses the discriminative concepts (Ca) to confirm the final prediction through Ane-grainedd contrastive analysis.
The prediction process requires the similarity comparison between concepts and samples, and we follow common practice use the text embedding similarities. With text embedding model e(.) that encodes texts into a unified d-dimensional vector space, the semantic relevance between and is then quantified by the cosine similarity of their respective embeddings: sim(x, c) cos(e(r) e(c)). This design choice is crucial for efficient inference, as it bypasses the need for expensive LLM calls during the online prediction phase.
3.3.1 STAGE 1: PROTOTYPICAL PRUNING
In this stage, our model aims to identify the most promising candidate labels by estimating the align- ment of the input x with each class's prototypes. For each class y¹ E Y, we calculate a Prototypical Support Score, s/ (z), by averaging the similarity of the top-m most relevant prototypical concepts:
(x)
= m
1
Σ op (CC.-
sim e)),,
(2)
where top-m(C),.0 x) is the set of m prototypical concepts from Cj that have the highest similar- ity scores with sample æ. We represent the prototype scores for all classes as a vector sp(x)) E RD' We then select the candidate set V% by taking the top-k classes with the highest (z) scores. This effectively prunes the label space, focusing subsequent analysis on the most plausible options.
3.3.2 STAGE 2: DISCRIMINATIVE RE-RANKING
The second stage finalizes the prediction by performing fine-grained, contrastive analysis ex- clusively among the candidate labels Vd. This stage quantifies the unique discriminative evi- dence for each candidate by a Relative Discrimination Score 2A. For any pair of candidate labels (y', v²) E Vk. a id (x) quantifies the net evidence favoring y² over y² as: (x) max(sim(2,, )''d E CC') max(sim(r,, c/i)) c(.))
(3)
We then aggregate these pairwise comparisons into a Net Discrimination Score, ()(() 3-d Eyt S. (z). for each candidate y² The final prediction for sample x is then determined by combining the initial prototype evidence with this fine-grained discriminative evidence. We compute a final score, s² (z), for each candidate y² .k:
s'(x)) (xr) + (1 a) s2(),,
(4)
where a E [0, 1] is a balancing hyperparameter. The prediction will be made on the label with the highest 8² (x)), that is arg max sxx)).
5

---

Published as a conference paper at ICLR 2026
3.4 CONCEPT TUNING
Finally, the framework closes the learning cycle with a concept tuning stage. This closed-loop mechanism uses classification errors from the prediction module as feedback to the LLM, guiding it to improve the concept library in semantical and logical ways. For Semantic Tuning, the LLM rewrites existing concept descriptions to strengthen their similarities to same-class samples and fix the errors, For more persistent errors, Logical Tuning is triggered, where the misclassified samples serve as targeted input to complete the concepts by re-invoking the generation process from Sec. 3.2
Semantic Tuning This stage rewrites existing concept descriptions to better align their semantical meanings with their logical purpose. Given residual misclassified sample r. there are two possible causes to the enror, recall error and ranking error, which are correspondingly handled during this process. To prevent concept drift and overfitting to single error samples, we enforce a regularization constraint: any updated concept (whether cp or ca) must maintain or improve its average similarity to the set of correctly matched positive samples while addressing the specific error. The algorithm details are shown in Algorithm | and related prompts are shown in Appendix, A.4
For a recall error, where the correct label ytrue is not in the candidate set, we identify the rel-
Algorithm 1 Semantic Tuning Function
1: Input: A residual case C. positive examples X;
evant prototypical concept Cp and prompt the
2: Output: Updated C²
LLM generate paraphrases. The paraphrase
3: type(() RecallError then
that maximizes the average similarity with all of its supported samples chosen to replace the original. For ranking error, where sam- ple misclassified, we identify the most mis- guiding discriminative concept ca and rewrite its description to minimize its similarity to the error-inducing sample.
4: Let ep be relevant concept.
5:
le Jema Prompt(c r, X.)
6:
op + argmax, of Exjateg. sim (²))
7: c'+cC' [ct] U ($))
8:
else if type(r) Ranking then
9:
cid c- arg mecx sim(r, c)
10: 1et' Jams Prompt(c .r)
Logical Tuning In case there are hard sam- ples that semantical tuning fails adapt, generate new concepts specially to cover these samples. The generation process for prototyp-
c2rin +- aar min d.s sim(c) (e .r)
12:
if cimlc""" .x.) > sim(c')) X.) then
13:
C eld (cf)) (c "
ical and discriminative concepts is the same in Sec. 3.2 with special focus on hard samples.
14:
end if
15:
end if
16: return Updated C
3.5
EMBEDDING MODEL FINETUNING
At the end of concept library construction, have an additional model training module that finetunes the embedding model to achieve a better matching performance between samples and concepts. The training data P is constructed by the samples and concepts that support the same label:
P= Uyy pt = {[((a', (((a".c))))) Metec, U ((x', (edeee)eees "
(5)
C, are excluded from data synthesis for their lack of generalizable information. After the preparation of P. we optimize the embedding model e(-) with cosine similarity loss:
é = arg min
Σ (art JEP
[18(i,)) cos(e(x'), e(c))))
where 5(1.j) = 1 if = j. and otherwise 0. In such a way, we could improve the embedding similarity accuracy and make better predictions. Note that such a procedure does not hinder the in- terpretability of StructCBM, since the prediction process is still honestly reflected through concepts.
4 EXPERIMENTS
We conduct extensive experiments on several text classification benchmarks with few-shot setting to evaluate the efficacy of our proposed framework, StMuctCBM. We introduce the experiment setup

---

Published as a conference paper at ICLR 2026
Table 2: Main results four benchmark datasets. Few-shot results are evaluated 10-shot per label setting. RoBERTa and DistilBERT are also finetuned on full training dataset to present performance upper bound.
SST2
AGNews
MedAbs
FinaQuery
Model
Acc
F1
Acc
F1
Acc
Acc
Black-box
RoBERTa
0,9462
0,9456
0.9508
0,9460
0.6457
0.6394
0.8561
0.8131
Full
DistilBert
0.8979
0.8978
0,9426
0.9428
0.6291
0.6296
0.8411
0.7600
RoBERTa
0,5008
0.3336
0.6284
0.7475
0.3778
0.5917
0.5056
0.6672
Black-hox
DistilBert
0,5859
0.5728
0.8359
0.8374
0.4349
0.4280
0.5289
0,4842
Few-shot
Deepseek-Diree
0.9610
0.9614
0.8047
0.7911
0.6395
0.6100
0.8499
0,8510
Deepseek-ICL
0.9630
0.9630
0.7900
0.7705
0.6374
0.6060
0.8360
0.8389
TBM
0,4526
0,4431
0.2490
0.2439
0.2890
0.2852
0.2788
0.2684
CBLLM
0.6326
0.6310
0.6834
0.6853
0.3521
0.3478
0.3465
0.3252
White-box
C
0,5392
0,5341
0.5568
0.6682
0.3109
0.5671
0,5056
0.6672
Few-shot
SparseCBM
0.5727
0.5628
0,6284
0.7475
0.3778
0.5917
0,4427
0.6656
StructCBM (Ours)
0.8390
0.8371
0.8545
0.8539
0.6070
0.6042
0.7742
0.7763
in Sec. 4.1 including the datasets, baselines, and evaluation metrics. We compare the effectiveness of StructCBM with baselines in Sec, |4.2, evaluate the contribution of each component in Sec. 4.3
examine the robustness of StructCBM in Sec. 4.44 and analyze its scalability in Sec. 4.5 Besides, we
quantitatively and qualitatively evaluate the interpretability of StructCBM in Sec. 4.6 and Sec. respectively.
4.1 EXPERIMENT SETUP
Datasets. We include four public text-classiffcation datasets from diverse domains ensure com- prehensive evaluation: AGNews on news topics (Zhang et al. 2015). SST2 on sentiment polarity (Socher al. 2013). MedAbs on medical abstract subjects (Schopf al. 2023), and FinaQuery on financial query category (Theerthala, 2025). For each of these datasets, we create a 10-shot training split by randomly sampling 10 examples per class and keep the original test dataset for evaluation. As there is no test set for FinaQuery, 25% of the full data is randomly sampled as test data. More details are shown in Appendix. [A.5]
Implementation Details. We extract, annotate. and logically refine both types of concepts by calling API from DeepSeek-V (DeepSeek) 2025). where the temperature is set to 0.7 and con- text length set to 8792. To align with baselines (Sun et al.) (025). the text embedding model is all-mpnes tompeetbase-eee package. The hyperparameters we used for inference are attached in Appendix.
(Song et al., (020) with implementation from the sentence transformer
Baselines. comprehensively evaluate StructCBM, we benchmark our model against three cat- egories of state-of-the-ar baselines. First, we compare with leading LLM-based CBMs. including TBM (Ludan et al. 20244. C (Tan et al. (2024b), SparseCBM (Tan et al. 20244)). and CB-LLMS (Sun al. 2025). Second, evaluate small black-box models, specifically RoBERTa-base (Liu ct al., 2019) and DistilBERT (Sanh et al. 2020). under both few-shot and fully supervised set- tings. Finally, to establish high-performance upper bound, we benchmark against DeepSeek V3 DeepSeek 2025) using both few-shot in-context learning and direct inference.
Evaluation Metric. Model performance is assessed using standard classification metrics, including Accuracy and Macro-F1 score. Accuracy provides an overall measure of correct predictions, while Macro-Fl is particularly useful in multi-class or imbalanced scenarios, offering a balanced view of precision and recall across all classes.
4.2 MAIN RESULTS AND ANALYSIS
We compare StructCBM against both white-box and black-box baselines under a strict 10-shot per label setting. For performance upper bound, also include RoBERTa and DistilBert models fine-tuned on the full training set. The comprehensive results are presented in Table StructCBM establishes a new state-of-the-art for interpretable few-shot learning by significantly outperforming
7

---

Published as a conference paper at ICLR 2026
Table 3: Ablation study different components of StructCBM. The results reported are based concept weighting of a = 0.75. "Reg." denotes the stricter regularization constraint during tuning. The best results for each dataset shown bold.
Components
SST2
AGNews
MedAbs
FinaQuery
Cp
Ca
Refine
Reg.
Train
0.7.07
0.7224
0.5530
0.6051
0,8029
0.7930
0.5547
0.6491
0.7677
0,8141
0.5900
0.6900
0.8127
0,8235
0,6004
0.7177
0.8390
0.8545
0.6070
0.7742
prior white-box box CBMs across all datasets. This result underscores the limitations of existing CBMs in data-scarce scarce scenarios and validates the effectiveness of our architecture. Furthermore, StructCBM demonstrates strong competitiveness against powerful black-box LLMs, especially on semantically rich datasets like AGNews and MedAbs. Notably, AGNews, our model surpasses the 10-shot Deepseek-ICL, and on MedAbs, our performance is comparable to the zero-shot Deepseek-Ding Direcc, all while offering the crucial advantages of full interpretability. A performance gap remains datasets with less semantic density, such as SST2 and FinaQuery. We attribute this to the advantage of the vast world knowledge embedded in black-box models for these specific tasks.
4.3 ABLATION STUDY
To validate the effectiveness of each key component in StructCBM, conduct comprehensive ablation study, with results presented in Table 3 For these experiments, the concept weighting parameter is set = 0.75. Our analysis focuses on two core aspects of our design: the dual-layer concept structure and the iterative refinement process.
First, incorporating discriminative concepts +C,+Cal consistently improves performance over using only prototypical concepts (C,) across all datasets, confirming that fine-grained boundary analysis is crucial for resolving ambiguity. Next, we evaluate the refinement (Refine) and regularization (Reg.) stages. While unconstrained refinement boosts performance semantically rich datasets (e.g.. AG- News), causes overfitting on the binary SST-2 dataset (accuracy drops from 0.8029 to 0.7677). Crucially, introducing the regularization constraint (Reg.) successfully resolves this instability, re- covering SST-2 accuracy to 0.8127. Finally, the embedding model training (Train) further enhances performance across all benchmarks, demonstrating the benefits of aligning the representation space.
4.4 SENSITIVITY AND ROBUSTNESS ANALYSIS
Impact of a on accuracy across datasets
Internal Concept Weighting (o.). We vary the weight a
[0, 1] to balance prototypical (C,) and discriminative (C₂) con- cepts. As shown in Figure 3 performance peaks when > 0.5, favoring C,. We attribute this to their functional roles: C, captures the core, generalized semantics (high-recall sig- nal), ensuring predictions land in the correct semantic vicinity. In contrast, Cd focuses on fine- mee-graneed boundary distinctions (high-precision signal). In few-shot settings, the robust signal from Cp serves as the primary driver, while Ca acts as a critical refiner for hard negatives.
0.8
AChens
Worics/oommet FraeviMQurry
0.03
0.25
0.50
0.75
1.00
Alpha (o) Weight of Prototype Concepts (C))
Figure Sensitivity of weighting a. 2 0.5 yields optimal results.
External Robustness. As detailed in Table we further val-
idated robustness against three components: (1) LLMss: Per-
formance remains robust across proprietary (GPT-40 (OpenAl) 2023) and open-weights models
(DeepSeek, Qwen3-Plus (Yang et al., (025)). (2) Embedding Models: StruciCBM effectively leverages stronger backbones. Switching from MiniLM (Wang et al. 2020 to Qwen3-embedding- 0.6B (Zhang et al. 2025) (noted as Qwen-emb) boosts accuracy (e.g.. +5.68% on AGNews), con- firming our framework scales with better representations. (3) Data Scarcity: Consistent gains observed from I-shot to 10-shot. Notably, the framework achieves respectable performance even in the extreme 1-shot setting (e.g., 0.6624 on AGNews), showing strong few-shot resilience.
8

---

Published as a conference paper at ICLR 2026
Table 4: Robustness analysis across different LLMS (left), embedding models (middle), and few- shot settings (right) on accuracy, StructCBM shows consistent stability across all configurations.
(a) Different LLM Generators
(b) Embedding Backbones
(c) Few-shot Settings
Datasct
DeepSock
GPT-4o
Qwen
Dataset
MiniLM
mpoct
Qwen-emb
Daraset
I-shot
5-shot
10-shot
SST-2
0.8390
0.8680
0.8011 0,8456
AGNews
0.8121
0.8545
0.8689 0.6143
AGNews
0.6624
0.8472
0.8545
AGNews
0.8545
0,8567
MedAbs
0.5907
0.6070
FinQuery
0.5873
0.7360
0,7742
4.5 EFFICIENCY AND SCALABILITY ANALYSIS
The efficiency of StructCBM is assessed by distinguishing between the one-time offline construction phase and the online inference phase. The primary compu- tational overhead arises exclusively dur- ing offline construction. Crucially, once the concept library is built, the online inference phase highly efficient, in- curring marginal API costs (see Ta- ble (5). Furthermore, the memory foot- print for concept embeddings is negligible (<10MB) compared to the backbone.
Table 5: Efficiency comparison. StructCBM incurs API costs only once during construction, whereas LLMs incur recurring costs per inference.
Dataset
Method
API Calls
Type
DeepSeek
1,820
Recurring
SST-2
Ours
75
One-time
DeepSeek
7,600
Recurring
AGNews
Ours
173
One-time
Regarding offline construction cost, empirical analysis indicates that the average API consump- tion per unit (class or pair) remains consistent across datasets. This stability is driven by our explicit stopping criteria (e.g., ensuring sufficient coverage for Cp and validity for Ca), which effectively bound the generation process. We provide a detailed quantitative breakdown of these unit costs in Appendix(A.8.. A.8 Although unit costs are stable, the number of pairwise comparisons grows quadrat- ically with classes. To ensure scalability for scenarios with hundreds of labels ([)) >>> 10), we propose similarity-
pre-filtering strategy (see Appendix (A.8.I) that restricts discriminative concept generation to semantic neighbors, reducing complexity from O(D))2) to (()))
4.6 INTERPRETABILITY AND RELIABILITY ANALYSIS
To rigorously validate interpretability, we recruited three evaluators with Finance backgrounds assess a stratified sample set. As shown in Table 6 eval- uators rated the concepts providing highly suffi- cient evidence for predictions (4.39/5.0). Further- more, blind agreement test demonstrated that the LLM's automated annotation matched human ex- pert consensus in 96.43% of cases, confirming the trustworthiness of our automated pipeline. Beyond human alignment, we verified internal stability on the MedAbs dataset, where the LLM maintained 98.6% consistency across independent trials, prov- ing that our framework produces deterministic fea- tures rather than stochastic hallucinations. Details are presented in Appendix. A.7
Table Quantitative validation of concept interpretability and reliability. Human eval- uation (Expert) confirms semantic validity, while LLM robustness tests confirm stability.
Evaluation Metric
Score
Human Evaluation
Contribution Faithfulness (1-5)
4.39
Annotation Agreement (%)
96.43
Annotation Consistency MedAbs (%)
98.60
4.7 CASE STUDY: TWO-STAGE REASONING IN ACTION
To intuitively demonstrate our model's reasoning process, we present a case study MedAbs about vascular surgical procedure. As illustrated in Figure E our model follows two-stage process to arrive at the correct classification. In Stage 1, the prototype score for each potential label is calculated by averaging the similarity scores of its most relevant Cp. For clarity, figure displays only the most similar prototypical concept of the top-2 candidate labels. The broad and general nature of the General pathological conditions label, led by its top concept 'Therapeutic intervention management', resulted a high sp. Despite this, our model successfully recalled the true label, Cardiovascular diseases into the top-2 candidate set.

---

Published as a conference paper at ICLR 2026
hatrograde recaraliation occluded posterior tiial artery using tibial
Recanalization antery outdown at
posterier artenies unsuccesofully attempted using
by utlizing a retrograde approech approech. Thus, the choice
ankle. Both
sit)
becomes crucial determinant angloplesty
Stage-one prediction: Label 1 General pathological conditions, Label 2 Cardiovascolar disceses (True label)
Concept type: Prototype
Concept type: Prototype
Concept name: Therapeutic intervention management and evalussion
Concept name:
repertusion intervention outcomes Description: Clinicl
following
Analysts of trestment modelites encompassing procedural implementation, technical approaches, comparative effectiveness of intervendons, and evalustion of therapeutic results end pasient outcomes.
Description:
restoring flow Ischemic cardiac tissue, induding success rates, complication, and ional improvements
Support label: Label 2 Cardiovascular diseases
Support label: Label 1
Geneval pathological condition
Protatype Support Save 4 0.3501
Protedgge Sieport Soore of 0.3341
Stage-two prediction: Label Carclovasculorr dieasses
Concept type: Discriminative
Concept type: Discriminative
anatomical Description: References surgical repair reconstruction anetomical structures spermatic card, oculer implants, and fistulss using techniques like onley coverage specific implant managsment Support label: Label 1
Concapt
Concapt Mantions surgical procedures speciiiclll Description:
vascular surgical intervention
urethrsi defects,
targeting bicod vessels, arteries, veins for repair, bypass, or revascuibrizat siritttinntationn purpases, including proceduras lika infrainguinal bypess, temorapopiittal bypass, remarodistal bypass, arterial graft placement
General pathological conditions
Support label:: Label 2 Cardivaazzuor disnaves
exclude_ Jabel: Label Cardlovascular discases
exclude labal: Label
pothoingical
Relative Diseriminative Score a
0.1241
Relative Discriminative Score of 0.1241
Score: oxl a)e? 0.2078
Finol Score: : nzg + (1 dj.. 0.2711
Figure 4: An illustrative example of our model's two-stage prediction process. Stage 1: A prototype score, calculated by averaging the similarities of the top three concepts for each label, is to select candidate set. For clarity, the figure displays the single most influential prototypical concept for each of the top two candidates. Stage 2: Discriminative concepts provide decisive contrastive evidence, correcting the final ranking in favor of the true label.
In Stage 2, the model leverages discriminative concepts to resolve the ambiguity. The concept vascular surgical intervention perfectly matches the text, providing positive evidence for the true label. Conversely, the concept non-vascular anatomical structure repair provides crucial negative evidence against the competing general label. This contrastive scoring corrects the initial ranking, leading to the accurate final prediction of Cardiovascular diseases with a higher final score. This case clearly shows how our architecture first ensures high recall and then uses discriminative evi- dence for precise, interpretable classification.
5 LIMITATIONS AND FUTURE WORK
While our framework demonstrates strong performance, we identify two key directions for future research. First, the pairwise formulation of our discriminative concepts C, scales quadratically with the number of classes, posing challenge for tasks with large label sets. Future work could ad- dress this by exploring on-demand concept generation for only the most confusable class pairs (also discussed in Appendix. A.8). Second, from causal perspective. our reasoning process provides sufficient condition for its predictions, but not a necessary one. A key future direction is to leverage the counterfactual generation capabilities of LLMs to compensate for necessity.
6 CONCLUSION
In this work, introduced StructCBM, novel framework for few-shot text classification that suc- cessfully merges the performance of large language models with the efficiency and interpretability of Concept Bottleneck Models. Our approach leverages dual-level, concept architecture and an error-driven refinement loop to generate high-quality concepts from limited data, enabling a lightweight, LLM-free inference process based on direct concept similarity. Extensive experiments confirm that StructCBM's performance is comparable to strong black-box baselines and surpasses prior CBMs, establishing a new paradigm for trustworthy few-shot learning-
cinottyicalldiserimnaaaa
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the General Research Fund (GRF) of the Hong Kong Research Grants Council (RGC) under grants 13310222, 16308221, and 16308421.
ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. this study, no human subjects or animal experi- ments was involved. All datasets used were sourced in compliance with relevant usage guidelines, ensuring no violation of privacy. We have taken care to avoid any biases or discriminatory out- comes in research process. No personally identifiable information was used, and experiments were conducted that could raise privacy or security concerns. We are committed to maintaining transparency and integrity throughout the research process.
REPRODUCIBILITY STATEMENT
The code and datasets for this paper are publicly available at blexis2lf/StructcMM The implementation details are shown in Sec. 4.1 and Appendix, A.6
com/
REFERENCES
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major. and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Madeleine Clare Elish, William Isaac, and Richard S. Zemel (eds.), FAccT '21: 2021 ACM Conference on Faimess,, Accountability, and Transparency, Virtual Event Toronto, Canada, March 3-10, 2021, pp. 610- 623. ACM, 2021. doi: 011145/74422188.4455922 URL
34421 88. 3445922
Milan Bhan, Yann Choho, Pierre Moreau, Jean-Noël Vittaut, Nicolas Chesneau, and Marie-Jeanne Lesot. Towards achieving concept completeness for unsupervised textual concept bottleneck models. CoRR, abs/2502.11100, 2025. doi: 10.4550/ARXXI2252..1100 URL https:
//doi. 48550 /a./0v//rivvvvv 2502. 11100
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL http::tttp:::/pccdtdiitt neurriiiiiinnnnnnnsseeee.eee.ies www.eeewww....peeeeeeeeeeeeeeeee
ottp::aa.aaaaaaaa..aaaaaaaaaaa.a .tmll
Abstract
DeepSeek.
Deepseek-v3 technical report, 2025. URL
19437
Finale Doshi-Velez and Been Kim. Towards rigorous science of interpretable machine learning, 2017. URL /ttt::////:: http:///wwxi..or///////////2//// 08608
Jean Feng, Avni Kothari, Luke Zier, Chandan Singh, and Yan Shuo Tan. Bayesian concept bottle- neck models with LLM priors. CoRR, abw/2410,15555, 2024, doi: 10.4850/ARXIV.2410..555 URL htt.:https:/eooo
.rgg 10.1
15555
Tianyu Gao, Adam Fisch, and Dangi Chen. Making pre-trained language models better few-shot learners. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/ICNNPP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pp. 3816-3830. Association for Computational Linguisties, 2021. doi: 10.18633VV//2021AAC 2021 2021.acl-long..0
URL //doi. www.oo.g..............o 18653/v11

---

Published as a conference paper at ICLR 2026
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 5338-5348 PMLR, 2020. http: / htt.:ttp:://procedngggss.s ht.m
m r. wt.ppess//ll1///e
Yinhan Liu, Myle Out Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, 2019. URL https:://///. /ttp:////aaivi.ieiaa httgg/bb///ggg 11692
Josh Magnus Ludan, Qing Lyu, Yue Yang. Liam Dugan, Mark Yatskar, and Chris Callison-Buree Interpretable----desiinn text understanding with iteratively generated concept bottleneck, 2024.
URL BRLLhhtp:///zaae org/ abs/ 20.///3/0 2310. 19660
Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Is- abelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vish- wanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 4765 4774. 2017. URL http::: paper/200 2tt/2/017ahhhh//e hww.//aaahhaaae 0aw0aa0022220 97711978888 978 7727677777777 2at.aaaaa8a222202200a2aa22222622 Aac .tmll
Abst
Max Ruiz Luyten and Mihaela van der Schaar. A theoretical design of concept sets: improv- ing the predictability of concept bottleneck models. Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024. Vancouver. BC, Canada, December 10 15, 2024, 2024. URL http: /ttp:::/ppppp papers wwappers.niisss nips wwtp:pppp.ccppppppeeeepeeppppppe fappeerr_lllssee wwweffileeppee http:////appeeeeeeeeeeee/r////2/ 1 1251115111111 2t1.1110000221111111000000000ar0
Amir Globersons, Lester Mackey,
html
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pp. 11048-1 1064. Association for Computational Linguistics. URL https::///oooo
2022. doi:
18653
759
OpenAl. techiiaac CoRR, abs/2bb3....7774.4..0000 2023. doi: 1048550/RXXV.230300077
URL
Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should I trust you?": Explaining the predictions of any classifier. In Balaji Krishnapuram. Mohak Shah, Alexander J. Smola, Charu C. Aggarwal, Dou Shen, and Rajeev Rastogi (eds.), Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA. USA, August 13- 2016, pp. 1135-1144 ACM, 2016. doi: 10.11457939672.22397777 114.1.1.11111... 11t.//10.111/29293 http:///oo..og11..11115333339999 2939777
URL https:://ooiii org
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, 2020. URL
01108
Tim Schopf, Daniel Braun, and Florian Matthes. Evaluating unsupervised text classification: Zero- shot and similarity-basee approaches. In Proceedings of the 2022 6th International Conference on Natural Language Processing and Information Retrieval, NLPIR '22, pp. 6-15, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9781450397629, 3582768.3582795 TRLLhttss:///o..rre
doi: 10.1145/
114 14/ 008.55876888888888888 3582795
Andrei Semenov, Vladimir Ivanov, Aleksandr Beznosikov, and Alexander V. Gasnikov. Sparse concept bottleneck models: Gumbel tricks in contrastive learning. CoRR, abs/2404.03323, 2024. doi: 10.4850/AARXI..244400552 URL /tttps://doi. org 48550/ arwivv/raivvivv. 2404.
03323
12

---

Published as a conference paper at ICLR 2026
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In David Yarowsky, Timothy Baldwin, Anna Korhonen, Karen Livescu, and Steven Bethard (eds.), Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631-1642, Seattle, Washington, USA, Octoher 2013. Association for Computa- tional Linguistics. URL pttps::/aacaamtmhoooyyyoooo
1170/
Kaitao Song, Xu Tan, Tao Qin. Jianfeng Lu. and Tin Liu. Mpnet: Masked and permuted pre- training for language understanding. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Ilsuan-Tier Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 12, 2020, virtual, 2020, URL
www.iee..c... wwwpsshhccaabbbbeee Baa 2eeeaae00eeeeeeeeeeeee.... Atto://eetbttttttttttt
Chung-En Sun, Tuomas P. Oikarinen, Berk Ustun, and Tsui-Wei Weng. Concept bottleneck large language models. In The Thirteenth International Conference Learning Repressenaations, ICLR 2025, Singapore, April 24-28, 2025, OpenReview.net, 2025, URL https:: net Dett/Eorum?id=RR5PPP
Zhen Tan, Tianlong Chen, Zhenyu Zhang, and Huan Liu. Sparsity-guided holistic explanation for Ilms with interpretable inference-tine time intervention. In Proceedings of the Thirty-Fights AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial AAAI Press, 2024a. ISBN 978-1-57735-887-9 doi:
Intelligence, AAAAA 10.1609/aaai, www.......1.... 100 WRLhtttp::.. //do:::///dooo org/ 1699
(1009//aaaa aaai v38119. 30160
Zhen Tan, Lu Cheng, Song Wang, Bo Yuan, Jundong Li, and Huan Liu. Interpreting pretrained language models via concept bottlenecks. In De-Nian Yang, Xing Xie, Vincent S. Tseng, Jian Pei, Jen-Wei Huang, and Jerry Chun-Wei Lin (eds.), Advances in Knowledge Discovery and Data Mining 28th Pacific-Asii Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Taipei, Taiwan, May 7-10. 2024, Proceedings, Part III. volume 14647 of Lecture Notes in Computer Science, pp. 56 Springer, 2024b. doi: 1007 978 981 2259-4_5
0000/99778-981-9-2259-9991 5. URL
btttps://doi.orr
Akhil Theerthala. Reddit personal finance q&a dataset, 2025. URL https://uuggigggfee ho//
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self-attention distillation for aank-agio compression of pre-trained transformers, 2020. URL 10957
https://arxiv:
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, OpenReview.ner, 2022. URL htttps:://ppereeeetete www. meweeefoormiiio gE2rGCozdqR
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang. Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin. Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhennu Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL
org/ 2505. 09388
Xiang Zhang. Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In NIPS, 2015.
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 embedding:
13

---

Published as a conference paper at ICLR 2026
Advancing text embedding and reranking through foundation models, 2025. /arxiv.org/abb// 05176
URL https:
Zixuan Zhou, Xuefei Ning. Ke Hong. Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, Shengen Yan, Guohao Dai, Xiao-Ping Zhang, Yuhan Dong, and Yu Wang. A survey on efficient inference for large language models, 2024. URL https:
7/ /arxiv.org/abbiia. htwpv//.rr/abb//24.11000
A APPENDIX
A.1
USE OF LARGE LANGUAGE MODELS
Large language models are used for finding related works and polishing the paper presentation. The authors manually surveyed representative related works and used LLM only for supplementary paper discovery. All the related works returned from LLMs are checked and reviewed by at least one author, and all of the references LLMs is always based on the original draft written by the authors, which also carefully reviewed by least one of the authors.
guaranteed authentic and real. The content that is polished by
A.2 ILLUSTRATIVE COMPARISON BETWEEN DIFFERENT TEXT CLASSIRICATION METHODS
We compare the pipeline between direct LLM inference, other CBM models and our models under text classification scenarios in Figure below.
X
Xest
fow sluc
LLM
Conoepts
Similarity Seore
Classifier
X
Current Text CBMs
LLM
Conocuta
LLM
Concenta
Similarity Score
Rank
Concepts
Xust
Direct Inference by LLM
Our Text CBM
Existing LLse- CBMs typically re- quire a large training (Ximin) generate concepts and then train a separate classifier map concept scores to labels. Bottom: Our proposed method utilizes a few-shot set (Xsnot to generate concepts with built-in one-to-one correspondence labels, enabling direct prediction via similar- ity ranking without trainable classifier.
Figure 5:
A comparison conceptual workflows. Top:
A.3 PROMPT DESIGN FOR CONCEPT GENERATION
In this section, we provide detailed overview of the prompt structures used to guide the Large Language Model (LLM) in generating our dual-layer concept library. The design of these prompts is crucial for eliciting high-quality, specialized concepts tailored for our few-shot learning framework.
A key instruction embedded both prompt types is the explicit guidance that the generated concept "description" will be subsequently used for embedding similarity matching. This encourages the LLM to produce descriptions that are not only semantically accurate but also rich contextual cues and varied expressions, making them more robust for downstream similarity calculations.
Furthermore, given the few-shot nature of our task, we cannot rely on statistical metrics to guide the concept generation process. To mitigate the risk of generating redundant overlapping concepts,
14

---

Published as a conference paper at ICLR 2026
implement simple yet effective de-duplication strategy. In each generation request, explicitly provide the LLM with a list of previously generated concepts for that class or class pair, and instruct it to avoid repetition, as shown in both Figure 6 and Figure
[une]] ("doneia!) reasuningg expurt.
are developing concept-bared claceifier label: "(1abeli",
are (Len tsanples) samples that confimmed belong to "(label]";
(sarplo texta]
Your task is to identify 1-2 semantic concepts that help daternine whether
a. new sangle abould belong to tato label.
We have alieady identiCied the Collowing known consepix (plnasu do not zupeus Ubwaa) (Koovo 1isi] Propose new rehigh-leoel conceplatt that helg idencify sapples belonging La label ([abel)" concept asould:
Capture an abstrant, broadly applinabla asnantic pattazr commonyy observed in labal Be general enough to satch varied expressions acrose samples
Avoid overditting to any single phrese oz exarple
inportant:
the "description" will later be convertad into an unbedding for similarity matching.
Ttazefore:
Wzita in clear, professional, academia style
Neturally incl senentia nows and related expression that freguently aosoceur with this labe] Eneure desoription is corprebeneive cooeept,
nultiple wording variations
"nare': abors
concept
concept noms)
2. "cescription's generalized
can other zamples senancioally
3. "explanation". reasoning way this concept usetul. for clansification Strictly zetues "egeenn
thias
[(DATAB7 COMPIOS dataset nans] I' l'exarple pccpose_cocepp"] [])c
Figure 6: The prompt structure for generating prototypical concepts (C,). The LLM tasked with identifying high-level, generalizable semantic patterns from few examples of single target label. is explicitly instructed not to repeat concepts from the provided linown
A.4 PROMPT DESIGN FOR SEMANTIC TUNING
The prompts for Semantic Tuning stage are tailored to the distinct roles of prototypical (C,)) and discriminative (C.) concepts.
To maintain the generality of prototypical concept during rewriting, our prompt includes not only the misclassified sample but also a reference set of correctly classified examples, as shown in Fig- ure 8 This crucial step guides the LLM to capture broader semantic essence and avoid overfitting to single error case.
In contrast, rewriting Discriminative Concept aims sharpen its specificity. The prompt, illus- trated in Figure focuses solely on the misclassified sample and the misguiding concept. It instructs the LLM revise the description more semantically dissimilar the error, thereby directly addressing the failure of discrimination.
A.5 DATASET DETAILS
The benchmark datasets are public text-classification datasets. Their statistical information is listed Table. 7
A.6 HYPERPARAMETER DETAILS
The weight a that determines the contribution of C, and C2 is selected from {0, 0.25, 0.5, 0.75, 1}, and the detailed effect analysis of the performances between different choices and their effects
15

---

Published as a conference paper at ICLR 2026
You are (DNIASET_ LOOPPIG (dataset_aaa) [aane] ('damain'1) reasoning expert.
building concegs-based classi fier
*aappport labal "[abeel_3" [*] and *ezoject label "[lbbe__]) +s
concepte help distinguieh
the target latel "[1abel_s" 11" contrasting "[1abci_3]"
do chie, enalyse the exaplles from both labels provided below,
You should propose t+1-2 conceptare acah that:
These concepts clearly appear are implied in camples from label "Clabel 11"
nut. they are Matrongly absent aven contradictary** in samplex From label "(label_55
this concept apgeare in sampie,
rssbould belong label "(label_3)"*
For concept:
conciso "conospt dctinition lolg., "bilsteral nourocog.cal cymptone")
Providee a *fgenenal iabile descriptiont for semantic comparison
3. Juscity why this csscept excludes label "[1abol_]FP usica petor knowledge and the examples
These concepts will be used to *ell milte lsbel "(wbell_5 [a "x when the concept is found.
A Exiating concepta already procosed (aXip daplicatea):
textsting)
Inout exanples:
jexamples_tent text)
For each concept, return JOON object with:
""nane""; short definition of the concept. (e.g., '"surgioal sointarvention")
"description""s general description how this appеaти in
"Msppaaattina'' why this concept helpa distingulah label "[label_4"" fron "(laba1_]) like:
[LDATASRT_ CONFTGE (datsset_neet neme)]
('exepple propooseepppee__eoo_o oo careecs__pppppeeeeee connept') 1b, ...
The prompt structure for generating Discriminative Concepts (C.). The LLM is given examples from two confusable labels and tasked with discovering exclusive concepts that are present the target label but strongly absent or contradictory the contrasting label. The list of "existing" concepts is provided to prevent duplicates.
Figure 7:
Table 7: Summary of Datasets Used
Dataset
Train
Test
Classes
Task
SST2 (Socher et al. 2013)
6.92k
1.82k
Sentiment classification
AGNews (Zhang et al.
2015
120.00k
7.60k
News topic classification
MedAbs (Schopt et 2023
11.6k
2.89k
Patient condition classification
FinaQuery Theerthaaa
2025
7.04k
8
Query topic classification
shown in Sec. As for training the embedding model, the hyperparameters are set to 3 epochs, 4 samples per batch for MedAbs and 32 samples for other datasets, 0.00001 learning rates. All the other hyperparameters are the same as the default setting.
A.7 HUMAN EVALUATION CONCEPT RELIABILITY AND VALIDITY
A.7.1 EXPERIMENTAL SETUP
Stratified Sampling Strategy. To ensure diversity and representative coverage across decision boundaries, we employed stratified sampling strategy. We randomly selected 2 samples from each class across three datasets (SST-2, AGNews, and FinancialQueries), resulting in a total of 28 evaluation instances (N 28).
Expert Evaluator Recruitment. Given the domain-specific complexity of the FinancialQueries dataset, general crowdsourcing platforms (c.g.. MTurk) may lack the necessary precision for eval- uating financial concepts. To ensure high-quality, expert-level judgment, recruited three inde-
16

---

Published as a conference paper at ICLR 2026
coMPrie ('dosst"]]
der a EDNTASST coorsee daccese, sanell "Jooass") classicter,
Ne Zave cancept fscicz "[Eector_naras para)" under label "[tabal]" vick the caseens description: deaociptiin)"
esprobl enyer
semplea that abould this cusoupt cureently being nisciaszitied recoynized beisoging "(abbe))"), "SSopple thes should ratch this but eurcently nirclessttiee:" Intaciarsiffed Stock]
#stupplat correctly
Ioorreet blook)
Revias the concept description to nate it ncee libely to natel, the visciarastied suppla
*Hequiresintai
aiepleste
ptlor ght wlosing? don't break what's slready vorking
pattorno they
"Macntate copatibilltyer
"Aroid owertttingg don't stam phrases from rissianstied surples, but сapсure their semantic exsence description abould sinllar beyand these specific
eetoo aodical abstant langugert professiinaly prosise, sot onerly techwiest
Genazata candidete reviied descriptions, Tach abould be contise,
Returr strletly is this 1938 fornst:
100n
"aator sase": "factorr naze"".
forigital descriptios"; "(original description)".
datddidate
*candldate dcseziption 25,
description
"candidete description 4*
Figure 8: Prompt for rewriting a prototypical concept (Cp). A reference set of correct samples ('correct blockk) is provided to maintain generality and prevent overfitting.
pendent evaluators with academic backgrounds in Finance. Each sample was independently assessed by all three evaluators.
A.7.2 TASK 1: CONTRIBUTION FAITHFULNESS
Protocol. The goal of this task assess the semantic validity of the concepts used for prediction. For each instance, evaluators were presented with:
1. The input text sample (drawn from the test set).
2. The model's predicted label.
3. The Top-3 concepts utilized by StructCBM for that prediction.
Evaluators were asked to rate, on a 5-point Likert scale, whether these concepts provided sufficient evidence to justify the classification
Irrelevant/No Evidence, Perfect Evidence).
Results.
StructCBM achieved high average rating of 4.39 5.0.. This result strongly indicates that the concepts identified by our framework are not merely statistical artifacts but offer semantically meaningful and sufficient explanations that align closely with human expert reasoning.
A.7.3 TASK 2: ANNOTATION RELIABILITY
Protocol. This task validates the reliability of the automated "Concept Annotation" phase (de- scribed in Section 3.2), where the LLM determines if a concept is present in a sample. We conducted blind agreement test where evaluators manually determined the presence of specific concepts in
17

---

Published as a conference paper at ICLR 2026
eirolaroitied
Freducted labal: "(abbe133"
3 Marsianeidled nurple:
Deccet trem lebs. 1itel_111 Isbeaud aupport t31p cazer:
Ica rane)
X concaps foor rlatal 31 jeua veuagay casdaadi
(o9_51_deac)
becenesce sжuaçiea to guide geoeralitacion:
Sequersing exeapled thim (laka) Li, sarch (or, MW.M Hawee]]-
Exciodiag excples (1abe2.) 33,
match fox.3 pealtt
tescovepp. Ledent c2t1110 2012 th toe in axclode sample tects), * iE exdicde, sasple, cesta elme (Ro01" Jepresant
Therstore revine the coscription of [1 or his expentlag is viglesernt the mtaclessicles narple betile still genteed enough). Aecias descriptios erics camg)es exbedding vedurster Secs sisclassitied Do wor weertt to the esatt aisniascitied sxpple vordting
cerized deveriprion. still
caumpico.
will oelace the candicace.
If connept aliwaity worle willy cetora in with ""ordlideree" II"
outpec: ststecly 2905 this domnect
Seoz
"type": "revines".
"same:: rare?",
"tspe": "reviaed".
"sow", Trex (raess
"candidates": Pfa. "...f]
11
Figure 9: Prompt for rewriting a Discriminative Concept (Ca). The goal is to enhance contrast by making the description more dissimilar to the misclassified sample.
the text (Yes/No) to establish a "Human Ground Truth." These manual annotations were then com- pared against the LLM's automated scoring.
Results. The LLM's automated annotation matched the human expert consensus in 27 out of 28 cases, achieving an agreement rate of 96.43%.
This high level concordance empirically validates our reliance on LLMs for feature annotation. demonstrates that, within the StructCBM framework, the LLM operates with precision comparable to human domain experts, ensuring that the input features for the concept bottleneck are robust and trustworthy.
A.8 DETAILED COST AND SCALABILITY ANALYSIS
To provide comprehensive assessment of the computational overhead and scalability of StructCBM, we quantified the API consumption required for concept construction across different datasets. We further detail the scalability strategies proposed for large-scale scenarios.
18

---

Published as a conference paper at ICLR 2026
A.8.1 QUANTITATIVE COST BREAKDOWN
Our empirical analysis indicates that while total costs naturally increase with the number of classes, the unit cost per class (or class pair) remains consistent.
Understanding Unit Costs. The reported unit costs encompass the full lifecycle of concept con- struction: (1) Generation, (2) Annotation (Validation), and (3) Retries. The number of API calls primarily determined by the stricteess of our quality-control criteria. For example, our 10-shot setting, we enforce a minimum requirement of 5 valid Cp concepts per class and ensure that every sample covered by at least one Cp concept. The LLM typically satisfies these criteria within ~7 generation attempts (each followed by annotation), resulting in the observed average of ~15 calls for Cy. This mechanism prioritizes concept quality over minimal API usage, ensuring robustness.
Total vs. Unit Costs. Table 8 presents the total API calls required for the 10-shot construction phase across three benchmarks. Despite the variation total calls (from 75 to 1072), Table 9 con- firms that the average consumption per unit is stable. For instance, the generation of discriminative concepts stabilizes at approximately 9 calls per pair in the 10-shot setting, regardless the dataset size.
Table 8: Total API Calls for Concept Construction (10-shot setting).
Dataset
Classes (N)
Total Pairs (N² - N)
Total API Calls
SST-2
2
2
75
AGNews
12
173
FinaQuery
8
56
1072
Table Average API Calls Per Unit Breakdown, Note that C, costs are per class, while C2 costs are
pair.
Component
I-shot
5-shot
10-shot
C, Generation (per Class)
4.83
11.12
15.38
C, Generation (per Pair)
4.07
9.38
9,08
Concept Tuning (per Class)
0.00
9.77
13.01
Cost Analysis of the Tuning Phase. We observe that the tuning cost for FinaQuery is relatively higher (~77 total calls). This is primarily driven by the Concept Tuning phase, which is inherently dynamic: API usage correlates with the number of misclassified samples in the few-shot supporr set. However, this cost is fully controllable. In experiments, we prioritized performance, allowing the tuning loop to run until convergence (or up to a high maximum round limit) to correct every possible error. In practice, this creates a flexible Performance-Com Trade-off:
1) Performance Mode (Used in Paper): Allow extensive retries (e.g., 5+ rounds) to maximize few-shot accuracy, resulting higher API calls.
2) Efficiency Mode: Restrict tuning to fewer rounds (e.g.. 2 rounds of Semantic + round of Logic Tuning). Empirical observations suggest that the majority significant corrections occur in the first 2 rounds, allowing users to cap costs significantly with minimal impact on final accuracy.
A.8.2 EFFICIENCY COMPARISON: ONE- TIEETIME vs. RECURRING COSTS
A critical advantage of StructCBM is that API costs are incurred only once during the offline con- struction phase. In contrast, LLM--bsed baselines (e.g., DeepSeek-ICL) require API calls for every inference sample.
As shown in Table 10 while the offline construction cost for StructCBM on AGNews is 177 calls, the LLM baseline requires 7,600 calls to process the test set. Consequently, our method's 's cost does not scale with the volume of test data, ensuring superior scalability for large-scale deployment.
19

---

Published as a conference paper at ICLR 2026
Table 10: Efficiency Comparison with LLM Baselines. StructCBM incurs one-time construction cost, whereas LLMs incur recurring inference costs.
Dataset
Method
API Calls
Type
DeepSeek-ICI. StructCBM (Ours)
1,820
Recurring (Per Inference)
SST-2
75
One-time (Construction)
DeepSeek-ICL
7,600
Recurring (Per Inference)
AGNews
StructCBM (Ours)
173
One-time (Construction)
A.8.3 SCALABILITY STRATEGY FOR LARGE-SCALE SCENARIOS (N > 10)
Although the unit costs are stable, the total number of pairwise comparisons for discriminative concepts (Ca) grows quadratically (O(N3) with the number of classes N. To prevent prohibitive API demands in scenarios with hundreds classes, we propose a Similarity- Pre-filtering Strategy.
Mechanism. Instead of exhaustively generating Ca for all N(N - 1) pairs, we utilize the rep- resentations of the Prototypical Concepts (Cp) (Cp)-which are generated linearly (O(N))-t map the semantic topology of the labels.
1) Cluster Candidates: We calculate the cosine similarity between the embeddings of C, for all classes.
2) Filter Pairs: For each target class i, identify the top-K most semantically similar classes (distractors).
3) Targeted Generation: We generate discriminative concepts Ca only for these specific top-K pairs.
Complexity Analysis. This strategy effectively prunes the search space. Because the filtering process relies local embedding similarity (zero API cost), the major API cost is restricted to the targeted generation. For N classes with a neighbor size K, the complexity reduced from ( to O(N K)). Since K (confusing classes) is typically small and independent of N, this ensures linear scalability.
20


Reviews:
[{"review_id": "KerKBSt23u", "reviewer": "Reviewer_BfVT", "summary": "This paper introduces StructCBM, a novel framework that successfully enables Concept Bottleneck Models (CBMs) to operate in few-shot text classification settings (e.g., 10-shot). The key innovation is a \"classifier-free\" paradigm that bypasses the need for a data-hungry trainable classifier, which is \"impossible\" to optimize with limited data. Instead, it relies on direct sample-concept similarity. This is operationalized via a dual-level \"prototypical-discriminative\" concept architecture and a \"generate-predict-refine\" loop where an LLM discovers and tunes concepts from a few samples. The method outperforms prior CBMs in the 10-shot regime and achieves performance comparable to black-box LLMs on semantically-rich datasets, all while being interpretable and efficient (LLM-free) at inference.", "questions": "1.  **On Embedding Model Sensitivity:** Your entire method relies on `sim(e(x), e(c))`. How sensitive is the model's performance to the choice of the base embedding model $e(\\cdot)$? Why was `all-mpnet-base-v2` chosen, and would a different model change the results?\n2.  **On Refinement Overfitting:** The refinement (Alg 1) appears to overfit on SST2. This seems due to the greedy, 1-sample error correction objective. Did you experiment with a more regularized tuning objective (e.g., one that forces a concept to remain similar to *all* its correct samples while becoming dissimilar to the one error sample)?\n3.  **Finetuning** The ablation study (Table 3)  shows that the \"Train\" step (Sec 3.5) provides a significant boost, especially on FinaQuery. How much does this finetuning of the embedding model contribute to the final performance, and does this reliance on a training step weaken the \"classifier-free\" premise?\n4.  **LLM Annotation Reliability:** The \"Concept Annotation and Selection\" step (Fig 2)  uses an LLM to score logical consistency. How reliable is this process? What is the impact of potential LLM annotation errors on the final concept library and performance?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novelty: This work successfully and practically applies LLM-augmented CBMs to the few-shot learning domain. It directly addresses a clear gap in the literature, as prior CBMs either required large training sets (like CB-LLM) or costly LLMs at inference (like TBM).\n- Strong Empirical Performance vs. Baselines: The model outperforms prior interpretable CBM baselines (TBM, CB-LLM) in the 10-shot setting across all datasets.\n- Competitive with Black-Box LLMs: The method achieves performance comparable to, and even exceeding (on AGNews), strong black-box LLM baselines (Deepseek-ICL). It achieves this while being \"white-box\" interpretable and \"lightweight\" at inference.\n- Sound and Validated Methodology: The dual-level prototypical  and discriminative architecture  is well-motivated for its recall-then-precision task decomposition. This design is empirically validated by both the ablation study (Table 3, showing $\\mathcal{C}_d$ is crucial)  and the case study (Fig 4, showing $\\mathcal{C}_d$ correcting an initial error).", "weaknesses": "- Overfitting: The concept refinement loop, a key part of the method, is shown to be brittle. The ablation study (Table 3) reveals that the full model (with refinement) performs worse than a simpler version on the SST2 dataset, which the authors attribute to overfitting\n- Additional Parameters: The \"classifier-free\" model rests on the quality of a base embedding model (`all-mpnet-base-v2`), which is a critical, unanalyzed hyperparameter. Furthermore, the \"Embedding Model Finetuning\" step (Sec 3.5)  re-introduces a training phase, which complicates the \"classifier-free\" narrative and appears responsible for a massive performance jump on FinaQuery (Table 3).\n\nMinor:\n\n- There is some invisible text on Page 15.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K9TFs2aB3O", "reviewer": "Reviewer_XLoV", "summary": "This paper proposes StructCBM, a novel framework for interpretable few-shot text classification that leverages an adaptive concept library. The core idea is to decompose the classification task into a two-stage process: (1) generating and maintaining a library of human-readable prototypical (positive) and discriminative (negative) concepts for each class using a Large Language Model (LLM), and (2) making predictions by measuring the similarity between an input sample and these concepts. The framework's key innovation lies in its closed-loop \"Concept Tuning\" mechanism. This mechanism uses misclassified samples as feedback to iteratively refine the concept library through Semantic Tuning (rewriting existing concepts to fix recall or ranking errors) and Logical Tuning (generating entirely new concepts for hard samples). Finally, the framework fine-tunes an embedding model to better align sample and concept representations. The authors evaluate StructCBM on four diverse text classification benchmarks (SST2, AGNews, MedAbs, FinaQuery) in a 10-shot setting, demonstrating superior performance over other white-box and concept-based baselines while maintaining full interpretability.", "questions": "-**Scalability**: How does the runtime and memory footprint of StructCBM scale with the number of classes? Have you experimented with datasets that have a larger number of categories (e.g., 10+)? What are the bottlenecks?\n\n-**LLM Cost**: Can you provide an estimate of the number of LLM calls (for both initial generation and tuning) required per dataset? How sensitive is the final performance to the quality/size of the LLM used for concept engineering?\n\n-**Human Evaluation**: Have you conducted any user studies to validate that the generated and tuned concepts are indeed meaningful, non-redundant, and helpful for a human to understand the model's decisions? This seems crucial for a paper centered on interpretability.\n\n-**Ablation on Tuning**: The paper mentions evaluating the contribution of each component. Could you share the performance drop when the Semantic Tuning and Logical Tuning stages are ablated? This would help quantify the value of the core adaptive mechanism.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-**Novel Adaptive Mechanism**: The Concept Tuning stage is the paper's standout feature. By using classification errors as direct feedback to improve the concept library, the framework creates a powerful closed-loop system that can self-correct and evolve, which is a significant step forward from static concept bottleneck models.\n\n-**Strong Empirical Validation**: The experiments are comprehensive, covering four distinct domains. The results clearly show that StructCBM outperforms other white-box and concept-based methods by a large margin, while remaining competitive with powerful black-box LLM approaches (Deepseek). This convincingly demonstrates the framework's efficacy.\n\n-**Preserved Interpretability**: The paper successfully achieves its primary goal of interpretability. Every prediction is grounded in explicit, human-readable concepts, and the tuning process itself is designed to improve the semantic quality of these concepts.", "weaknesses": "-**Scalability to Many Classes**: The paper focuses on benchmarks with a relatively small number of classes (e.g., SST2 has 2, AGNews has 4). It is unclear how the concept library and the tuning process would scale to problems with dozens or hundreds of classes. The cost of maintaining and tuning a large number of prototypical and discriminative concepts per class could become prohibitive.\n\n-**LLM Dependency and Cost**: The framework relies heavily on an LLM for concept generation and rewriting. The paper does not discuss the computational cost or latency implications of these LLM calls, which is a critical factor for real-world deployment. It would be useful to know the number of LLM calls required per tuning cycle.\n\n-**Lack of Human Evaluation**: While the concepts are human-readable, the paper provides no human evaluation to assess their actual quality, coherence, or usefulness to an end-user. An expert or crowd-sourced study on the concepts' validity would strengthen the interpretability claim.\n\n-**Black-box Baseline Choice**: The black-box baselines (Deepseek-Direct/ICL) are strong, but it would be more compelling to also compare against a fine-tuned smaller model (e.g., a fine-tuned DistilBERT on the 10-shot data) to better isolate the benefit of the concept-based approach versus just using a more powerful LLM.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KJM5HPAhUa", "reviewer": "Reviewer_swGd", "summary": "This paper presents StructCBM, an LLM-augmented Concept Bottleneck Model (CBM) for interpretable few-shot text classification. Unlike traditional CBMs that require large datasets and trained classifiers, StructCBM predicts labels through sample–concept similarity, removing the need for parametric training. It introduces a dual-level concept structure: prototypical concepts (Cp) capture representative class features for recall, while discriminative concepts (Cd) distinguish confusable classes for precision. A generate–predict–refine loop further improves concept quality through LLM-based semantic and logical refinements, while inference relies purely on embedding similarity without invoking an LLM. Experiments on SST-2, AGNews, MedAbs, and FinaQuery under a 10-shot setting show that StructCBM consistently outperforms prior CBMs such as TBM and CB-LLM, achieving performance close to black-box LLM baselines like DeepSeek-ICL, with the additional benefits of interpretability and computational efficiency.", "questions": "n/a", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- S1: The paper explores an underexamined intersection of interpretability and few-shot learning, where existing LLM-based CBMs have been ineffective.\n\n- S2: The two-stage architecture (prototypical vs. discriminative concepts) introduces a clear and interpretable reasoning process, akin to human recall–then–refine reasoning. The closed-loop refinement mechanism adds adaptivity and resilience in low-data regimes. Its non-parametric design avoids fine-tuning while maintaining transparency and efficiency.\n\n- S3: Experiments across diverse domains show consistent and significant gains over interpretable baselines. The ablation and sensitivity studies clearly isolate each component’s contribution. The qualitative case study effectively demonstrates interpretability and transparent reasoning.\n\n- S4: Inference is entirely LLM-free, yielding substantial computational savings compared to other LLM-augmented CBMs. Each prediction is explicitly grounded in human-readable concepts, supporting transparency and trustworthiness.", "weaknesses": "- W1: The pairwise Cd formulation scales quadratically with the number of classes, limiting applicability to large-label tasks. While acknowledged by the authors, there is no quantitative analysis of computational or memory overhead.\n\n- W2: Although inference is lightweight, concept generation and refinement heavily depend on LLM capabilities. The paper does not assess the impact of different LLMs (e.g., GPT-4 vs. DeepSeek-V3), leaving robustness across models unclear. \n\n- W3: The evaluation is restricted to 10-shot experiments, without studying performance trends as data increases. Some baselines (e.g., TBM, CB-LLM) may be under-optimized for few-shot scenarios, possibly skewing comparisons. Interpretability is validated only qualitatively, with no quantitative concept fidelity or human evaluation.\n\n- W4: The generate–predict–refine loop lacks analysis of computational cost, convergence, and stopping criteria, leaving uncertainty about deployment feasibility.\n\n- W5: The paper claims that code is released at an anonymous link, but the provided URL returns “The requested file is not found.”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p1S2yj1gVk", "reviewer": "Reviewer_GAko", "summary": "StructCBM is a novel Concept Bottleneck Model (CBM) framework designed to overcome the high inference costs and \"black-box\" nature of Large Language Models (LLMs) in few-shot text classification. StructCBM introduces a new CBM paradigm that relies solely on sample-concept similarity for prediction, enabling a lightweight, LLM-free inference process. Experiments show that StructCBM surpasses prior CBMs and achieves performance comparable to LLMs.", "questions": "1. Regarding the quadratic scaling of Cd, the paper suggests exploring \"on-demand concept generation\" for only the most confusable class pairs as a solution. How would StructCBM implement this on-demand concept selection or generation in a practical, real-world inference scenario? Does identifying the \"most confusable\" pairs inherently require a costly preliminary analysis that might undermine the overall efficiency gain?\n\n2.The Concept Tuning stage (generate-predict-refine workflow)—is critical for discovering higher-quality concepts. Yet, the ablation study showed that this refinement process negatively impacted performance on the semantically sparse SST2 dataset, which was hypothesized to be due to overfitting. So what practical criteria or internal heuristics were considered or could be developed to automatically detect when concept refinement, particularly the Semantic Tuning process (Algorithm 1), is beginning to induce overfitting on semantically poor data (like SST2)? Could a metric reflecting concept redundancy or semantic density be incorporated into the refinement trigger to ensure robustness across diverse datasets.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The StructCBM framework introduces a novel paradigm for Concept Bottleneck Models (CBMs) that addresses the primary limitations of Large Language Models (LLMs)—high inference costs and lack of interpretability—while overcoming the challenges faced by prior CBM approaches in few-shot learning scenarios. The method achieves performance comparable to powerful black-box LLMs. Notably, on semantically rich datasets like AGNews, StructCBM surpasses the 10-shot Deepseek-ICL baseline, and on MedAbs, its performance is comparable to the zero-shot Deepseek-Direct.", "weaknesses": "1. The paper clearly emphasizes that StructCBM's strength lies in its LLM-free, lightweight inference process. However, the pairwise formulation of Discriminative Concepts (Cd) scales quadratically with the number of classes, which is noted as a significant challenge for tasks with large label sets.\n\n2.The final step involves fine-tuning the embedding model e(⋅) to improve matching performance. Crucially, the fine-tuning data P (Equation 5) is constructed only using the prototypical concepts (Cp) and associated samples, explicitly excluding Cd.  More ablation study is suggested to compare performances when Cd was included in the fine-tuning set.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2Sa6qzX2l8", "reviewer": "Reviewer_mVk8", "summary": "This paper proposes an interpretable few-shot text classification framework based on a Concept Bottleneck Model (CBM) paradigm. The approach introduces a separation of concepts into prototypical and discriminative sets, and leverages Large Language Models (LLMs) to dynamically generate concept proposals for both. Furthermore, a two-stage prediction process is proposed, consisting of prototype pruning followed by re-ranking of discriminative concepts. The overall framework aims to improve interpretability and adaptability in few-shot scenarios.", "questions": "1. Can the authors provide examples to illustrate the distinction between prototypical and discriminative concept sets?\n\n2. How does the model handle unreliable or irrelevant concepts generated by the LLM, especially under few-shot conditions?\n\n3. Could the proposed method generalize to fewer samples (e.g., 1-shot or 5-shot)? If not, what modifications would be necessary?\n\n4. Minor: How many classes are there for each task used in the experiments?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Clarity and readability: The paper is well written and easy to follow, with a logical presentation of motivation, methodology, and results.\n\n2. Novel paradigm: The use of LLMs to dynamically generate and structure concepts into two interpretable sets (prototypical vs. discriminative) is interesting and aligns with the goal of interpretable few-shot learning.\n\n3. Interpretable design: The combination of concept separation and the two-stage inference mechanism provides a clear and interpretable reasoning process.", "weaknesses": "1. Over-reliance on LLM-generated concepts:\nThe proposed approach heavily depends on the reliability of concepts generated by LLMs. Given the limited few-shot data, the LLM-generated concepts and the closed-loop feedback mechanism may introduce noise or bias. The poor performance on SST-2 (Table 3) supports this concern.\n\n2. Limited evaluation scope:\nThe experimental comparison is restricted to only two white-box baselines. It would strengthen the paper to include more interpretable or concept-based baselines, especially those already listed in Table 1.\n\n3. Lack of discussion on few-shot text classification challenges:\nFew-shot text classification is inherently ill-posed, and LLMs are known to perform well in such settings even without explicit concept modeling. The experimental results from black-box methods reinforce this observation, but the paper lacks a sufficient discussion or justification on why the CBM approach is preferable in this context.\n\n4. Insufficient evaluation diversity:\nExperiments are conducted only on the 10-shot setting. Including 1-shot and 5-shot results would provide a more comprehensive understanding of the model’s robustness under varying data constraints.\n\n5. Unclear concept definitions:\nThe definitions of prototypical and discriminative concept sets remain somewhat abstract. Clearer explanations and concrete examples (e.g., illustrating how a sentence or phrase maps to each set) would greatly improve the reader’s understanding of how the model differentiates between the two concept types.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)