Hierarchical Concept-based Interpretable Models
Authors: Oscar Hill
Mateo Espinosa Zarlenga
Mateja Jamnik
Venue: ICLR 2026 Poster (2026)
Keywords: Explainable Artificial Intelligence
Concept-based Explainability
Concept Discovery
Concept Hierarchy
Concept Bottleneck Models
Concept Embedding Models
Clustering
Sparse Autoencoders
Primary area: interpretability and explainable AI

Abstract: Modern deep neural networks remain challenging to interpret due to the opacity of their latent representations, impeding model understanding, debugging, and debiasing. Concept Embedding Models (CEMs) address this by mapping inputs to human-interpretable concept representations from which tasks can be predicted. Yet, CEMs fail to represent inter-concept relationships and require concept annotations at different granularities during training, limiting their applicability.
In this paper, we introduce *Hierarchical Concept Embedding Models* (HiCEMs), a new family of CEMs that explicitly model concept relationships through hierarchical structures. To enable HiCEMs in real-world settings, we propose *Concept Splitting*, a method for automatically discovering finer-grained sub-concepts from a pretrained CEM’s embedding space without requiring additional annotations. This allows HiCEMs to generate fine-grained explanations from limited concept labels, reducing annotation burdens. 
Our evaluation across multiple datasets, including a user study and experiments on *PseudoKitchens*, a newly proposed concept-based dataset of 3D kitchen renders, demonstrates that (1) Concept Splitting discovers human-interpretable sub-concepts absent during training that can be used to train highly accurate HiCEMs, and (2) HiCEMs enable powerful test-time concept interventions at different granularities, leading to improved task accuracy.

Published as a conference paper at ICLR 2026
HIERARCHICAL CONCEPT-BASED INTERPRETABLE MODELS
Oscar Hill University of Cambridge ogh220cam. ac.uk
Mateo Espinosa Zarlenga University of Oxford t trim trin43800ox.0 accuk
Mateja Jamnik
University of Cambridge mj2 @cam. ac.uk
ABSTRACT
Modern deep neural networks remain challenging interpret due the opacity of their latent representations, impeding model understanding, debugging, and debiasing. Concept Embedding Models (CEMs) address this by mapping inputs to bumen-interpretable concept representations from which tasks can be predicted. Yet, CEMS fail to represent inter-concept relationships and require concept annota- tions different granularities during training, limiting their applicability. In this paper, we introduce Hierarchical Concept Embedding Models (HiCEMs), a new family of CEMs that explicitly model concept relationships through hierarchical structures. To enable HiCEMs in real-world settings, we propose Concept Split- ting, method for automatically discovering finer-grained sub-concepts from a pretrained CEM's 's embedding space without requiring additional annotations. This allows HiCEMs generate fine-grat graiggraied explanations from limited concept labels, reducing annotation burdens. Our evaluation across multiple datasets, including a user study and experiments on PseudoKitchens, newly proposed concept- based dataset of 3D kitchen renders, demonstrates that (1) Concept Splitting discovers interpiinterreetblee sub-concepts absent during training that can used to train highly accurate HiCEMs, and (2) HiCEMs enable powerful test-time concept interventions at different granularities, leading to improved task accuracy.
INTRODUCTION
State-of-the-art Deep Neural Networks (DNNs) can achieve very high task accuracies but fail to explain their reasoning in human-understandabbe terms (Barredo Arrieta et al., (020). Concept Embedding Models (CEMs) (Espinosa Zarlenga et al., 2022) address this limitation by learning to predict set of human-understanddbbe concept representations provided training time (e.g.. "size" or "colour"), and then using these concept representations (or embeddings) for learning to predict downstream task. Within this framework, CEM's concept predictions serve explanation for its downstream task prediction. However, CEMs cannot model relationships between concepts, treating all concepts independent entities from each other, leading to their representations failing to capture known inter-concept relationships (Raman et al., 2024). This is problematic because real- world concepts are often interrelated, and human cognition inherently utilises such relationships for reasoning (McClelland & Rogers 2003). Additionally, CEMs require expensive concept annotations training time to learn their embeddings (Espinosa Zarlenga al. 2022), limiting their usability.
While numerous researchers have explored concept discovery Yukkekgonul et al. 2023 Oikarinen al. 2023 Rao al.) 2024). they typically overlook the hierarchical relationships between dis- covered concepts. Additionally, these methods rarely support numan-in-the-looe refinement, where expert interventions, such as corrections concept predictions at test time, could improve model performance. These gaps limit their applicability to real-world scenarios where hierarchical concept structures and iterative human feedback are critical.
In this paper, we show that CEMs capture sub-concepts not provided during training as part of their embedding spaces. For example, a CEM trained with the concept "contains vegetables" may encode subspaces corresponding finer-grained sub-conceple like "contains onions" and "contains carrots" within its embedding manifold. To exploit this structure, we propose (1) Concept Splitring (Figure []] method for discovering sub-concepts from CEM's concept embeddings using sparse autoencoders

---

Published as a conference paper at ICLR 2026
Onion
Not "has vegetables'
+ Has vegetables:
Encode
Decode
Cucumber feature not active
Cucumber
Cucumber feature active
Train CEM, extract concept embeddings.
Train SAE the concept embeddings.
Create discovered concept labels from the SAE features.
Figure 1: Concept Splitting- (a) Train a CEM and calculate concept embeddings. (b) Train SAEs on the embeddings (the image depicts single embedding set). (c) Create concept labels. The green points are marked as having the new concept, and the black points (where the parent concept is not active where the SAE feature is not active) marked as not having the new concept.
(Bricken et al. 2023). and (2) Hierarchical CEM (HiCEM, Figures 2 and 3 model designed
to support hicrarchical concept relationships like those discovered by Concept Splitting. Together, Concept Splitting and HiCEMs significantly reduce annotation costs by requiring only coarse, high- level concept labels during training while automatically discovering more granular sub-concepts. The coarse concept labels could be obtained cheaply by using LLMs (Oikarinen al. [023] Yang et al. 2023) or by grouping semantically related classes. Our evaluation across several datasets, including user study, strongly suggests that Concept Splitting HiCEMs can then utilise to construct more fine tine-grained explanations for their predictions without compromising predictive or intervention performance. Hence, our contributions are:
discover human-interpretable concepts that
We introduce Concept Splitting, a method to discover sub-concepts in CEM's embeddings. This reduces the need for exhaustive concept annotations and enhances the granularity of explanations.
We propose HiCEMs, family of inherently interpretable concept-bass models that capture hierarchical concept relationships and support human interventions at multiple hierarchy levels.
We introduce PseudoKitchens, synthetic dataset of photorealistic 3D kitchen renders with perfect ground-truth concept annotations and precise spatial localisation.
We demonstrate, through empirical, qualitative, and quantitative experiments, including a study, that HiCEMs trained via Concept Splitting can accurately discover interpretable sub-concepts that were absent during training. Moreover, our experiments show that HiCEMs trained with Concept Splitting achieve competitive task accuracies and are receptive to test-time concept interventions at different granularity levels.
BACKGROUND AND RELATED WORK
Concept learning Concept-based methods aim to explain model's predictions using human- understandable concepts (e.g., "colour" or "size") (Bau et al. 2017 Fong & Vedaldi 2018 Kim al. 2018). Some methods, for example, Concept Bottleneck Models (CBMs) (Koh al. (020).
explicitly incorporate concepts in their architecture, leading to inherently interpretable models that provide concept-based explanations. These methods typically require a concept-annotate training set and may suffer from suboptimal predictive performance due to conflicting training objectives (Espinosa Zarlenga al. 2022). Furthermore, they often do not consider the relationships between concepts; instead, they treat all concepts independent variables (Havasi et al. (2022). Other methods attempt to construct post-hoc concept-based explanations, instead of producing inherently interpretable models. For example, Automatic Concept-based Explanations (ACE) (Ghorbani et al.) 2019) clusters a DNN's latent space to discover relevant concepts. contrast to ACE, we (1) use the discovered concepts to construct inherently interpretable models, (2) exploit the relationship between existing and discovered concepts, and (3) demonstrate effective discovered concept interventions.
Concept Embedding Models Concept Embedding Models (CEMs) (Eppnosa Zarlenga et al. 2022) provide concept-basee based explanations while achieving higher task accuracies than CBMs. CEMs. and their variants (Kim et al.) Essinosa Zarlenga et al. 2024] Xu et al. 2024 Espinosa Zarlenga et al. 2025). improve task accuracy by representing concepts using high-dimensional supervised
2

---

Published as a conference paper at ICLR 2026
vectors, embeddings. For each concept c, CEM learns two embeddings: one for when it is active (a "positive" embedding, é ), and another for when is inactive (a "negative" embeddingg è, Each concept embedding is aligned to its corresponding ground truth concept through scoring function which learns to assign an activation probability Pú to each concept c;. These probabilities are used to output embedding é, for each concept G, via mixture of positive and negative embeddings weighted by the predicted probability (i.e., ć; p, + (1 p. Jc, ). Finally, the mixed concept embeddings are concatenated into bottleneck è and passed to a label predictor f(ē) (usually a linear layer). This predictor outputs a downstream task prediction ý.
An important property of CEMs is that they support concept interventions: at test time, an expert can correct mispredicted concepts, allowing the model to update its downstream label prediction based on these corrections. Interventions can be performed by fixing a concept's ss embedding to the embedding that is semantically aligned with the ground truth concept label (e.g., setting é; e.- if concept oj is determined to be present in 2)). However, a major limitation of CEMs that they fail to capture known inter-concept relationships (Raman al. (024). deviating from how humans tend to reason about task hierarchically (McClelland & Rogers 2203). Moreover, lacking a mechanism for representing hierarchical inter-concept relationships means that CEMs need large number of concept labels in their training datasets to capture different concept granularities. Our work addresses these issues by exploiting a pre-traine pre-trained CEM's embedding space to discover sub-concepts that were not provided during training, and using them to train HiCEMs that can exploit sub-concept relationships.
Concept discovery Several methods (Bricken et al., 2023 Huang et al. 2022 Huben et al. 2024
O' O'Mahony et al., [023] |Ra et al. 2024] Vielhaben et al. 2023] Yuksekgonul et al.] 2023 Zhang et al.] 1222)) aim to identify human-interpretable concepts that can help explain how a model makes its predictions. Some methods try to discover concepts encoded by a model's neurons (Fel et al.
2023 Graziani et al. 2023 Oikarinen & Weng 2023 Panousis & Chatzis 2023), some ask large
language models to suggest relevant concepts ( (Oiaarinen et al. 2023] Yang et al. (002). and some look for meaningful features of inputs (Ghorbani et al.) 2209)). A few approaches attempt to assign concepts to individual neurons, however this can be problematic, as neurons often represent complex, uninterpretable features (Elhage et al. 2022). However, most of these works typically fail to evaluate interventions on discovered concepts or to consider the relationships between discovered concepts. For example, hierarchical relationships among concepts imply that some sub-concepts (e.g.. "contains onions" and "contains carrots") " can only exist in the presence of another parent concept (c.g.. "contains vegetables"). Therefore, here we present a method for discovering such sub-concepts and propose an inherently interpretable architecture that explicitly represents these sub-concept relationships. Furthermore, we demonstrate that interventions on discovered sub-concepts are effective.
Modelling concept relationships Most early concept-based models unrealistically assumed inter- concept independence. More recent works model inter-concept relationships through autoregressive architectures (Havasi et al. 2022), causal graphs (Dominici et al. 2024), probabilistic methods (Van- denhirtz al., 22044] Xu et al. 2024), or ceervention-speciin mechanisms (Singhi et al., 2024). However, these capture interactions without explicitly organising concepts into hierarchies or discov- ering sub-concepts. A related direction by Panousis et al. (2024) models hierarchical concepts but uses spatially- groundded approach (high-level for whole images, low-level for patches) dependent on vision- laniuage models and textual concept descriptions. In contrast, we discover hierarchies by decomposing concept embeddings, independent of spatial structure or vision-language constraints. To enable this, we introduce Concept Splitting, a method to automatically discover latent sub-concepts from a model trained with only high-level labels, and HiCEMs, an architecture that explicitly models these hierarchical relationships to enable fine- fine-graines grained explanations and multi-level interventions without exhaustive annotations.
3 CONCEPT SPLITTING
Previous work has found that Sparse AutoEncoders (SAEs) can uncover interpretable concepts in the representation spaces of neural networks (Bricken et al. 2023 Bussmann et al.) 2024). SAEs a type of autoencoder trained reconstruct their input while enforcing sparsity constraint high-dimensional latent representation, effectively learning a dictionary of (hopefully interpretable) features. We apply this idea in CEMs to discover sub-concepts, and then we train HiCEMs (Section
3

---

Published as a conference paper at ICLR 2026
that use the discovered concepts to provide finer-grained explanations. Specifically, sparse autoencoders (Bussmann et al. 2024), which keep the top activations across a batch. While we focus using SAEs to discover concepts, they not the only option, and Appendix A discusses an alternative approach using clustering to find mutually exclusive sub-concepts.
BatchTopK
Our approach, which we run M on concept-annotated training set D {(x() c()), ("),))""))
Concept Splitiing (Figure D. takes input trained CEM M. First,
storing the concept embedding vectors and concept predictions {ë,, p. for each sample in D and for each concept want to split (Figure [ [))) For simplicity, we describe how to split a single concept C₁. However, this operation be performed for all training concepts.
Next, let E; be the set of embedding vectors for Gi and let E and E.fase be the subsets of embeddings in E, where c; was predicted by M to be present (i.e., p, 0.5) or absent, respectively. We partition the embedding vectors using M concept predictions for c,, as these predictions tell us whether the dominant component in the mixture is a positive embedding negative embedding.
Here, we want to discover sub-concepts of G, where we consider the sub-concepts of c; to be groups of concepts that are either only active when c; (positive sub-concepts), or only active when o is not (negative sub-concepts). To this end, we train SAEs on Lttue and Lt separately (Figure ()). Using the SAE trained on Litrue, we discover sub-concepts that are present when c2 also present, and using the SAE trained Exfalse we discover sub-concepts that present when G is not.
Once we have trained a SAE on an embedding set, we create new concept labels using the features learned by the SAE (Figure[Tc) Bussmann al. (024) describe how to calculate threshold for determining when feature is active during inference. Every feature can be treated as a discovered sub-concept. The examples that activate the feature are marked as having the new sub-concept, and the examples that do not activate the feature are marked as not having the new sub-concept.
Once Concept Splitting has been performed, we can interpret a discovered sub-concept using proto- types, which provide training examples that strongly activate the concept. This approach, similar to that used by previous works (Alvarez Melis & Jaakkola 2018: Espinosa Zarlenga et al. 2023 Yeh
et al. 2020). enables experts to assign potential semantics to discovered concepts.
4 HIERARCHICAL CEMS
We introduce the HiCEM architecture (Figure 2 which explicitly models hierarchical relationships between concepts. For simplicity, we focus on two-level hierarchies, however our architecture could be extended to support deeper hierarchies.
4.1 ARCHITECTURE
Like in CEMs, for each top-level concept, HiCEM learns mixture of two embeddings with semantics representing the concept's s activity. In HiCEM, each top-level concept is represented with the embeddings 9 e,- E RTT. Here, &t represents e, 's active state, and é, represents its inactive state. contrast to CEMs however, we also want 6 and 6, to contain information about oj positive and negative sub-concepts, respectively. To achieve this, a backbone network (x(x) (e.g.. pre-trained ResNet model) produces a latent representation
mi which is the input to top-level embedding generators of and o, These top-level embedding generators produce intermediate embeddings ey' o, (h), 6,' o, (h) E m Following the work of Espinosa Zarlenga et al. (2022). we implement the tep-level embedding generators as single fully connected layers.
To produce final concept embeddings that contain information about sub-concepts, the embeddings att a and à.' are passed through positive and a negative sub-concepts module, respectively. The positive sub-concepts module, which we describe in further detail below, is responsible for learning the positive sub-concepts of q. It outputs the positive concept embedding for c; a as well as the probability of its most likely positive sub-concept, p. Similarly, the negative sub-concepts module outputs the negative concept embedding for c,, o,_ as well as the probability of its most likely negative sub-concept, p.- If concept e; has no positive sub-concepts (i.e., concept C; is a leaf node in the hier- archy), then we take ट = at and 인. = s(ēt)) ). where s is a shared scoring function that calculates concept probabilities from concept embeddings. We proceed analogously in the absence of negative
4

---

Published as a conference paper at ICLR 2026
"vegetables"
b (h
Positive sub-concepts module
Mean
Φ (h.
Negative sub-concepts module
000000
Input
f(e y
(xx)
"fruit"
d (b)
Positive sub-concepts module www.eeeeeee
2
Mean
dx (h)
Negative sub-concepts module
Figure 2: Hierarchical CEM: in CEM, from latent code h, we learn two embeddings per concept (e," and e. These embeddings are then passed through sub-concepts modules (Figure 3 which produce new embeddings (e and ? that include information about sub-concepts. The sub- concepts modules also output the most likely sub-concept probabilities, which are used to calculate top-level concept probabilities. These probabilities are used to output an embedding for each concept via weighted mixture of positive and negative embeddings.
sub-concepts. P can be taken an estimate for the probability the top-level concept c;, because oa can only be present if one of its positive sub-concepts is. Similarly, the complement p, can be taken as another estimate for the probability of The predicted probability of concept c, P.. is calculated as the average of p and the complement of p p; 1/ (p) embedding è for c is calculated as a weighted mixture of ट and ट.: è; = p -11 - p.)0,
p; ). As in CEMs, the final concept
Like in previous embedding based concept models (Espinosa Zarlenga et al.) 2022 2023
Xu et al..
2024), before making a task prediction, all mixed concept embeddings are concatenated, resulting in bottleneck g(x) = è with k units. This is put through label predictor to get downstream task label. Following previous work (Koh et al. 2020 Espinosa Zarlenga et al.) 2022). ). we use an interpretable label predictor f parametrised by simple linear layer.
As in previous concept-based models, HiCEMs provide a concept-based explanation for the predicted downstream task label through predicted concept probabilities p(x) & [p...... ..e] However, unlike previous architectures, HiCEMs explicitly model the relationship between concepts and sub-concepts: a concept's 's positive embedding contains information about its positive sub-concepts, and a concept's negative embedding includes information on its negative sub-concepts.
4.2 SUB-CONCEPTS MODULES
We describe a positive sub-concepts module, as illustrated in Figure but negative sub-concepts modules operate in exactly the same way. Inside the positive sub-concepts module for concept ck. sub-concept embedding generators ot, produce embeddings for each of c 's positive sub-concepts:
& o (e Like the top-level embedding generators, these are implemented as single fully connected layers. Similarly CEMs (Espinosa Zarlenga et al., (022), these embeddings aligned with ground-truth sub-concept ctty via learnable and differentiable scoring function 8: IR + [0, 1], trained to predict the probability Ptz A s(Cit) a (W,at, + b.) of sub-concept cty being active from its cobecone embedding. As in CEMs, the scoring function is shared across all sub-concepts. C 's positive embedding, èt is constructed as weighted mixture of all the n positive sub-concept
embeddings: Oz A mi ot
5

---

Published as a conference paper at ICLR 2026
P
Positive sub-concepts module
Soft maximum
containn apples" 1hi
PA
s(-)
contains pears*
(0
king
s(-)
Figure 3: Positive sub-concepts module: positive sub-concept embedding generators o (illustrated for "contains apples" and "contains pears") produce sub-concept embeddings from the preliminary parent concept embedding et" A shared scoring function s(.) predicts probabilities for each sub- concept. The positive parent concept embedding & is computed weighted mixture of sub-concept embeddings, whilst estimate for the parent concept probability pit obtained via differentiable soft maximum operation over the sub-concept probabilities.
The highest positive sub-concept probability p which is estimate for the probability of ck. is calculated in a differentiable way as P = ΣE so coftmax(a: p - 3), Pky. where (p = P and the constants a and 8 scale the input of the softmax so that its output strongly weighted towards the largest Pky- In practice, we use 200, 3 100 to scale the probabilities from [0, 1) to 100, 100). so that the calculated highest sub-concept probability is very close to the true maximum of the probabilities.
4.3 TRAINING
HiCEMs are trained by jointly minimising a weighted sum the cross- loss on both task prediction and concept predictions: L A E(x.y.e) [Lusk (y. (g(())) + AC (c, pxx))] Here, the hyperparameter À E R+ controls the relative importance of concept and task accuracy.
When training HiCEMs, et al. 2022) to improve intervention effectiveness. That is, during training, we randomly perform independent concept interventions with probability Pint
the RandInt regularisation strategy proposed by Espinosa Zarlenga
4.4 CONCEPT INTERVENTIONS
HiCEMs support interventions on both top-level concepts and sub-concepts natural way. Similarly to CEMs (Espinosa Zarlenga et al. 2022), to intervene on top-level concept the embedding è; is updated by replacing with the output embedding semantically aligned with the concept's ground truth label. This intervention ensures that é; contains information about the relevant sub-concepts.
Intervening on sub-concept c differs depending on whether the human expert decides o or is not actually present in the example. (While we focus on positive sub-concepts here, the analogous process applies negative sub-concepts.) the human expert determines that ct not present, then the model's predicted probability p is simply set to zero. On the other hand, if the expert identifies one, and Pty- is
o present, then f.j Additionally, because the presence of ctj implies that the top-level concept e; is present, we also intervene G. Therefore this operation results in the update é; c 67-
zero for all j' (this means o, ct))

---

Published as a conference paper at ICLR 2026
5 EXPERIMENTS
We evaluate Concept Splitting and HiCEMs by exploring the following research questions:
RQ1 Does Concept Splitting discover interpretable sub-concepts?
RQ2 How do HiCEMs' task and provided (top-level) concept accuracies compare to those of the original CEM and other baselines?
RQ3 Can a HiCEM's task accuracy improved by intervening on discovered sub-coneepts?
5.1
PSEUDOKITCHENS
To rigorously evaluate concept- models, we introduce PseudoKitchens (Appendix B) synthetic dataset of photorealistic 3D kitchen renders with perfect ground-truth concept annotations. Our approach provides complete control over scene generation and pixel-perfect labels for all concepts.
5.2
SETUP
We evaluate our methods across six diverse datasets: MNIST-ADD (based on Cun et al. (2010). a procedural SHAPES dataset (similar to Matthey et al. (2017)), Caltech-UCe 200-2-200-2011 (CUB) (Wah et al.) 2011), Animals with Attributes (AwA2) (Xian et al.] 2019). our new PseudoKitchens dataset, and ImageNet (Russakovsky et al. 2015). These datasets span simple synthetic tasks, fine-grained visual classification, and large-scale recognition, providing a comprehensive testbed. Full details on each dataset, including concept definitions and splits, are provided in Appendix
Datasets
Metries For each dataset, we run Concept Splitting on the provided concepts in an initial CEM, and then train HiCEM with the provided top-level concepts and the discovered sub-concepts.
Following Espinosa Zarlenga et al. (2023), we evaluate discovered sub-concept accuracy and perform interventions by automatically pairing sub-concepts with human- -umman--dddrttaddablle "left-out" concept from a predefined "concept bank". This bank contains anticipated concepts excluded during initial CEM training (e.g.. "the first digit 6" in the MNIST-ADD dataset), with each concept pre-associated with the parent concept whose sub-concept it might represent. To align discovered sub-concepts with the bank, we compute the area under the receiver operating characteristic curve (ROC-AUC) scores between the discovered sub-concept labels and their potential parent-concepl-associate the bank. Each concept in the bank is assigned to the sub-concept with the highest ROC-AUC score, as long that score is greater than 0.7. Discovered sub-concepts that are not matched with a concept bank concept are not included in the evaluation. We do not have ground-truth concept annotations for ImageNet (Russakovsky et al.) 2015), Discovered sub-concepts that were not matched to the concept bank, or that were not selected for the user study in the case of ImageNet, are not included when we train our HiCEMs.
matches in
evaluate the discovered sub-concepts with user study.
To answer RQI, we present the results of our user study conducted with ImageNet, and for the other datasets, we report the average discovered sub-concept ROC-AUC of the HiCEM. To address RQ2, we report the task accuracy and the provided concept ROC-AUC of the initial CEM and the HiCEM. For RQ3, we measure the change in HiCEMs' task accuracies as concepts are intervened. All metries in our evaluation, apart from those calculated on ImageNet, are computed on test datasets using three random seeds, from which we compute mean and standard deviation. Because ImageNet large, we compute metrics using a single random seed. Whenever we measure concept accuracy, we use the mean concept ROC-AUC to avoid being misled by a majority-class classifier.
Baselines We compare HiCEMs with black box models, CEMs (Espinosa Zarlenga al. 2022). CBMs (Koh et al., 2020). Label-free CBMs (LF-CBMs, (Oikarinen et al. 2023)) Post-hoc CBMs (PCBMs, (Yuksekgonul et al. 2202)), ), and PCBMs with residual connections (PCBM-hs), Our black box models have the same architecture our CEMs, but without any concept supervision. To evaluate the usefulness of the discovered concept labels produced by Concept Splitting. each our runs we train control HiCEM that has the same architecture the HiCEM with discovered sub-concepts, but no sub-concept supervision ("HiCEM w/o concept splitting"). That is, the HiCEM is trained with access lep-l concept labels but without any labels supervision for sub-concepts. We
7

---

Published as a conference paper at ICLR 2026
Table 1:
Mean ROC-AUC for discovered concepts. LF-CBMs were unable discover concepts on the PseudoKitchens dataset. Sub-concepts discovered with Concept Splitingg are predicted accurately.
MNIST-ADD
SHAPES
CUB
AwA2
PseudoKitchens
LF-CBM
0.75_10.00
0.7710.00
0.7810.00
HiCEM w/o concept splitting (control)
1..01
0,88 0..02
0.81 0.81 10.00
1.7300000
0.75_0.06 _0006
CEM + Concept Splitting (ours)
0.93
0.98
0.85.o.m1
0.88
0.
HiCEM Concept Splitting (ours)
0.93 0.03
0.98
0.85
0.88
0.88
User study results. Users are far more likely to say sub-concepts generated by our method are examples of their parent concept than randomly chosen words are (first two rows). Users also agree that, in a lot of cases, the images labelled as having discovered sub-concept are consistent with the automatically generated name of that sub-concept (second two rows).
Table 2:
Yes
No
Not sure
They are the same
Sub-concept relationship (control)
16 (3.8%)
387 (91.3%)
20 (4.7%)
1 (0.2%)
Sub-concepi relationship (experimental)
241 (60.6%)
109 (27.4%)
19 (4.8%)
29 (7.3%)
Image labels (control) Image labels (experimental)
4 (0.9%)
424 (97.9%)
5 (1.2%)
244 (54.8%)
172 (38.7%)
29 (6.5%)
match the "sub-concepts" in this bascline to concept bank using sub-concept predictions the training dataset. We also train a CEM with top-level concepts and discovered sub-concepts we can compare this to the HiCEM. The CUB dataset is used by Oikarinen et al. (023) to evaluate LF-CBMs, so we take their discovered concepts and manually match them with concepts for which we have ground-truth labels. For the other datasets, and for all the datasets with the PCBM baseline, we use the names of concepts for which we have ground-truth labels to create the models. For further details on how we train each of these baselines, including the architectures and hyperparameters used, see Appendices D and E Due to the computational cost of training models on ImageNet, we do not include results for all baselines on it.
("CEM + Concept Splitting")
User Study To evaluate the sub-concepts discovered on ImageNet, we ran a user study. We automatically named discovered sub-concepts with the CLIP vision-language model (Radford et al. 2021), using a method similar to previous work (Rao et al. 2024). We then asked users whether the discovered sub-concept names were semantically related the name of their parent concept ("Is [sub-concept name] an example of [parent concept name)?"). As a control we picked words at random from the dictionary used to name the sub-concepts. We also asked users whether images labelled by Concept Splitting as having the discovered sub-concepts were consistent with the sub-concept names ("Does this image show [sub-concept name]?"), and used images selected random from the dataset as control. Full details are contained in Appendix F
5.3 RESULTS
Discovered concepts are human-interpretable and can be predicted accurately (RQ1, Ta- bles [nd We report the accuracy of the discovered concept predictions made by our models using the ground truth labels of the corresponding human-interpretable concept bank concepts on the test datasets. For example, the meaning assigned to one of the concepts discovered on the MNIST- ADD dataset was "the top digit is 6", and therefore we compute the accuracy of this discovered concept with respect the ground-trut truth labels of the concept "the top digit 6" in our concept bank. In Appendix ered concept, to demonstrate how is straightforward to interpret it. [ discovered sub-concept ROC-AUCs are high, exceeding 0.9 in some cases. The mean discovered concept ROC-AUC of HiCEMs always higher than that of both LF-CBMs and HiCEMs trained without sub-concept supervision (HiCEM w/o concept splitting in Table [], showing that the labels produced by Concept Splitting align the sub-concept activations in HiCEMS with human-interpretabll concepts. CEMs with discovered sub-concepts and lev-l concepts side by side have similar discovered concept accuracies to HiCEMs after Concept Splitting, however sub-concept interventions in HiCEMs can work better than in CEMs (Figure 4 discussed later).
G we show samples from the training dataset that were labelled as having this discov-
that the mean
8

---

Published as a conference paper at ICLR 2026
Table 3: Task accuracies. The task accuracy of HiCEMs is competitive with all our baselines. indicates reported accuracy, with ResNet-50 backbone.
MNIST-ADD
SHAPES
CUB
AwA2
PseudoKitchens
ImageNet
Black box (not interpretable)
4.94
0.8920.00
0.80
0.98
0.67
0.77
LF-CBM
0.5920.01
0. 0.8020.00
0.94
0.72
PCBM
0.16
0.54g0.01
0.65
0.95=0.00
0.16:0.00
PCBM-h
0.53.10.0
0.73.0.00
0. 0.73
0.96
0.52
CBM
0.23:40.01
0.7820.01
0.65
0.97 =0.00
0.68
0.77
CEM
0.92
0.89 20.00
0. 0.70
0.98 oe
0.66
0.79
HICEM w/o concept splitting (control)
0.93
0.8820.01
0.77
0.98
0.61
CEM + Concept Spliting (ours)
0.93 01
0.87=0.01
0. 0.700
0. 19 oo
0.66 0.66=2.02
HICEM Concept Spliting (ours)
0.92
0.87go.02
0.74g.0.0
0. 0.98 oo
0.65
0.78
Table 4: Mean ROC-AUCs for provided concepts. HiCEMs are able to predict provided concepts just well CEMs.
MNIST-ADD
SHAPES
CUB
AwA2
PseudoKitchens
ImageNet
CBM CEM
0.09
1.00_a.00
0.89
1.00
0.91
0.90
1.00
0.95
1.00
0.92
1.00
HiCEM w/o concept splitting (control)
0.99=0.00
1.00=0.00
0.93=:0.0
1.00
0.91 +0.00
CEM Concept Splitting (ours)
0.09
1.00
0.95
1.00
0.92
HiCEM + Concept Splitting (ours)
0.99
1.00
0.93
1. 1.00
0.91
0.99
Our user study on ImageNet (Table conducted with 20 participants, shows that the sub-concepts discovered by our method are both semantically coherent and accurately labelled. When evaluating the names of sub-concepts generated by Concept Splitting, participants found them to be semantically related their parent concepts 67.9% of the time, a dramatic increase over the 4.0% agreement rate for randomly chosen words in the control group. Furthermore, users confirmed that images labelled by our method were consistent with the discovered sub-concept name in 54.8% of cases, far exceeding the 0.9% agreement for the control. A Chi-Square test (where discard "Not sure" and group "Yes" and "They are the same" together) confirms that both of these improvements over the control groups are statistically significant (p 0.01).
To assess the robustmess of Concept Splitting across different embedding representations, we further explore an idealised setting in Appendix H where concept embeddings contain perfect one-hot information about sub-concepts. Here, Concept Splitting recovers the encoded sub-concepts almost perfectly. This suggests that method can be applied arbitrary concept embedding spaces long as they encode sub-concept information: is not dependent on the embedding structure of CEMs.
We measure the task and provided (top-level) concept accuracies of HiCEMs and our baselines. The results are in Tables 3 and 4 HiCEMs achieve both high task accuracy and high provided concept accuracy, compared to the baselines. In particular, the task and provided concept accuracies of HiCEMs are never more than 2% below those of CEMs, so running Concept Splitting and replacing CEM with a HiCEM that supports more detailed explanations does not lead to a reduction in task or provided concept accuracy. Overall, the effect of Concept Splitting on the task and provided concept accuracy is insignificant, so the additional interpretability offered by the discovered sub-concepts does not come the cost of accuracy.
HiCEMs have high task and provided concept accuracies (RQ2, Tables 4
Intervening on sub-concepts identified through Concept Splitting can enhance task accuracy, with these interventions sometimes yielding even greater improvements in HiCEMs compared to CEMs. (RQ3). We investigate how intervening on provided and discovered concepts affects task accuracy in HiCEMs and relevant baselines. Figure 5 demonstrates that provided concept interventions perform equally well in HiCEMs as in CEMs.
As shown in Figure intervening on discovered sub-concepts can lead to an increase in task accuracy, although interventions on some discovered sub-concepts have no effect or very slightly decrease task accuracy. Interventions in HiCEMs trained with sub-concept labels from Concept Splitting tend to increase task accuracy, whereas interventions in HiCEMs without sub-concept supervision (our control) can decrease it, highlighting the value of Concept Splitting. On the CUB and PseudoKitchens

---

Published as a conference paper at ICLR 2026
MNIST-ADD
SHAPES
CUB
AwA2
PseudoKitchen
Task accuracy
00000000000
0.9
0.8
0.98
0.7
0.6
0.8
0.6
0.65
0.4
0.97
0.4
2
6
0
5
10
0
75
150
10
2
8
Number intervened Number intervened Number intervened Number intervened Number intervened
CEM + Concept Splitting
HiCEM w/o concept splitting
HiCEM Concept Splitting
Figure 4: Task accuracy discovered concepts are intervened. Intervening discovered sub- concepts improves task accuracy. In some cases, such as CUB and PseudoKitchens, interventions in HiCEMs lead to greater increase in task accuracy than in CEMs trained with Concept Splitting's discovered concepts. LF-CBMs (Oikarinen et al.) (023) do not easily support interventions.
MNIST-ADD
SHAPES
CUB
AwA2
PitceoKitchens
Task accuracy
0.8
0.8
0.8
0.99
0.6
0.6
0.8
0.4
0.4
0.6
0.98
0.2
0.2
0.4
0.97
0.6
0
1
2
0
2
4
0
10
20
30
0
10
20
0
10
20
Number intervened Number intervened Number intervened Number intervened Number intervened
CEM
CBM
HiCEM + Concept Splitting
Figure Change in task accuracy as provided concepts are intervened. Provided concept interventions on ImageNet are shown in Appendix ! as they do CEMs.
Provided concept interventions work just as well in HiCEMs
datasets, sub-concept interventions HiCEMs are more effective than the equivalent interventions in CEMs with both top-level concepts and discovered sub-concepts, supporting the use of the HiCEM architecture over the regular CEM architecture when we have hierarchical concept relationships. HiCEMs with discovered sub-concepts support both the fine-grained interventions enabled by these sub-concepts and the broader concept interventions available in standard CEMs.
LIMITATIONS AND CONCLUSION
In this work, introduced Concept Splitting and HiCEMs to enable the discovery and modelling of hierarchical sub-concepts in interpretable models. Our experiments, which use our new Pseu- doKitchens dataset and include user study, show that Concept Splitting discovers interpretable sub-concepts. HiCEMs incorporating these concepts provide fine-grained explanations while requir- ing few manual concept annotations, without sacrificing task or provided concept accuracy. After they have been introduced, discovered sub-concepts can be predicted with high accuracy, and human expert can correct sub-concept mispredictions. One limitation of our approach that SAEs are not guaranteed to discover meaningful concepts. Additionally, we only looked at modelling sub-concept relationships. Future work could explore extending our approach to encompass deeper hierarchies. It would also be valuable to investigate, perhaps through user study, whether offering more fine- mne-grained intervention options is beneficial to a user's s cognitive load when interacting with the model. Nonetheless, capturing the hierarchical structure present in the concept embeddings of CEMs helps to fill a gap in earlier concept- architectures and represents a meaningful step forward in concept-based interpretable modelling.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our research aims to enhance the transparency and accountability of neural networks. We have taken care to address ethical considerations related our experiments. The user study (Appendix F)) was conducted under institutional ethics approval, with all participants providing informed consent and their data being fully anonymised. Our new PseudoKitchens dataset is entirely synthetic, containing no personally identifiable information, and all real-world datasets are public benchmarks used in accordance with their licenses.
REPRODUCIBILITY STATEMENT
We have made a comprehensive effort to ensure the reproducibility of our results. Our source code, which includes implementations of our HiCEM architecture and the Concept Splitting method, has been released in a MIT-licensed public repository The architectural details of HiCEMs are described in Section 4 and our Concept Splitting method is detailed in Section 3 with an alternative clustering- based method in Appendix A, Experimental settings are specified in Appendices D [ Details regarding our new PseudoKitchens dataset are Appendix B Finally, the methodology for our user study is provided in Appendix F
ACKNOWLEDGMENTS
This work was supported by the Engineering and Physical Sciences Research Council [EP/Y030826/1) significant portion this work was carried out whilst MEZ was at the University of Cambridge, funded by the Gates Cambridge Trust via Gates Cambridge Scholarship.
REFERENCES
David Alvarez Melis and Tommi Jaakkola. Towards robust interpretability with self-explaining neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Desa-BBancchi, and
R. Garnett (eds.), Advances Neural Information Processing Systems, volume 31. Curran Asso- ciates, Inc., 2018. http:httts:::::/eeeeeettt 201 2018/f1 2218/fff14aSsff00000222222222222
meurips.co..o wwwpprrr..pppppppppppppppppp.... fi Files lesl /eppp/per/
Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Ben- jamins, Raja Chatila, and Francisco Herrera. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion, 58:82- 115, 2020. ISSN 1566-2535. doi: http:://ao..ng//000..000ff.fmff.
URL https!
7 w/www. www.wwwcceecciicccccm rect http://www.........a......oaaaaa
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dis- section: Quantifying interpretability of deep visual representations. In 20/7 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), pp. 3319-3327, Los Alamitos, CA, USA, 07 2017. IEBE Computer Society. doi: 10. 11110//CVP..201...333 URL https . www..eeemeeteetssccc.e........e. wtt.:///.111111///11.. 110.//11.//...... 2017
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zae HatfieldDDodss Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023, https//ttansformee-cccuitts http://ree.i.iiiioo/ooo/mooooooe features/index.hie html.
Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders. In NeurlPS 2024 Workshop on Scientific Methods for Understanding Deep Learning, 2024. URL. https
7/ //ppannweeesseeee/oooommaaaaadap

---

Published as a conference paper at ICLR 2026
Gabriele Dominici, Pietro Barbiero, Mateo Espinosa Zarlenga, Alberto Termine, Martin Gjoreski, Giuseppe Marra, and Marc Langheinrich. Causal concept graph models: Beyond causal opacity in deep learning. arXiv preprint arXtv:2405.16507, 2024.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac HatfieldDDodss, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition. Transformer Circuits Thread, 2022. https:/transformer- httpub/200022oyommm///mmeeooooo
Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, Pietro Lió, and Mateja Jamnik. Concept embedding models: Beyond the aecuracy-exxpaainablie In Advances in Neural Information Processing Systems, volume 35, pp. 21400-21413, Curran Asso- ciates, Inc., 2022. URL
off.
www.::/iiiinnnrreree...........e ppppr//pppppppppppppppeee/e/pppp pdf
Mateo Espinosa Zarlenga, Zohreh Shams, Michael Edward Nelson, Been Kim, and Mateja Jamnik. Tabebm: Concept-basee based interpretable neural networks for tabular data. Transactions on Machine Learning Research, 2023.
Mateo Espinosa Zarlenga, Katie Collins, Krishnamurthy Dvijotham, Adrian Weller, Zohreh Shams, and Mateja Jamnik. Learning to receive help: Intervention-awan concept embedding models. Advances in Neural Information Processing Systems, 36, 2024.
Mateo Espinosa Zarlenga. Dominici Gabriele, Barbiero Pietro, Zohreh Shams, and Mateja Jamnik. Avoiding leakage poisoning: Concept interventions under distribution shifts. In International Conference on Machine Learning (ICML), 2025.
Thomas Fel, Agustin Martin Picard, Louis Béthune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène, and Thomas Serre. CRAFT: concept recursive activation factorization for explainability. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pp. 2711 2711-2721. IEEE, 2023. doi: 10.1 199CCVP55222 2023.00266. URL httph:: doi. org/ 10. 109/ CVPR//72222222/ 2023. 2023.00266
Ruth Fong and Andrea Vedaldi. Net2vec: Quantifying and explaining how concepts are encoded by filters in deep neural networks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7730-8738. Los Alamitos, CA. USA. 06 2018. IEEE Computer Society, doi: 10.1109/CVPR.2011 00910. URL http:://oo....eccmmmpreroooeeeeo
1109/CVPS 201 1100000000 00910
Artyom Gadetsky, Yulun Jiang, and Maria Brbić. Let go of your labels with unsupervised transfer. In International Conference on Machine Learning, 2024.
Amirata Ghorbani, James Wexler, James Y Zou. and Been Kim. Towards automatic concept- based explanations. In Advances in Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc., 2019. 2019 http://ee.7772222c11311136333332
wwwrreuuri.... cc/paper_ sillespappe
Mara Graziani, An phi Nguyen, Laura O' O'Mahony, Henning Müller, and Vincent Andrearczyk. Concept discovery and dataset exploration with singular value decomposition. In ICLR 2023 Workshop on Pitfalls of limited data and computation for Trustworthy ML, 2023. URL https
//openrevies net./ forum? 01 YmD Pt.C
Marton Havasi, Sonali Parbhoo, and Finale Doshi- Velez. Addressing Leakage in Concept Bottleneck Models. In Advances in Nearal Information Processing Systems, 2022.
Haiyang Huang, Zhi Chen, and Cynthia Rudin. Segdiscover: Visual concept discovery via unsuper- vised semantic segmentation, 2022. UR//Epppee//aaxaiv..rr/////////2 10996
Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2024. URL https: http:s:://ppeeeeeeee net. f id=F7 7kww51.e
12

---

Published as a conference paper at ICLR 2026
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Proceedings of the 35th International Conference Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 2668-2677 PMLR, 07 2018. URL https::/pproeeei:: wtt.ggg..................
ml press vv00 1m1 8d. html
Eunji Kim, Dahuin Jung, Sangha Park, Siwon Kim, and Sungroh Yoon. Probabilistic Concept Bottleneck Models. arXiv preprint WxXiv2336.001744 2023.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.). 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL //arxiva www.ivv.rrgbbbbb//eeboo
6980
Pang Wei Koh, Thao Nguyen, Yew Siang Tang. Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 5338-5348. PMLR, 07 2020. URL https: http:::://ppccceeinngeoooo
mlr.. www.s.ppess/eeee bexees/vi19/ee www.///////ha..... html
Christoph Lampert. Daniel Pucher, and Johannes Dostal. Animals with attributes 2, 2017. URL https://cves //cm] i sta at AwA2/
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. Online, 2010. URL http://yann, www.nn.eccccc.cmm com//nccom///ccc www.coom//oee//ee/.oo /mnist
Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Dis-
entanglement testing sprites dataset, 2017. URL https: com/ hemm//deemiiin nd/
dsprites-datasee
James L. McClelland and Timothy T. Rogers. The parallel distributed processing approach to semantic cognition. Nature Reviews Neuroscience, 4(4):310-322, 04 2003. ISSN 1471-0048. doi: 10.1038/mm1076.
Tuomas Oikarinen. Label-free bottlneec
bottleneck models, 2023. URL BRLLttts://githbb.om
Trustworthy LLL-LLalaabel-ee- free CBM
Tuomas Oikarinen and Wei Weng. CLIP-dissect: Automatic description of neuron repre- sentations in deep vision networks. In The Eleventh International Conference on Learning Representations, 2023. URL (ttthhtp:://// httpps:://peeeeerieee wttp://wwww.eee.oooiiiiiiiiiiiie
Tuomas P. Oikarinen, Subhro Das, Lam M. Nguyen, and Tsui-Wei Weng. Label-free concept bottleneck models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL http:s::/opeeeeeeeit net/pdf?id=Fi Cg4 7MNvBA
Maxime Oquab, Timothée Darcet. Théo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khali- dov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojeiech Galuba, Russell Howes, Po- Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features with- out supervision. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL Featured Certification.
https:://pennrevieys net forum id a68 6zFt
Laura O'Mahony, Vincent Andrearczyk, Henning Müller, and Mara Graziani. Disentangling neuron representations with concept vectors. In 2023 IEEE/CVF Conference on Computer Vision and Puttern Recognition Workshops (CVPRW), pp. 3770-3775, 2023. doi: 10.1109/CVPRW:59228..
2023.00390.
Konstantinos P. Panousis and Sotirios Chatzis. DISCOVER: Making vision networks interpretable via competition and dissection. Seveth- Conference on Neural Information Processing Systems, 2023. URL hhtpps: http:::://peeeeiieeeeeeee/e//eee
/ htwp:e/wwwwweeeeeoooeoioooeeeeee http:///oorrumt.omsssoootttt
13

---

Published as a conference paper at ICLR 2026
Konstantinos P. Panousis, Dino lenco, and Diego Marcos. Coarse-to-fine concept bottleneck models. In The eight Annual Conference on Neural Information Processing Systems, 2024. URL atttps://opente
net/ PoorummiddRMdd.....
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach Veti,o, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: an imperative style, high--peformance deep learning library. Curran Associates Inc., Red Hook, NY, USA, 2019.
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathicu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Edouard Duchesnay, Scikit-learn: Machine learning in python. J. Mach. Learn. Res., 12(ull):2825-28800 November 2011. ISSN 1532-4435.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark. Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference Machine Learning, ICML 2021, 18- 28-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 8748 8763. PMLR, 2021. URL http: pestp://peee http:///roeeedingge ml press/v139/ radford21a html
Naveen Raman, Mateo Espinosa Zarlenga, and Mateja Jamnik. Understanding inter-concepl relation- ships in concept-based models. In Proceedings of the 41st International Conference on Machine Learning, ICML'24 24. JMLR.org, 2024.
Sukrut Rao, Sweta Mahajan, Moritz Böhle, and Bernt Schiele. Discover-then-naaee aggsstgnostic concept bottlenecks via automated concept discovery. In Ales Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (eds.). Computer Vision ECCV 2024 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LXXVII, volume 15135 of Lecture Notes in Computer Science, pp. 444-661. Springer, 2024. doi: 10. 1007/ URL https: /doi org 1007 978 3 - 978-3-031-72980-53 2,.
26
Peter J. Rousseeuw. Silhouettes: graphical aid the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20:53-65, 1987. ISSN 0377-0427. doi: http://ccoooeeeerr.aeaa///////72 com/
URL https: wwwp//wwww..mm/////////////ww/w/ www.eecceeneeireettttt
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115 (3):211-252,, 2015. doi: 10. 1263-015-0816->)
Nishad Singhi, Karsten Roth, Jae Myung Kim, and Zeynep Akata. Improving intervention efficacy via concept realignment in concept bottleneck models. In ICLR 2024 Workshop on Representational Alignment, 2024. htttpss:/openrevieew
forum wow.:rmmiidmmmmmmmu.uue.o..
Moritz Vandenhirtz, Sonia Laguna, Ričards Marcinkevičs, and Julia E Vogt. Stochastic concept bottleneck models. In Thirty-eightt Conference on Neural Information Processing Systems, 2024.
URL httlhtts////ppeeeiiiiee
Johanna Vielhaben, Stefan Bluecher, and Nils Strodthoff. Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees. Transactions Machine Learn- ing Research, 2023. ISSN 2835-8856. URL https:://ppeneeveew..ett www.neeee/fottttomm wtt./oorrmmmmmm
MxXRPZ7HKX
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd binds-200-2011 dataset. Technical Report CNS-TR-201 1-001, August 2011.
14

---

Published as a conference paper at ICLR 2026
Yongqin Xian, Christoph H. Lampert. Bernt Schiele, and Zeynep Akata. Zero-shot learning comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattem Analysis and Machine Intelligence, (1(9):2251-2266 2019. doi: 10.1 10.109/TPAMM.20112257777
Xinyue Xu, Yi Qin, Lu Mi, Hao Wang, and Xiaomeng Li. Energy-based concept bottleneck models: Unifying prediction, concept intervention, and probabílistic interpretations. In The Twelfth International Conference on Learning Representations, 2024. URL/attps://ppeeeeess
net Det//orum?id-Inee Iig.oodxi11g
Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language bottle: Language model guided concept bottlenecks for interpretable image classification. In IEEE/CVF Conference
Computer Vision and Pattern Recognition, CVPR 2023. Vancouver, BC, Canada, June 17-24, 2023, pp. 19187-19197 IEEE, 2023. doi: 10.
1109/CVPR537222222...133 URL https: /tt:::://ooooooooo/ wogp http:///0....00001....0100000 wttp://11..11101111111 2023
01:39
Chih- Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Raviku- mar. On completeness-aware concept-based explanations in deep neural networks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, pp. 20554-20565. Curran Associates, Inc., 2020. URL httpss://proceedings neurips cc cc/paper [irf_iiii /r1les/paper/2220 [i220008227711111111155232222223
Mert Yuksekgonul. Post-hoc concept bottleneck models, 2022. URL httpp::/github.cem com/
mertyg/post- www.:possttthh-hhoo wwchhocchoe cbm
Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. In
The Eleventh International Conference on Learning Representations, 2023. URL https:
//openreview net formee/oommmmem Sattsuummiinn--a n--AAA2ECEyo
Ruihan Zhang, Prashan Madumal, Tim Miller, Krista Ehinger, and Benjamin Rubinstein. Invert- ible concept-based explanations for cnn models with non-negative concept activation vectors. Proceedings of the AAAI Conference on Artificial Intelligence, 35:1 1682 11690, 05 2021. doi: 10..:/9/aaai..3551.....7.7
15

---

Published as a conference paper at ICLR 2026
A SPLITTING CONCEPTS USING CLUSTERING
To investigate other methods for extracting sub-concepts from a CEM's embedding space, we explored an alternative approach based on unsupervised clustering. While the main paper focuses on the SAE-basee method, present the clustering- based alternative here for completeness. Unlike the feature-based discovery of SAEs, where single input can activate multiple features, this clustering method naturally produces groups of mutually exclusive sub-concepts.
Specifically, we make use of the TURTLE framework (Gadetsky et al. 2024). which is designed to discover the labels of dataset without any supervision by finding the labelling that induces maximal margin classifiers in the representation spaces of different foundation models. TURTLE can be understood method cluster examples using embeddings from multiple models simultaneously. This is useful in our setup, as we can leverage initial CEMs trained with different backbones (e.g.. CLIP (Radford et al. 2021) and DINOv2 (Oquab et al., 2024)) to produce more robust clustering.
The process begins by partitioning the training data's concept embeddings. For a given top-level concept c₁, we create two sets of embeddings: one set containing embeddings from examples where the initial CEMs agree C; is present, and another where they agree it is absent. We then run the TURTLE clustering algorithm on each of these sets independently. This separation allows us to discover "positive" sub-concepts (which are only active when c; is present) and "negative" sub-concepts (which are only active when 6; is absent).
To determine the optimal number of sub-concepts to discover for given set of embeddings, test a range of cluster counts (from a minimum a to a maximum B). For each count, we compute the average silhouette score (Rousseeuw) 1987) of the resulting clustering. The number of clusters that yields the highest silhouette score is selected. Once the optimal clustering is found, each cluster is treated distinct discovered sub-concept. A key property of this approach is that the clustering algorithm creates a hard partition of the embedding space, meaning each example can belong to only one cluster. Consequently, the discovered sub-concepts (within either the positive or negative set) are inherently mutually exclusive. We generate new binary concept labels for each cluster, where examples belonging to the cluster are labelled as having the sub-concept, and all other examples are labelled as not having it.
Our complete clustering based Concept Splitting method is detailed in Algorithm
Algorithm 1:
Concept Splitting using clustering for a single concept
Input: Set of examples D where all the initial CEMs M agree that the concept we are splitting is present (or they all agree e is not present), and concept embeddings Z for e from the models in M for all examples in D.
Output: Discovered concept labels
Hyperparameters: Minimum and maximum number of clusters, a, 3 E N. Note: TURTLE refers to the TURTLE clustering method proposed by Gadetsky et al. (2024).
T+ Ø0
+ arg mix hiwp::/ceesSSottttttssrrCCCCCCC/
clusters + TURTLE(Z, n) for cluster in clusters do
new concept labels + on for all examples in cluster and off for the remaining examples in the training dataset.
T T (new_concept _ labels)
end return L
A.I EVALUATING AND NAMING DISCOVERED SUB-CONCEPTS
To quantitatively evaluate the interpretability of the sub-concepts discovered via clustering, we automatically assign und meaning each one. This is achieved by matching them against a predefined "concept bank" of ground-truth concepts that were intentionally excluded from the initial CEM training.
16

---

Published as a conference paper at ICLR 2026
Table Mean ROC-AUC for discovered concepts. The clustering method performs better in some cases, and the SAE method performs better in others.
MNIST-ADD
SHAPES
CUB
AwA2
PscudoKitchens
HiCEM Concept Splitting (clustering)
0.9 10.02
1..03
1..01
0.81 10.03
0.86 10.03
HiCEM Concept Splitiigg (SAF)
0.93
0.93
0.85
00.01
0.88 20.00
Table 6: Task accuracies. The clustering method and the SAE method perform similarly.
MNIST-ADD
SHAPES
CUB
AwA2
PseudoKitchens
HiCEM Concept Splitting (clustering) HiCEM Concept Splitting (SAE)
10.00
0.802
0.76.0.00 0000
0.9810.00
0.63
0.92=0.00
0.87
0.74
00.00
0.65
Table 7: Mean ROC-AUCs for provided concepts. identically.
The clustering and SAE methods perform
MNIST-ADD
SHAPES
CUB
AwA2
PseudoKitchens
HiCEM + Concept Splitting (clustering)
0.99 10.00
1. 1..00
0.93 o
1.00
0.91 10.00
HiCEM Concept Splitting (SAE)
0.99
1.00=0.00
0.93
1.00
0.91
MNIST-ADD
SHAPES
CUB
AwA2
PseudoKitchens
Task accuracy
0.95
0.7
0.94
0.9
0.98
0.68
0.9
0.92
0.8
0.66
0.85
0.97
0.64
0
2
4
10
75
150
0
5
10
0
2
4
6
8
Number intervened
Number intervencd Number intervened
Number intervened
Number intervened
HiCEM + Concept Splitting (clustering)
HiCEM + Concept Splitting (SAE)
Figure 6: Task accuracy as discovered concepts are intervened. Interventions are mostly effective on concepts discovered with clustering and with SAEs.
The matching methodology used for this clustering- approach differs from the one used for SAE-based method in the main paper. Here, for each discovered sub-concept (i.e., each cluster), compute its ROC-AUC score against every compatible concept within the concept bank. The discovered sub-concept is then assigned the semantic label of the bank concept that yields the highest ROC-AUC score. This procedure ensures that every discovered cluster is assigned interpretation. consequence of this approach is that multiple discovered sub-concepts may be matched to the same ground-truth concept from the bank. In analysis, treat such instances duplicates and merge them into a single, final sub-concept before reporting accuracies and performing interventions.
Using clustering discover sub-concepts is compared and Figures and 7 Both methods can discover sub-concepts, but the SAE method is less compu- tationally demanding, as does not require repeated clustering find good number of clusters. Additionally, the SAE method does not require deduplication step. Therefore, it has several advantages over the clustering method.
SAE-based method in Tables 5 6 and
B PSEUDOKITCHENS
This Appendix describes PstudoKitchens, our synthetic dataset of photorealistic 3D kitchen renders with ground-truth concept annotations.
17

---

Published as a conference paper at ICLR 2026
MNIST-ADD
SHAPES
CUB
AwA2
PseudoKitchens
Task accuracy
0.95
0.95
0.94
0.9
0.99
0.9
0.9
0.8
0.93
0.8
0.98
0.7
0.92
0.85
0
0
2
4
0
10
20
30
0
10
20
10
20
Number intervened Number intervened Number intervened Number intervened Number intervened
HiCEM + Concept Splitting (clustering)
HiCEM Concept Splitting (SAE)
Figure 7: Change in task accuracy as provided concepts are intervened. The methods perform similarly.
Figure 8: Images from PseudoKitchens demonstrating the dataset's 's photorealistic quality and diversity kitchen layouts, ingredient combinations, lighting conditions, and camera perspectives. Each scene contains ingredients for recipe classification.
PseudoKitchens (Figure & is generated using Blender 4.5 professional open source 3D graphics software package. We use Blender's Python API to automate scene generation. Our approach
http::::/www..bener.cco
18

---

Published as a conference paper at ICLR 2026
Figure 9: An example from the PseudoKitchens dataset. Left: a photorealistic kitchen scene. Right: ground-truth concept annotations, with colours indicating the spatial locations of individual concepts.
leverages physically-busee rendering to create photorealistic images whilst maintaining complete experimental control over scene properties. The dataset consists of kitchen scenes containing ingredients for recipe classification tasks, with each scene accompanied by annotations that describe the location of every ingredient in it.
B.1
3D ASSETS
Kitchen environments are constructed using 3D assets sourced from BlenderKi all licensed under Royalty Free or Creative Commons CCO licences. The base kitchen layouts feature countertops, cabinets, appliances, and storage areas. We manually curated five distinct kitchen layouts.
B.2 RECIPES
We designed 10 distinct recipes that define valid combinations of ingredients for the classification task. Some ingredients organised into groups shown in Table g If recipe contains ingredient group, random number ingredients are selected from that group, unless the group is pasta in which only type of pasta selected. The recipes used are shown in Table All together, the recipes use 29 different ingredients. Where possible, for each ingredient we found multiple distinct 3D models provide variation. For each instance, recipe is chosen uniformly at random.
Table 8: Ingredient groups in PseudoKitchens.
Group
Ingredients
Fruit
Banana, Orange, Apple, Pear, Pineapple Onion, Carrot, Potato, Pepper, Courgette Macaroni, Spaghetti
Vegetables
Pasta
B.3 INSTANCE GENERATION
For each generated image:
1. A kitchen layout, floor and wall textures are selected uniformly at random. The light position, intensity and colour temperature chosen randomly.
https://www.btt com
19

---

Published as a conference paper at ICLR 2026
Table 9; Recipes in PscudoKitchens.
Recipe
Ingredients
Fruit Salad
Fruit
Vegetable Pasta
Pasta, Onion, Garlic, Oil, Vegetables, Spice, Tin Tomatoes
Risotto
Cheese, Onion, Garlic, Vegetables, Oil, Spice, Rice
Chips
Potato, Oil, Flour, Garlic, Spice
Chilli
Mince, Oil, Onion, Garlic, Chilli, Tin Tomatoes, Spice, Rice
Smoothie
Milk, Yoghurt, Fruit
Hot Chocolate
Chocolate, Milk
Banana Bread
Butter, Sugar, Egg. Flour, Banana
Chocolate Fudge Cake
Egg. Sugar, Oit, Flour, Chocolate, Syrup. Milk Garlie, Meat, Butter, Cheese, Egg, Spaghetti, Spice
Carbonara
2. The camera viewpoint is randomised within predefined bounds for each kitchen, varying both angle and distance ensure diverse perspectives whilst maintaining ingredient visibility. In some cases, not all of the ingredients placed will be visible. This mirrors real images, where some features might be occluded or of the shot. The spatial concept annotations (Figure describe exactly which ingredients are visible in each image.
3. A physics-aware placement system positions ingredients on available surfaces (countertops or tables) using weighted random selection based on surface area. Objects are allowed to overlap, and are randomly rotated and scaled to provide variation. Task inreevntelevant objects, such as saucepans and cooking utensils, are randomly placed scenes.
B.4 GROUND-TRUTI ANNOTATIONS
A key advantage of synthetic approach is the automatic generation of perfect ground-truth -round-truth annotations. For each rendered scene, we provide:
1. Concept Location Annotations: perfect segmentation masks for every ingredient placed in every image. An example is shown in Figure 9
Using Blender's Cryptomatte SxytommattSsspport support, we produce pixel-
2. Instance Information:
For every image in the dataset, we save a JSON file containing all the information needed recreate it. This includes the names of all the objects in the image, along with complete scene parameters including camera position, lighting conditions, material assignments, and object transformations. This enables reproducible generation and systematic manipulation for controlled experiments.
B.5 DATASET COMPOSITION
The complete PseudoKitchens dataset comprises 10,000 training images, 1,000 validation images, and 1,000 test images. Each image is rendered at 512 512 resolution using the Cycle Iath renderer, providing photorealistic images whilst maintaining reasonable computational requirements. It takes approximately 10 seconds render image on an NVIDIA GeForce RTX 4090 GPU.
path tracing
C DATASETS
C.I MNIST-ADD
Examples in MNIST-ADD contain two handwritten digits from the MNIST dataset (LeCun et al. 2010 each between 0 and (inclusive). The label is the sum of the digits (so there are classes). There are two provided concepts: the first one indicates whether the first digit is greater than three, and the second one indicates whether the second digit greater than three. The concepts in the
https://www.cyyllss--eo wwn..eeer..e.rrrddeer.rr....
20

---

Published as a conference paper at ICLR 2026
Table 10: High-level concepts and their corresponding merged fine-grained concepts in the AwA2 task.
High-level concept
Merged concepts
patterned distal_limi Jimb teeth weapons
patches, spots, stripes flippers, hands, hooves, pads, paws chewteeth, meatteeth, buckteeth, strainteeth horns, claws, tusks
concept bank consist of a one-hot representation of the digits (for example, one of the concepts is "the first digit is 4"). The dataset includes 10,000 training, 2,000 validation, and 10,000 test samples.
C.2 SHAPES
Images in our SHAPES task (inspired by the dSprites dataset (Matthey et al. 2017)) contain either square, circle, triangle or hexagon. The shape and the background are of different colour, and can be red, green, blue or purple. The label of sample encodes the shape, its colour and the background colour. There are classes. Images are covered small black polygons to make the task harder. The provided concepts are "the shape is a polygon", "the shape has a light colour", "the shape has dark colour", "the background is light" and "the background is dark". The concepts in the concept bank form one-hot representations of the shape, its colour and the background colour (for example one of the concepts "the shape is square"). The dataset includes 10,000 training, 2,000 validation, and 10,000 test samples. The code we used generate the dataset is included in the supplementary material, licensed under the MIT License.
C.3 CUB
CUB (Wah et al. 2011) contains images birds. Each image is labelled with the species of the bird it contains, along with many concept annotations. There 200 different species of bird in the dataset. We copy the concept preprocessing performed by Koh et al. (Koh et al.) (2020), except we do filter out any concepts. Some of the concepts the CUB dataset encode the colour of various parts of the bird. For each bird part b that has concepts indicating its colour, our task contains two provided concepts: "b has a light colour" and "b has dark colour". This leaves us with 32 provided concepts. The concept bank contains concepts corresponding to the actual colour of the bird parts. The code we use to process the concepts is included in the supplementary material.
C.4 AwA2
The AwA2 dataset (Xian et al. 2019) comprises images of 50 animal classes, each annotated with semantic concepts. For our task, we define four high-level concepts: patterned, distal limb, teeth, and weapons. Each of these constructed by grouping related fine-grained concepts, as detailed in Table 10 The concept bank contains the fine-grained concepts. The initial CEM's 's concepts are the four high-level concepts, as well as the AwA2 concepts that are not used to construct the high-level concepts. We only split the four high-level concepts, as these are the concepts for which have sub-concepts in the concept bank. The AwA2 image data was collected from public sources, such as Flickr, in 2016 (Lampert al. 2017). The dataset curators ensured that only images licensed for free use and redistribution were included.
C.5 PSEUDOKITCHENS
PseudoKitchens is described detail in Appendix The concepts provided to the initial CEM are the ingredient groups in Table 8 (e.g.. "contains fruit' ) well all the ingredients that not part of a group. The concept bank concepts correspond to the ingredients the ingredient groups (e.g.. "contains apples").
21

---

Published as a conference paper at ICLR 2026
C.6 IMAGENET
ImageNet (Russakovsky et al. 2015) (the ImageNet Large Scale Visual Recognition Challenge 2012- 2017 image classífication and localization dataset) spans 1,000 object classes (organised according to the WordNet hierarchy) and contains 1,281,167 training images, 50,000 validation images and 100,000 test images. As labels for the test images are not publicly available, we use the validation images as our test set and split the training images into training set (1,231,167 images) and validation set (50,000 images).
The concept labels provided to the initial CEM are generated automatically using the WordNet hierarchy underlying ImageNet. Each ImageNet class mapped to its WordNet synset, and we collect all of its hypernyms (ancestor categories). A fixed set of 55 high-level concept synsets (e.g. plant, tool, vehicle) is then checked against these hypernyms, and all images from a class are labelled with every concept descends from. We do not construct concept bank for the ImageNet task.
D LF-CBM AND PCBM BASELINES
Label-free CBM baseline Label-free CBMs (LF-CBMs) discover concepts by prompting large language models for concept names, and then using multimodal models to align LF-CBM representa- tions with these concepts (Oikarinen al. 2023). Evaluating the accuracy of the discovered concepts requires access to ground truth labels. Since Oikarinen et al. (Oikarinen et al.) (2023) evaluate LF-CBMs on the CUB dataset, we use the concepts they discovered and manually align them with our ground truth labels for evaluation. The resulting matchings are shown in Table IT For our other datasets, we skip the language model prompting step and directly use the names of labelled concepts to train LF-CBMs. This approach exclude those results. As Askiariinnnn et al. (2023) provide limited discussion of concept interventions, we do not attempt them in our experiments. Throughout, we use the same hyperparameters those in the authors' released code for the CUB dataset (Oikarinen 2023).
not successful on MNIST-ADD or PseudoKitchen.
Post-hoc CBM baseline Post-hoc CBMs (PCBMs) project the embeddings from a backbone network onto a concept subspace defined by set of concept vectors (Yuksekgonul et al., 2023). An interpretable predictor then classifies examples based on these projections. To improve predictive performance, hybrid variant (PCBM-h) includes an additional residual predictor alongside the concept-based one. When concept annotations are available even only for part of the training data-they can be used to compute concept vectors (Kim al. 2018). Otherwise, multimodal models by encoding concept names with a text encoder to obtain the vectors. In our experiments, we use this approach for concepts where have ground truth labels, which use to evaluate concept accuracy. For all runs, we adopt the same hyperparameters as those used in the authors' released code [Yuksekgonul] 2022).
can leverage
E MODEL ARCHITECTURES, TRAINING AND HYPERPARAMETERS
Model architectures We use the CLIP ViT-L/14 foundation model Radford et al. 2021 as the backbone for all of our models and baselines. We do not fine-tune the foundation model: we just use the representations it outputs. We use the CLIP ViT-B/16 (Radford et al. 2021 multimodal model in the LF-CBM baseline to align the models' representations with concepts (although the backbone of the LF-CBMs is the CLIP ViT-L/14 model). For all the models, we precompute the representations with standard image preprocessing pipelines and do not use any data augmentations.
When training BatchTopK SAES, we use the default hyperparameters in the code released by Bussmann et al. 2024) for all datasets The key distinction of this method is its enforcement of sparsity: it selects the top n activations across an entire batch of n samples. This allows the number of active features to vary per sample, targeting an average of 32 active features. The SAES trained for 300 epochs with dictionary size of 12,288 and a learning rate of 3 x 10-4.
When performing Concept Splitting with clustering (Appendix A we use concept embeddings from two CEMs to cluster examples. One of the CEMs has the DINOv2 ViT-g/14 foundation model
com http://www.....aaaaaaaaaaaaaaaaa
22

---

Published as a conference paper at ICLR 2026
Table 11: Matches between concepts discovered by the LF-CBM baseline and ground truth concepts in the CUB dataset.
Discovered concept
Ground truth concept
A red eye
has eye color:red has wing color::black has_back. bllck:
Glossy black wings
black back
a black beak
has_bill_co b.l.lllolorrrblaee
black breast
has_breast has :otossooooo:b:baaee
a black throat
bright orange breast
has www.sst.cooooooorrr.r.
brownish back
has back color:brown
greenish back
has back color:ggreen
grey back
has_ has_back_color::
a large bill
has mas.bbll.eength::logge. than head
red belly
has_belly -colrrrcolo
a red breast
has baa__beeattcoleee
red throat a streaked back
cossttroat..colorro
has back parit has breast pattern:striped has has_beellycolor:s coolo::wie
streaked breast a white belly white breast a white underside yellow beak yellow crown yellow eye all black coloration black eyes
bas.breasttcolor:wiie
has underparts whitr:whiie
has bill color:yelloo
has_crown_com crown_color:yellow
has jas__ye_com eye_color:yeeloow
has mas-prmmayy...mm .color:bbackk
has_eye_coooeeeeeeee eye_color::blac
blue upperparts
has hopperparts.colo::ie
blue wings duck-like bird
has wing color::blue has shape::duck-like
23

---

Published as a conference paper at ICLR 2026
(Oquab al. 2024) as its backbone, and the other uses the CLIP ViT-L/14 model (Radford et al. 2021). When clustering, we run the TURTLE method (Gadetsky et al. 2024) for 1000 epochs, with the warm-start hyperparameter set true. We set the y hyperparameter to 10 suggested by Gadetsky et al. (024)).
All of our CEM and HiCEM we always use single fully connected layer for label predictor f.
embeddings have 16 activations. Across all datasets
Training hyperparameters Our models are trained using the Adam optimisation algorithm (Kingma Ba 2015) with learning rate of x 10-3 3. They are trained for a maximum of 300 epochs (or 50 epochs for ImageNet, 25 for the ImageNet HiCEM), and training is stopped if the validation loss does not improve for 75 epochs. We use a batch size of 256.
In all CEMs, HiCEMs and CBMs the weight of the concept loss is set to À 10. Following (Koh et al., 2020), in MNIST-ADD. CUB and PseudoKitchens we use a weighted cross entropy loss for concept prediction to mitigate imbalances in concept labels. In MNIST-ADD, and in the linear probe used to name concepts for the user study, we also use weighted cross entropy loss for task prediction to mitigate imbalances in task labels.
When training CEMs and HiCEMs, the RandInt (Espinosa Zarlenga et al. 2022) regularisation strat- egy is used: at training time, concepts are intervened independently random, with the probability of an intervention being Pint 0.25. We choose = 0.25 because [Espinosa Zarlenga et al. (2022) find that it enables effective interventions while giving good performance.
F
USER STUDY
This appendix provides details on the user study conducted to evaluate the quality of sub-concept labels generated by Concept Splitting on ImageNet.
F.1 NAMING IMAGENET DISCOVERED SUB-COO CUB-CONCEPTS
The sub-concept names evaluated in the study were generated through an automated process, similar to the one used by Rao et al. (2024). To assign human readable name discovered sub-concept, we first trained a linear classifier (a probe) on the CLIP ViT-L/14 image embeddings for the ImageNet training dataset. This probe and those that do not. After training, we iterated through a vocabulary of 20,000 common English words (following Oikarinen & Weng (2023). ). For each word, computed its text embedding using CLIP's text encoder. This text embedding was then passed as input to the trained linear probe to obtain score. The word whose embedding received the highest score from the probe was selected as the name for the sub-concept.
trained distinguish between images that belong the sub-concept
F.2 ETHICAL CONSIDERATIONS
Prior to commencing the user study, application for ethical review was submitted to our institution's 's ethics committee. The project received approval before any participant recruitment data collection began. All participants were provided with detailed consent form informing them of the study's purpose, the nature of their participation, data handling, and their right to withdraw at any time.
F.3 PARTICIPANT RECRUITMENT
Twenty participants were recruited via word-of-mouth and snowball sampling. This convenience sample was primarily composed of students and colleagues from the authors' institution and other academic institutions, as well as friends and family of the authors. Participation was voluntary, and monetary other incentives visual recognition abilities and access to a computer with an internet connection. No specific domain knowledge was necessary.
provided. The only requirements for participation were basic
24

---

Published as a conference paper at ICLR 2026
Is "mound" an example of "medical instrument
Some the words the question are generated automatically, so if it does not make sense or is ambiguous, please answer it best you can choose "Not sure". !gnore the plurality words.
Yes
No
They are the same
Not sure
Figure 10: A sub-concept verification question from our user study.
F.4 STUDY DESIGN AND INTERFACE
The user study was administered through web-based application, eliminating the need for any software installation on the participants' devices. Upon accessing the study, participants were presented with consent form. After giving consent, they were assigned a random participant ID to anonymise their responses and allow them to resume the study they wished.
Participants were then shown a set of instructions detailing the two types of tasks they would be asked to complete. The study consisted of a maximum 100 questions, and participants could stop at any point.
The two types of questions were:
1. Sub-Concept Verification: Participants asked evaluate the relationship between parent concept and a sub-concept by answering the question, "Is [sub-concept] an example of [parent concept]?" (see Figure IOp.
In the experimental group, the sub-concept names provided by naming method were used.
In the control group, the sub-concept names were chosen randomly from the dictionary used name sub-concepts, and paired with a parent concept.
2. Image Label Verification: Participants were shown an image and sub-concept name and asked, "Does this image show [concept]?" (see Figure I.
In the experimental group, the images shown those method had labelled as exhibiting the given sub-concept.
In the control group, the images were selected at random from the ImageNet training dataset.
To account for potential learning curve participants familiarised themselves with the tasks, the first five responses from each participant for each question type were discarded and not included in analysis. For each question, participants could also select "They are the same" (for sub-concept verification tasks only) or "Not sure". All participants were shown mixture of experimental and control questions.
The data was anonymised by assigning a unique, randomly generated ID to each participant, with no personal identifiable information being collected. The collected responses were stored securely and were only accessible to the study's authors.
G INTERPETIIG DISCOVERED CONCEPTS
Figure 12 shows random sample of 16 images from the MNIST-ADD training dataset that were labelled as having one of the discovered sub-concepts. From this sample, is clear that the meaning of the discovered sub-concept is "the top digit 6".
25

---

Published as a conference paper at ICLR 2026
Does this image show "ships"?
Some of the words in the question are generated automatically, so does not make sense or ambiguous, please answer it as best you can or choose "Not sure", Ignore plurality: for example, you are asked 'Does this image show "lions""" and the image contains single lion, you should answer "Yes".
Yes
No
Not sure
Figure 11: An image label verification question from our user study.
H SPLITTING ONE-HOT ENCODED CONCEPT EMBEDDINGS
To explore how Concept Splitting and HiCEMs perform in an idealised setting where concept embeddings contain perfect information about sub-concepts, we conducted controlled experiment on the MNIST-ADD dataset. Specifically, we created one-hot encoded concept embeddings: for each top-level concept (e.g., the first digit greater than 3), the positive embedding for that concept was one-hot vector with single non-zero entry corresponding to the true sub-concept present in the example (c.g., the first digit is 4). This ensures that the concept embeddings encode perfect, unambiguous information about the sub-concepts that present. We then applied Concept Splitting to these one-hot embeddings and evaluated whether it could recover the ground ground-truth sub-concepts.
Concept Splitting with idealised one-hot encoded concept embeddings is compared to Concept Splitting with CEM embeddings in Tables 12 13 and 14 and Figure 13 When Concept Splitting is applied concept embeddings that contain perfect (one-hot encoded) information about sub-concepts, it is highly effective, recovering the encoded sub-concepts almost perfectly. In real-world applications, one will not have such ideal concept embeddings. However, as we demonstrate in Section Concept Splitting is also effective when applied to concept embeddings extracted from a CEM.
Table 12: Mean ROC-AUC for discovered concepts. When concept embeddings contain perfect information about sub-concepts, our method effectively recovers them.
MNIST-ADD
HiCEM + Concept Splitting (one-hot embeddings)
1.00
HiCEM Concept Splitting (CEM embeddings)
0.93
26

---

Published as a conference paper at ICLR 2026
q
6
bb
6
2
2
53
6
6
6
6
3
3
I
6
6
6
6
2
/
/
3
6
6
6
S
2
A random sample of images from the MNIST-ADD training dataset that were labelled as having one of the discovered sub-concepts. The interpretation assigned to this concept was "the top digit is 6".
Figure 12:
Table 13: Task accuracies. HiCEM achieves similar task accuracy whether trained on concepts discovered from one-hot encoded or CEM embeddings.
MNIST-ADD
HiCEM + Concept Splitting (one-hot embeddings) HiCEM + Concept Splitting (CEM embeddings)
0.93
0.92+0.00
Table 14: Mean ROC-AUCs for provided concepts. HiCEM achieves similar provided concept accuracy whether trained on concepts discovered from one-hot encoded or CEM embeddings.
MNIST-ADD
HiCEM Concept Splitting (one-hot embeddings)
1.00 +0.00
HiCEM + Concept Splitting (CEM embeddings)
0.99
27

---

Published as a conference paper at ICLR 2026
Discovered Concepts
Provided Concepts
0.97
0.97
0.96
0.96
0.96
0.96
0.95
Task accuracy
0.95
0.95
0.95
0.94
0.94
0.94
0.94
0.93
0.93
0.93
0.92
0.93
0.92
0.92
0.91
4
Number intervened
Number intervened
HiCEM + Concept Splitting (one-hot embeddingg)
HiCEM + Concept Splitting (CEM embeddings)
Interventions discovered concepts slightly more effective with idealised one-hot encoded embeddings, whilst provided concept interventions perform similarly in both scenarios. Mean and standard deviation over three shown; the standard deviation negligibly small.
Figure 13:
I IMAGENET CONCEPT INTERVENTIONS
As shown in Figure 14 provided concept interventions ImageNet perform similarly in the initial CEM and in the HiCEM with discovered sub-concepts. Due to the size of ImageNet, the experiment was only run once so Figure dolo not contain error bars.
0.9
0.8
0.7
Task accuracy
0.6
0.5
0.4
0.3
0.2
0.1
0
0
5
10
15
20
25
30
35
40
45
50
55
60
Concepts intervencd
CEM
CBM
HiCEM Concept Splitting
Figure 14: Change in task accuracy as provided concepts are intervened on ImageNet. Provided concept interventions work just as well in HiCEMs as they do in CEMs.
28

---

Published as a conference paper at ICLR 2026
J CODE, LICENSES, AND RESOURCES
Assets We used the DINOv2 foundation models (Oquab et al. 2024
battps://gitiin
com/ http/aaccbookreseecccciiiii 2 whose code and model weights are released under the Apache License 2.0. We also used the CLIP foundation models (Radford et al. 2021) (https:
whose code is available under the MIT license. To run our experiments, we made use of the CEM (Espinosa Zarlenga et al.] [2022) /attps://qitie con//nateecapplnoaa/ceeel cem MIT license), Post-hoc CBM (Yuksekgonul al. 2023) (https
//github. com/ wttp//peaaaa1111
MIT license) and Label-free CBM (Oikarinen et al. T.aella.eeee......... ree- repositories.
We implemented our experiments in Python 3.11 and used open- source libraries such as PyTorch 2.5
Paszke et al. 2019) (BSD license) and Scikit-learn (Pedregosa et al. 2011) (BSD license). We have released the code required to recreate experiments in MIT-- public repository
Resources All of our experiments were run on virtual machines with at least CPU cores, 18GB of RAM, and an NVIDIA GPU (Quadro RTX 8000 GeForee RTX 4090). Including preliminary experiments, we estimate that approximately 300 GPU hours were required to complete our work.
Use of Al We used Large Language Models (LLMs) as assistants for drafting and improving the clarity and grammar of this manuscript. LLMs were also used to generate boilerplate code. However, all core research ideas, experimental design, and analysis of the results were conducted by the authors.
httere///tttt...oooooorrrreorrer scovery sccovrr
29


Reviews:
[{"review_id": "qeH01NPAvi", "reviewer": "Reviewer_5QNW", "summary": "This paper introduces two new tools for interpretability specifically targeting the hierarchical nature of concepts/conceptual reasoning. To this end, they introduce a \"discovery\" interpretability method, as well as a new concept-based deep model to leverage this hierarchical structure. In a nutshell, the discovery method proposed aims to find novel subconcepts from \"high-level\" concept embeddings using sparse autoencoders. The hierarchical concept embedding model (HiCEM) introduces a new sub-concept neural module that explicitly models a concept starting from its sub-concepts. The authors provide a series of programmatic experiments, as well as human user study to demonstrate their new tools.", "questions": "1) Regarding clarity of evaluation of sub-concept discovery: \n  - L. 310, do all the provided datasets have concept-subconcepts pairing? If not, how is \"between the discovered sub-concept labels and their potential parent-concept-associated matches in the bank.\" (L.315) evaluated?\n  -  Isn't it possible that maybe a (sub-)concept is correctly discovered but for some reason is not in the left-out pool of concepts from the concept bank?\n  - How many concepts are left-out? How is the number of features/sub-concepts selected during discovery with SAE? if the number of left out concepts and \"features\" are not the same, is this taken into account in evaluating the discovery process? I.e. is this considered a multi-label classification problem? \n2) How does concept splitting depend on the quality of the top-level concepts? On first thought, I would perhaps think that if the concept embeddings from the pretrained CEM are not good enough, this would hinder the discovery of sub-concepts? \n3) From your description, it seems to me that concept splitting does not necessarily depend upon the concepts from CEM. If so, would it then make sense to create some sort of synthetic evaluation experiment, where you run the concept splitting on predefined \"perfect\" top-level concepts. For example, it shouldn't be a problem to define the concept embeddings as essentially one-hot encoded vectors, I think?\n4) As a minor nitpicky suggestion, I would maybe try to expand on why concept splitting is such a pivotal contribution from the authors. Personally, for how it is phrased now, concept splitting sounds like an application of concept discovery methods to pre-trained concept embeddings instead of directly to a latent representation of a sample. So if we put it like this, concept splitting sounds more like a necessary step to then train good HiCEM rather than a key standalone contribution.\n5) Based on the authors results, the main advantage of HiCEM vs normal CEMs is essentially the higher intervenability as demonstrated on, eg., the CUB dataset. Could the authors advance hypotheses regarding why this is the case? Is it because CUB and PseudoKitchens are more complex datasets and there actually exists complex parent-subconcept relationships that need direct modeling? Based on this, I believe the limitations section could be expanded by commenting on when it is expected that concept splitting+HiCEM is an extra effort that is worth to do.\n6) It would be interesting to have some intervenability human user study to strengthen even more the paper. Something along the lines of \"How would you change this explanation to get this prediction?\"", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- *Originality and significance*: while the idea of hierarchical concepts is not new per se, the authors create a novel solution specific for Concept Embedding models which seems compelling.\n- *Quality*: overall, the experiments seem reasonable to support the authors claims. In particular, I like the inclusion of user studies and intervention experiments.\n- *Clarity*: the paper as a whole is generally understandable.", "weaknesses": "- *Originality*: as mentioned above, the idea of hierarchical concepts is not new, e.g. \"Hierarchical Concept Discovery Models: A Concept Pyramid Scheme\" Panousis 2023. Could the authors expand how their work relates to previous work specifically in the context of hierarchical models of concepts?\n- *Significance*: I am completely sold on the idea of concepts being hierarchical (or at the very least not being completely independent). That being said, this does not necessarily mean that modeling them is necessary for better interpretability or intervenability, for the simple reason that often full transparency may be too overwhelming for a person to fully process what a model is doing. Do the authors have in mind a specific real-world use case where their model and concept splitting idea brings a practical advantage compared to other SOTA models?\n- *Clarity*: I understand the limitation in space, but I think it would be beneficial to expand on the evaluation of the discovered (sub-)concepts. More below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7AvHPgXEin", "reviewer": "Reviewer_ga1u", "summary": "The authors introduce HiCEMs, a variant of concept embedding models (CEMs) that 1) supports inter-concept (hierarchical) relationships and 2) in addition to the usual supervised concepts, discovers additional sub-concepts in an unsupervised manner using a sparse autoencoder (SAE).  HiCEMs are compared against a number of CBM-like competitors and the interpretability of sub-concepts the SAE discovers is evaluated through a (small scale) user study.", "questions": "Feel free to comment on any of my observations.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- **Novelty**: HiCEMs combine CEMs with concept discovery.  While the two\n  ingredients are well known, their combination (and especially the\n  rather intricate sub-concept module that is introduced to combine them) look\n  novel to me.\n\n- **Quality**:  Using SAEs/clustering for concept discovery in a\n  well-structured embedding space (obtained via per-concept supervision) is\n  reasonable, but see below. The choice of research questions is good and the\n  experimental setup (choice of datasets and competitors) is sound.\n\n- **Clarity**: the text is generally well written, except for the sub-concept\n  modules, see below. The related work is relatively complete.\n\n- **Significance**: the idea of pairing SAEs with concept annotations will\n  probably inspire follow-up work. This work also partially fills the gap\n  between concept-bottleneck models and neuro-symbolic architectures, which is\n  interesting.  The new dataset is also a welcome addition.", "weaknesses": "While I am generally positive about the paper, I'd like to raise some issues I found.\n\n- **Quality**\n\n    - My main concern is that SAEs do not provide any sort of guarantee.\n      Recent works (which may or may not be under submission at ICLR, I have\n      not checked) suggest they *can* recover the underlying generative\n      concepts provided these are sparse, but in general the jury is still out\n      on this, to the best of my knowledge.  Using SAEs is fine, but the\n      authors should be careful in not overselling their ability to recover\n      good concepts.\n\n  - The interpretability of SAE sub-concepts rests on the user study.  This is\n    however quite small scale (20 participantes; by the way, this information\n    is important and belongs in the main text; currently it is \"hidden\" in Appendix E).\n    Are these results statistically significant?\n\n  - I also have a (relatively modest) issue with the structure of the user study itself.  The thing is the sub-concepts are named using english words through a \"discretization\" step (specifically, first english words are mapped to clip embeddings and then hicem sub-concepts via a probe, then sub-concepts are named accordingly). It is possible the sub-concepts don't match the name exactly, but only very approximately, which means that this step essentially \"cleans up\" the discovered sub-concepts.  To what extent do the results rely on this step? How dissimilar are the names and the associated sub-concepts?\n\n- **Clarity**\n\n  - My main concern is the description of the sub-concept modules, which is\n    quite dense and consisting of pure free-form text.  It should be unpacked,\n    ideally with the help of equations, to make it clearer and more precise.\n\n  - The CLIP-based automatic naming procedure should be described in the main\n    text, as it is integral to the method -- without it, sub-concepts cannot be\n    properly interpreted by human stakeholders, breaking self-explainability.\n\n  - I am confused about the term \"sub-concept supervision\" used in p 7 onward.\n    Does it refer to the fact that Concept Splitting is absent? If so, it'd\n    make sense to either rename it \"sub-concept weak supervision\" or to call\n    the baseline HiCem-NoSplitting or somesuch.\n\n  - Table 2: \"Even better results might be obtained by manually selecting\n    high-quality sub-concepts, and naming them manually instead of using our\n    crude automatic naming method with CLIP.\" Very likely, but this is\n    equivalent to requiring expert sub-concept supervision - you claim to\n    tackle settings in which this is not available.  Moreover, depending on how\n    this is obtained (UI-wise), experts could misidentify concepts, so there is\n    no a prior guarantee that this would work better than the CLIP probe\n    procedure. I think this sentence should be removed.\n\n - The MNIST-ADD dataset was not introduced by LeCun et al.  Please, fix the\n   reference.\n\n - It should be more clear in the text that the \"SHAPES\" dataset is similar\n   to the relatively well-known dSprites dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oC4lm9SE7a", "reviewer": "Reviewer_JuNX", "summary": "The paper presents Hierarchical Concept Embedding Models (HiCEMs), extended from Concept Embedding Models (CEMs) to represent inter-concept relationships with various granularities. The work also introduces Concept Splitting, a technique for automatically discovering finer-grained subconcepts from pretrained concept embeddings without requiring additional annotations. Together, these methods enhance interpretability, reduce annotation costs, and enable hierarchy concept interventions while archives comparative performance. The paper also introduces a new synthetic dataset, PseudoKitchens, with precise ground-truth concept annotations and spatial localization.", "questions": "1. It would be helpful if the authors could provide qualitative comparison results between different methods, which would make the improvements in interpretability more intuitive and easier to observe.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The HiCEM archives comparable performance comparing to previous methods.\n2. The extensive experiments show that the alternative concept splitting method (clustering) also achieves comparable performance to the original design (SAE).\n3. The paper proposes an additional synthetic dataset, PseudoKitchens, a controllable scene with precise annotation.", "weaknesses": "1. What is the actual benefit of the hierarchical design? Although the experiments show that HiCEM can leverage subconcepts to intervene in the model and achieve better performance on the CUB dataset, in most cases, its performance is comparable to that of CEM.\n2. Although the paper includes a user study to evaluate the connections between subconcepts and their parent concepts, it remains unclear whether these subconcepts also contribute to improving the overall interpretability.\n3. In the current version, the authors focus only on a two-level hierarchical concept structure. It would be interesting to explore deeper hierarchies and examine the relationships across multiple levels.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gJrh0D5waN", "reviewer": "Reviewer_hpPz", "summary": "This work introduces multiple novelties to make classification more interpretable. Basing on CEM embeddings, Concept Splitting discovers new, hierarchical subconcepts based on SAEs. Then, HiCEM integrates these sub-concepts into the CEM structure. To evaluate, PseudoKitchen is introduced, a granular dataset with concept control. Also, for imagenet, a user study is conducted to evaluate the meaningfulness of the discovered subconcepts.", "questions": "I invite the authors to address any of the aforementioned weaknesses, albeit I deem it unlikely that it will positively affect my score.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* This work integrates SAE's concept discovery capabilities into the CEM framework. While there have been attempts at CBM + SAE combinations, the application of SAEs on the CEM embeddings is a neat idea, as these embeddings naturally lend themselves to such an analysis. How to integrate the discovered concepts into CEMs is nontrivial and it seems the authors engineered a way that works nicely.  \n* In interpretability research, human user studies are the gold standard, but nearly never done. As such, I appreciate the authors' efforts to conduct a user study, whose results have also positively affected my perception of this work. In general, the results are extensive and span many datasets and different axes of evaluation, which support the claims made in the paper.\n* Also, the introduction of PseudoKitchens might be a valuable addition to the literature, as it provides realistic, controllable, sample-specific concept control and could help the field get away from class-based concepts.\n* I appreciate that reproducible code is provided.", "weaknesses": "* My main concern of this work is that it bases on CEMs. I do not consider CEMs truly inherently interpretable, mainly due to their embedding-based design, which has also been demonstrated by [1,2] that show that CEMs contain a lot of leakage. Therefore, any derivative work also has questionable interpretability in my view. For example, the embeddings of the discovered sub-concepts might contain a lot more information than just with regard to their sub-concept. In this line of thought, the added complexity of the sub-embeddings makes the model more opaque, thereby even harder to interpret. There is a trend of arguing that intervention effectiveness implies that there is no (strong) leakage, however, due to the RandInt strategy at training time, this argument does not hold in my opinion. I can be convinced otherwise, if intervention effectiveness was retained without RandInt training for both CEM and HiCEM.\n* Understanding the architecture of HiCEM in Sec. 4.1. took a while. I wonder whether it might be easier to understand by replacing the flowtext with more structured formulas or by using a pseudoalgorithm. Also, I just realized there is no Sec. 4.2, and there shouldn't be any single subsections, but that's unimportant here.\n* The paper reiterates that related work does not model concept relationships. I am referring to the sentences \"Furthermore, they do not consider the relationships between concepts; instead, they treat all concepts as independent variables\" and \"However, the modelling of these relationships addresses a gap in previous concept-based architectures, advancing the state of the art in concept-based explainability\". In my opinion these statements are misleading. There is a considerable line of literature that tackles precisely this problem [3,4,5,6] and given the focus of this work on modeling these concept relations, this related field should at least be mentioned.\n\n[1] Parisini, Enrico, et al. \"Leakage and interpretability in concept-based models.\" arXiv preprint arXiv:2504.14094 (2025).\n\n[2] Almudévar, Antonio, José Miguel Hernández-Lobato, and Alfonso Ortega. \"There Was Never a Bottleneck in Concept Bottleneck Models.\" arXiv preprint arXiv:2506.04877 (2025). \n\n[3] Havasi, Marton, Sonali Parbhoo, and Finale Doshi-Velez. \"Addressing leakage in concept bottleneck models.\" Advances in Neural Information Processing Systems 35 (2022): 23386-23397.\n\n[4] Vandenhirtz, Moritz, et al. \"Stochastic concept bottleneck models.\" Advances in Neural Information Processing Systems 37 (2024): 51787-51810.\n\n[5] Xu, Xinyue, et al. \"Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations.\" The Twelfth International Conference on Learning Representations.\n\n[6] Singhi, Nishad, et al. \"Improving intervention efficacy via concept realignment in concept bottleneck models.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)