Coarse-to-Fine Concept Bottleneck Models
Authors: Konstantinos P. Panousis
Dino Ienco
Diego Marcos
Venue: NeurIPS 2024 poster (2024)
Keywords: Interpretability
Explainability
Concept Bottleneck Models
Sparsity
Multimodal Models
Concepts
Textual Descriptions
Bayesian
Masking
Primary area: interpretability_and_explainability
TL;DR: We propose a novel coarse-to-fine construction for concept discovery; we do not solely rely on the similarity between concepts and the whole image, but  we also consider granular information residing in patch-specific regions of the image.

Abstract: Deep learning algorithms have recently gained significant attention due to their impressive performance. However, their high complexity and un-interpretable mode of operation hinders their confident deployment in real-world safety-critical tasks. This work targets ante hoc interpretability, and specifically Concept Bottleneck Models (CBMs). Our goal is to design a framework that admits a highly interpretable decision making process with respect to human understandable concepts, on two levels of granularity. To this end, we propose a novel two-level concept discovery formulation leveraging: (i) recent advances in vision-language models, and (ii) an innovative formulation for coarse-to-fine concept selection via data-driven and sparsity inducing Bayesian arguments. Within this framework, concept information does not solely rely on the similarity between the whole image and general unstructured concepts; instead, we introduce the notion of concept hierarchy to uncover and exploit more granular concept information residing in patch-specific regions of the image scene. As we experimentally show, the proposed construction not only outperforms recent CBM approaches, but also yields a principled framework towards interpetability.

Coarse-to-Fine Concept Bottleneck Models
Konstantinos P. Panousis 1.2,5
Dino Ienco 1,2,3,4
Diego Marcos 1,2
Inria
University of Montpellier
$ Inrae
UMR TETIS
Department of Statistics, AUEB
panousis@aueb gr diego www.mmmmiaiii...i... dino ieno.iinco@inare...
Abstract
Deep learning algorithms have recently gained significant attention due to their impressive performance. However, their high complexity and un-interpretable mode of operation hinders their confident deployment in real-world safety-critical tasks. This work targets ante hoc interpretability, and specifically Concept Bot- tleneck Models (CBMs). Our goal is to design framework that admits a highly interpretable decision making process with respect human understandable con- cepts, two levels of granularity. To this end, propose novel two-level concept discovery formulation leveraging: (i) recent advances in vision- language models, and (ii) innovative formulation for coarse-to-fine concept selection via data-driven and sparsity-inducing Bayesian arguments. Within this framework, concept information does not solely rely on the similarity between the whole image and general unstructured concepts; instead, we introduce the notion of concept hierarchy uncover and exploit more granular concept information residing in patch-specific regions of the image scene. As we experimentally show, the pro- posed construction only outperforms recent CBM approaches, but also yields a principled framework towards interpetability.
Introduction
The recent advent of foundation models has greatly popularized the deployment of deep learning approaches variety of tasks and applications. However, in most cases, deep architectures are treated in an alarming black-box manner: given an input, they produce a particular prediction, with their decision- their mode of operation and complexity preventing any potential investigation making process. This property not only raises serious questions concerning their deployment in safety-critical applications, but at the same time, could actively preclude their adoption settings that could otherwise benefit societal advances, e.g., medical applications.
This conspicuous shortcoming of modern architectures has fortunately gained lot of attention from the research community in recent years, expediting the design of novel frameworks towards deep learning interpretability. Within this frame of reference, there exist two core approaches: ante and post hoc. The latter aims to provide explanations to conventional pretrained models, e.g., Network Dissection [1] while the former aims to devise inherently interpretable models. In this context, Concept Bottleneck Models (CBMs) constitute one of the best-known approaches [9]; these comprise: (i) an intermediate Concept Bottleneck Layer (CBL), a layer whose neurons are tied to human understandable concepts, e.g., textual descriptions, followed by (ii) a linear decision layer. Thus, the final decision constitutes linear combination of the CBL's concepts, leading to a more interpretable decision mechanism. However, typical CBM approaches
accompanied by four significant drawbacks: (i) they commonly require hand-anmotated concepts, (ii) they usually exhibit
"Code available at: htttp:://gtttbbbccm/////// https///ggtttboooo////opppssosss Toneeee---------eeee CBMs
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

lower performance compared to their non- non- interrpreeeee counterparts, (iii) their interpretability is substantially impaired due to the sheer amount of concepts that need be analysed during inference. and (iv) they are not suited for tasks that require greater granularity.
The first drawback has been recently addressed by incorporating vision- aasion-language models in the CBM pipeline; instead of relying on a fixed concept set, images and text can be projected in the common embedding space and compared therein. Mechanisms restore performance have also been proposed, e.g., residual fitting [25 The remaining two limitations however, still pose significant challenge.
Indeed, CBMs commonly rely on large amount of concepts, usually proportional to the number of classes of the given task; with more complex datasets, thousands of concepts may be considered. Evidently, this renders the investigation of the decision making task an arduous and unintuitive process. In this context, some works aim to reduce the amount of used concepts by imposing sparsity constraints upon concept activation. Commonly, post-hoc class-wise sparsity methods are considered [22 13: however, these tend to restrict the number of concepts on a per-class basis, enforcing ad hoc application-speeific sparsity/performacce thresholds, greatly limiting the flexibility of concept activation for each example. Recently, a data-driven per-example discovery mechanism has been proposed in [17; this leverages binary indicators that explicitly denote the relevance of each concept towards the downstream task on a per-example basis, thus allowing for greater flexibility.
Even though these approaches aim to address the problem of concept over- abundanun,, they do not consider ways to emphasize finer concept information that may be present in a given image; they still exclusively target similarity between concepts and the whole image. In this setting, localized, low-level concepts (e.g., shape or texture), are predicted from a representation of the whole image, potentially leading the undesirable use of top-down relations. For instance, the model detects some high-level concept (e.g., elephant), resulting in associated lower-level concept activations (c.g., tusks, wrinkled skin) that may not even be actually be visible. This can further lead significant concept omission, i.e., information potentially crucial for tasks that require greater granularity, e.g., fine-grained part discovery, or even cases where the input is susceptible multiple interpretations.
Drawing inspiration from this inadequacy of CBM formulations, we introduce a novel coarse-to- coureeto--fie fine paradigm that allows for discovering and capturing both high and low level concept information. We achieve this objective by devising an end-to-end trainable hierarchical construction; in this setting, we exploit both the whole image, well information residing in individual isolated regions of the image, i.e., specific patches, to achieve the downstream task. These levels are linked together by intuitive and principled arguments, allowing for information and context sharing between them, paving the way towards more interpretable models. We dub our approach Coarse-to-Fine Concept Bottleneck Models (CF-CBMs); in principle, our framework allows for arbitrarily deep hierarchies using different representations, e.g., super-pixels. Here, we focus on the two-level setting, as a proof of concept for the potency of the proposed framework. Our contributions can be summarized as:
We introduce novel interpretable hierarchical model that allows for coarse-to-fine concept discovery, exploiting finer details residing in patch-specific regions of an image.
We propose a novel way of assessing the interpretation capacity of our model based on the Jaccard index between ground truth concepts and learned data-driven binary indicators.
.
We experimentally show that CF-CBMs outperform other SOTA approaches classification- wise, while substantially improving interpretation capacity.
2
Related Work
n: "n a dataset comprising N Concept Bottleneck Models. Let us denote by D = {X
image/label pairs, where Xn, E R xc and nn E {0, 1) Within the context of CBMs, concept set A = {a....., aa comprising H concepts, e.g., textual descriptions, is also considered; the main objective to re-formulate the prediction process, constructing a bottleneck that relies upon the considered concepts, in an attempt to design inherently interpretable models. In this context, early works on concept- based models [I 19, were severely limited by requiring an extensive hand- annotated dataset comprising all the used concepts. To enhance the reliability of predictions of diverse visual contexts, probabilistic approaches, such as ProbCBM 7 introduce the concept of ambiguity, allowing for capturing the uncertainty both in concept and class prediction. The appearance of vision- language models, chiefly CLIIP [18), has mitigated the need for hand-annotated data, allowing to easily make use of thousands of concepts. followed by a linear operator on the concept similarity to solve
2

---

the downstream task [3 24). However, this generally means that all concepts may simultaneously contribute to given prediction, rendering the analysis of concept contribution an arduous and counter-intuitive task, severely undermining the sought-after interpetabilityy. This has led to methods that seek sparse concept representation, either by design [2 or data-driven
perspectives.
Concept-based Classification. To discover the relations between images and attributes, vision- language models, e.g., CLIP (18), are typically considered. These comprise an image and a text encoder, denoted by (-) and Er(-) respectively, trained in a contrastive manner (20) 2 to learn a common embedding space, After training, we can then project any image and text this common space and compute the similarity between their (f2- -ormmallllzd) embeddings. Assuming a concept set A, with |
H. the most commonly considered measure is the cosine similarity S:
S x Ev ( E RNN
(I)
This similarity-based characterization yields a unique representation for each image and has recently been exploited to design models with interpretable decision processes such CBM-variants 255 13) and Network Dissection approaches 14, [5]- Within this context, let us consider a C-class classification setting; by introducing a linear layer We E RC*H we can perform classification via the similarity representation S. The output of such network yields:
Y = S=W E RNCC
(2)
The image/text encoders are typically kept frozen; training only pertains to the weight matrix W... c.
However, this formulation comes with a key deficit: is by-design limited to the granularity of the concepts that it can potentially discover in any particular image. Indeed, VLMs are commonly trained to match concepts to the whole image; this can lead to a loss of granularity, that is, important details may be either omitted or considered irrelevant. Yet, in complex tasks such as fine-grained classification or in cases where the decision ambiguous, this can potentially hinder both the downstream task, but also interpretabilityy In these settings, it is likely that any low-level information present is not exploited, hindering any potential low-level investigation on the network's ' process. Moreover, this approach considers the entire concept set describe an input; this not only greatly limits the flexibility of the considered framework, but also renders the interpretation analyses questionable due to the sheer amount of concepts that need to be analysed during inference beyond the classical definition of CBMs and consider the setting of coarse-to- concept- based classification based on similarities between images, patches and concepts.
[19]. In this work, take step
3 Coarse-to-fine CBM
To construct our proposed CF-CBM framework, we first introduce two distinct modeling levels: High (H) and Low (L). The High level aims to model the whole image, while the Low level investigates and aggregates information stemming from localized regions. At the outset, we define these levels as separate modules that can be individually used towards a downstream task using some independent concepts sets Аи and Aμ; intuitively, the former should comprise descriptions that characterize the main scenes/objects in the considered dataset, e.g., an ImageNet class name, such as Arctic Fox, while the latter, descriptions that are inferrable from localized regions of the image, e.g., characteristics of parts of the animal or the background.
Then, we present the notion of hierarchy of concepts. Specifically, we introduce interdependence between the sets to capture information in a noarse-o-fiine-rainee manner; in this setting, the concepts Aи encapsulate concepts that characterize each image, in turn determining the allowed subset of concepts from the low-level concept pool Az. Essentially, the concepts in Aи capture a holistic representation of the image, while the low-level, their sub-characteristics. delving deeper into regions, aiming to uncover finer grein-g information. Each level aims to achieve the given downstream task, while information sharing takes place between them as we describe next.
3.1 High Level Concept Discovery Module
For the formulation of the High Level view of our CF-CBM model, we consider: (i) the whole image, (ii) set of H concepts Ag, and exploit the definitions of concept-based classification, i.c., Eqs. !
3

---

2 To this end, we introduce a single linear layer with weights W E RC*" yielding:
S x Fly (X) RRNH = = SH W He ERNNC
(3)
(4)
Evidently, this formulation, fails to take into account the relevance of each concept towards the downstream task or any information redundancy, since all the considered concepts are potentially used; this also limits its emerging interpretation capacity due to the large amount of concepts that need to be analysed during inference. To bypass this drawback, we consider a novel, data-driven mechanism for concept discovery based on auxiliary binary latent variables.
Concept Discovery. To discover the subset of high-level concepts present in each example, we introduce the auxiliary binary latent variables Z E {0, 1 JjNWXH; ; these operate in an "on-off" fashion, indicating. for each example, if a given concept needs to be considered to achieve the downstream task, i.e., [Z n = 1 if concept h is active for example n, and O otherwise. The output of the network now given by the inner product between the classification matrix W He and the discovered concepts as dictated by the binary indicators Z ZH:
Y = [Z W He T R
(5)
A naive definition of these indicators would require computing and storing one indicator per example. To avoid the computational complexity and generalization limitations of such formulation, consider an amortized approach similar to [7]. To this end, we introduce data-driven random sam- pling procedure for H. and postulate that the latent variables are drawn from appropriate Bernoulli distributions; specifically, their probabilities are proportional to a separate linear computation between the embedding of the image and an auxiliary linear layer with weights WHs Ha E R where K is the dimensionality of the embedding, yielding:
Bernoulli (IZ sigmoid (Bv(X,) ))
q(Z
(6)
where I-ln denotes the n-th row of the matrix, i.e., the indicators for the n-th image. This formulation exploits an additional source of information emerging solely from the image embedding; this allows for an explicit mechanism for inferring concept activation in the context of the considered task, instead of exclusively relying on the implicit VLM similarity. The described process is encapsulated in what we call a Concept Discovery Block (CBD); this is illustrated in Fig] I (Left and Upper Right).
3.2 Low Level Concept Discovery Module
Here, we present variant of the described architecture that aims individually exploit finer information potentially present in the image. In this setting, re-using the whole image may hinder concept discovery since fine-grained details may be ignored; prominent objects may dominate the discovery task, especially in complex scenes, while omitting other significant attributes present in different regions of the image.
To facilitate the discovery of lowelevel information, avoiding conflicting information in the context of
whole image, we split each image n into set of P non-overlapping patches: P,, = (PP.. P!I }. where P E R Pa Pw and РH. Pwr denote the height and width of each patch respectively. and c the number of channels. In this context, each patch now treated as a standalone image. To this end, we first compute their similarities with respect to set of low-level concepts Ar. For each image n split into P patches, the cotches-coneepts similarity computation reads:
[L)n (P,)
RPL
Vn
We define a single classification layer with weights W E CCL, while for obtaining a single rep- resentation vector for each image, we introduce an aggregation operation to combine the information from all the patches. This can be performed before or after the linear layer. Here, we consider the latter, using a maximum rationale. Thus, for each image n, the output L) E R reads:
Y 12 max [|L W...WT..
ER,
Vn
(8)
Le
where H) denotes the p-th row of the matrix. Similar to the high-level, we define the corresponding concept discovery mechanism for the low level to address information redundancy; then, we introduce an information linkage between the different levels towards context sharing between them.

---

Concept Discovery. For each patch of image we consider latent variables Iz, E {0,1)*, 1)", operating in an "on"-"off" fashion as before. Specifically, we introduce an amortization matrix Wis E l K being the dimensionality of the embeddings. In this setting, [Zr.ng are drawn from Bernoulli distributions driven from the patch embeddings, S.L.:
L(Zpp) Bernoulli [Z sigmoid (Er(PP)))) L. LD"))
(9)
The output is now given by the inner product between the discovered low level concepts as dictated by z and the weight matrix W Les yielding:
Y Lh = max ( (1Zz), [SL]] W Le T
E RR,
Vn
(10)
The formulation the low-level, patch-focusee variant is now concluded. This module can be used standalone network to uncover information residing in patch-specific regions of image and investigate the network's decision making process as shown in (Lower Right). However, by slightly altering its definition, we can straighffrwarrdy introduce linkage between the described levels, allowing the flow of information between them.
3.3 Linking the two levels
For formulating finer concept discovery mechanism, introduce the notion of concept hierarchy. In this setting, we do not assume individual concepts sets, but instead posit that each high-level concept in Aµ is characterized by L low-level attributes, such that |Az] H x L. For tying the two different levels together, we augment the dimension of the low level module to account for this modification, and exploit the resulting latent variables ZH and Z devise novel way of deciding which low-level attributes should be considered according to the active high level concepts dictated by Zr; this allows for context exchange between the two levels and end-to-end training.
The underlying principle is that we can now mask the low-level concepts, i.e., zero-oul the ones that irrelevant, following top-down rationale. During training, learn which high-level concepts are active, and subsequently discover the essential low-level attributes, while the probabilistic nature of our construction allows for the consideration of different configurations of high and low level concepts. This leads to a rich information exchange between the levels of the network towards achieving the downstream task.
High
Concept Discovery Block
Lavel
Concanes Seaheiiinr
A2i
Concage Discooery Block
Text
A3::
Linoe
Leni
Class
Pepil
Albe
Laye:
Link
Eutlediig
Encoder
Image
A
WTw
Coocept Disomary
Beaodl (Aaus))
Lou
W. gras
dargee
Crocept Disomary Block
Falckiy Enagge
Cheespl Block
Level
Figure 1: (Left) The Concept Discovery Block (CDB). Given a of concepts and image, compute their similarity via a VLM; we consider a data-driven mechanism for concept discovery, sampling from amortized Bernoulli posterior. (Right) A schematic of the envisioned CF-CBMs. We consider set of high level concepts, each described by number of attributes; this forms the pool of low-level concepts. Our objective exploiting information residing in, in this case P = 9, patch-specific regions by matching low-level concepts to each patch and aggregate the information to obtain a single representation. Each level comprises CDBs, while the levels are linked together via the binary indicators Z ZH and Z
discover concepts that describe the whole image, while
To formalize this linkage, we first consider which high-level concepts are active via fr to uncover which low-level attributes should be considered; then, we use the augmented indicators Z₁ to further
5

---

mask the remaining low-level attributes according to the values therein. This yields:
Z m. & Σ In.
[Z - E 10,
(11)
Thus, by replacing the indicators Z in Eq. (T)) with Z, the two levels linked together and can be trained in an end-to-end fashion. A graphical illustration of this linkage and the proposed Coarse-to-Fine CBM (CF-CBM) is depicted on Fig. (Middle Right) and (Right) respectively. The introduced framework can easily accommodate more than two levels of hierarchy, while allowing for the usage of different input representations, e.g., super-pixels.
3.4 Training& & Inference
Training. Considering a dataset D = {(X,, Wn) we employ the standard cross loss, denoted by (É f f(X A)). where f(x,,, n. A) Softmax((Y]nn are the class probabilities. For the simple concept-based model, i.e., without any discovery mechanism, the logits [Y], correspond to either [Y al (Eg. 4 ), or [Y 21n (8), depending on the considered level. In this context, the only trainable parameters are the classification matrices for each level, i.e., W He or W WLe- Le-
For the full model, the presence of the indicator variables, i.e., 51 and/or z. necessitates a different treatment of the objective. To this end, and in line with recent literature [16] 17]. we turn to the Variational Bayesian (VB) framework, and specifically to Stochastic Gradient Variational Bayes (SGVB) [8. We impose appropriate prior distributions the latent indicators н and Z L. S.L.:
ZH ~ Bernoulli(aH),
Z ~ Bernoulli(a.)
(12)
where ou and ar, are non-negative constants. When the levels are linked together, the model comprises two outputs, and thus, the loss function consists of two distinct CE terms: (i) one for the high, and (ii) one for the low level. The objective function takes the form of an Evidence Lower Bound (ELBO) provided in the Appendix. Obtaining the objective for single level is trivial; one only needs to remove the other level's terms. For training, we turn to Monte Carlo (MC) sampling using a single reparameterized sample for each latent variable. Since, the Bernoulli is not amenable to the reparameterization trick [8. we turn to its continuous relaxation, i.e., the Gumbel-Softmax trick [10 6]; present the exact sampling procedure in the appendix.
Inference. After training, we can directly draw samples from the learned posteriors and perform inference. However, the stochastic nature of the indicators could potentially lead to multiple interpre- tations for the same input when drawing different samples. Commonly, there are two approaches to address this issue in the VB community: (i) draw multiple samples and average the results, or (ii) directly use the mean as an approximation to the aforementioned process; here, we opt for the latter. In our framework, the binary indicators are modeled via a Bernoulli distribution; thus, the mean corresponds to the probability interpretable threshold т; if the probability of a particular concept is
concept being active for a particular example. Within this context,
can further introduce greater than 7, we consider it to be active, i.e., its indicator has a value of one, and zero otherwise. We use this formulation during inference, obtaining a single sparse interpretable representation for each image.
Experimental Evaluation
Experimental Setup. We consider three benchmark datasets for evaluating the proposed framework, namely, CUB 21) SUN(3), and ImageNet-Ik[3] These constitute highly diverse datasets varying in both number of examples and applicability: ImageNet is a 1000-class object recognition benchmark. SUN comprises 717 classes with a limited number of examples for each, while CUB is used for fine-grained bird species identification spanning 200 classes. For the VLM, we turn to CLIP (T8) and select a common backbone, i.e., ViT-B/16. To avoid having to calculate the embeddings of both images/patches and text at each iteration, we pre-compute them with the chosen backbone, Then, during training, we load them and compute the necessary quantities. For the high level concepts, we consider the class names for each dataset. For the low-level concepts, for Imagenet, we randomly select 20 concepts for each class from the concept set described in (4 while for SUN and CUB, we exploit a per-class summary of the included attributes comprising 102 and 312 descriptions respectively. Since these shared among classes and the number of active attributes differ for

---

Table Classification Accuracy and Average Percentage of Activated Concepts (Sparsity). By bold
blue/red, we denote the best persfrerformnn high/low level
concept-based model.
Dataset (Accuracy (%)
Sparsity (%)
Architecture Type
Model
Concepts
Sparsity
CUB
SUN
ImageNet
Baseline (Images)
X
X
76.70
42,90
76.13
Non-Interpretable
CLIP Embeddings"
x
X
81,90
65.80
79.40
CLIP Embeddings
X
X
47.80
46,00
62.85
Label-Free CBMs
V
74.59
71.98
Concept-
CDM
x
80.30
66.25
75.22
Whole Image
CDM
/
78.
90 19.00
64.55||13.00
76.55||14.00
High Level
CF-CBM (Ours)
5
79.
(9.50||50.00
64.00||44.55
(00.447.558
77.40
Concept-Based
CDM
x
39.05
37.00
49.20
Patches
CDM
59.62| (0.0002||800 |58.00
(6.00//67.00
58.20[[25.60
Low Level
CF-CBM (Ours)
73. 20 [29.80
|29.880
57.10 [28.32
78.45 [15.00
each class, we devise an efficient alternative linkage formulation to accommodate this setting: this provided in the Appendix. These distinct sets enables us to assess the efficacy of the proposed framework in highly diverse configurations. We use P = 16 patches and set 0.05; this translates to concept being active for the input only if has probability of being active greater than 5%. We observed no significant variation when using smaller values. Larger values translate to fewer concepts being active, despite having significant activation probability before thresholding. We consider both classification accuracy, as well as the capacity of the proposed framework towards interpretability.
Accuracy. We begin our experimental analysis by assessing both the classification capacity of the proposed framework, but also its concept sparsification ability. To this end, we consider: (i) baseline non-intepretable backbone, (ii) the recently proposed SOTA Label-Free CBMs [3] (iii) classification using only the clip embeddings either of the whole image (CLIP Embeddings H or the image's patches (CLIP Embeddings"), (iv) classification based on the similarity between images and the whole concept set (CDM# Xdiscovery). and (v) the approach of 17 that considers data-driven concept discovery mechanism only on the whole image (CDM# discovery), We also consider the proposed patch-specific variant of CDMs defined in Sec.
3.2 denoted by CDM-. The baseline results and the Label-Freee CBMs are taken directly from 13 We denote our framework as CF-CBM.
In this setting, CLIP" and CDM" consider the concept set Ar, while patch-focused models, i.c., CLIPL and CDM-, solely consider the low-level set AL. Here, it worth noting that the CDM setting corresponds to a variant of the full CF-CBM model, where all the high level concepts are active; thus, all attributes are considered in the low-level with no masking involved. In this case, since the binary indicators Z are not used, there is no information exchange taking place between the levels; this serves as an ablation setting of the impact of the information linkage. The obtained comparative results are depicted in Table [ Therein, we observe that the proposed framework exhibits highly improved performance compared to Label-Free CBMs, while on par or even improved classification performance compared to the concept discovery- based CDMs on the high-level. On the low level, our approach improves performance up to R 20% compared to CDM
At this point, it is important to highlight the effect of the hierarchical construction and the linkage of the levels to the overall behavior of the network. In all the considered settings, we observe: (i) a drastic improvement of the classification accuracy of the low-level module, and (ii) a significant change in the patterns of concept discovery on both levels. We posit that the information exchange that takes place between the levels. conveys context of the relevant attributes that should be considered. This is reflected both to the capacity to improve the low-level classification rate compared to solely using the CLIPL or CDM-, but also on the drastic change of the concept retention rate of the low level. At the same time, the patch-specific information discovered on the low-level alters the discovery patterns of the high-level, since potentially more concepts should be activated in order to successfully achieve the downstream task. This behavior is extremely highlighted in the ImageNet case: our approach not only exhibits significant gains compared to the alternative concept-based CDM on the high-level, but also the low-level accuracy of our approach outperforms it by a large margin. These first investigations hint at the capacity of the proposed framework to exploit patch-specific c information for improving performance on the considered downstream task.
Attribute Matching. Even though classification performance constitutes an important indicator of the overall capacity of a given architecture, it is not an appropriate metric for quantifying its behavior within the context of interpretability. To this end, and contrary to recent approaches that
7

---

Table 2: Attribute matching accuracy. We compare our approach the recent CDM model trained with the considered Ag set. Then, we predict the matching between the inferred per-example concept indicators to: (i) class-wise and (ii) per-example ground truth attributes found in both SUN and CUB.
Datasct (Matching Accuracy (%)] Jaccard Index (%)
Model
Attribute Set Train
Atrribute Set Eval
SUN
CUB
CDM 112
whole sel whole set
class-wise
51.43 26.00
29.00 17:220
CDM
class-wise
30.95 26.70
25.81 [99.6
CF-CBM (Ours)
hierarchy
class-wise
53.10 28 20
79.85 [32 50
CD7
whole set whole hicrarchy
example-wise
48.45 15.70
36.15 [09.50
CDM
example-wise
20.70
15.00
17.65 110.40
CF-CBM (Ours)
example-wise
49.92 16 80
81.00 |17.60
solely rely on classification performance and qualitative analyses, introduce metric measure the effectiveness of a concept-based approach. Thus, we turn to the Jaccard Similarity and compute the similarity between the binary indicators z that denote the discovered concepts and the binary ground truth indicators that can be found in both CUB and SUN; we denote the latter as zst.
Let denote by: (i) M11 the number of entries equal to in both binary vectors, (ii) M0,1 the number of entries equal to 0 in 2, but equal to 1 in œ and (iii) M1 o the number of entries equal to 1 in z, but equal to 0 in 28t; consider the asymmetric case, focusing detecting the presence of a concept. Then, we can compute the Jaccard similarity as:
the importance of correctly
Jaccard (2, (M,1/(M,, + M1.0 + Mo,1)
(13)
This metric can be exploited as objective score for evaluating the quality of the obtained concept- based explanations across multiple frameworks, given that they consider the same concept set and ground truth indicators exist.
For a baseline comparison, we train a CDM with either: (i) the whole image (CDM) or (ii) the image patches (CDM*), using the whole seí of low-level attributes as the concept set for both SUN and CUB. We consider the same set for the low-level of CF-CBMs; due to its hierarchical nature however, CF-CBM exploits concept hierarchy as described in See 3.3 to narrow down the concepts considered on the lowelevel. For both SUN and CUB, we have ground truth attributes on a per-example basis (example-wise), but also the present attributes per class (class-wise) We assess the matching between these ground-trute indicators and the inferred indicators both in terms of binary accuracy, but also in terms of the considered Jaccard index.
In Table 2 the attribute matching results are depicted. Therein we observe, that our CF-CBMs outperform both CDM and CDM in all the different configurations and in both the considered metries with up to 15% absolute improvement. These results suggest that by exploiting concept and representation hierarchy, we can uncover more relevant low-level information. However, it is also important to note how the binary accuracy metric can be quite misleading. Indeed, the ground truth indicators, particularly in CUB, are quite sparse; thus, if model predicts that most concepts not relevant, we yield very high binary accuracy. Fortunately though, the proposed metric can successfully address this false sense of confidence
more appropriate measure for concept matching.
Ablation Study, One important aspect of the proposed CF-CBM framework is the impact of the considered number of patches and subsequently the number of Concept Discovery Blocks on the low level. To this end, we perform an ablation study on this parameter, using the CUB dataset. We consider five configurations: (i) P = 4, (ii) P P= 9, (iii) P = 16, (iv) P = 64, and (v) P P= 256. The obtained results are presented in Table 3 We observe that setting the number of patches to or 64 yields significant performance improvement in the attribute matching capability. In this context, it is very important to emphasize that despite the fact that classification wise, the performance is similar to other configurations, the capacity of the model towards interpretability is substantially different. Consequently, it is essential to have an objective measure of the interpretation capacity in the context of interpretability focused methods as the one proposed in this work.
Qualitative Analysis. For our qualitative analysis, we focus on the ImageNet- 1k validation set; this decision was motivated by the fact that it is the only dataset where attribute matching could not be assessed due to the absence of ground-truth information. Thus, in Fig. - we selected a random class (Sussex Spaniel) and depict: (i) the 20 originally considered concepts and (ii) the results of the concept discovery. In this setting, we consider a concept to be relevant to the class it is present in more than 40% of the examples of the class; these concepts are obtained by averaging over the class
8

---

Table An ablation study on the effect of the number of patches used on the low-level on the CUB dataset for both classification and interpretation capacity of the CF-CBM framework.
P
Acc. Spars.
High (%)
Acc.
Spars. Low (%)
Example-wise Jaccard (%)
Class-wisc Jaccard (%)
79,05
55.00
73.70
35.27
16.60
27.20
79.20
47.80
72.00
27.00
16.10
27.20
16
79.50
50.00
73.20
29.80
17.60
32,50
64
79.00
47.60
73.40
29.00
18.00
31.50
256
79.15
47.60
73.40
25.00
16.70
27.40
examples' indicators. We observe that CF-CBM is able retain highly relevant concepts from the original set, while discovering equally relevant concepts from other classes such as australian terrier, seft-coated wheaten terrier and collie.
Original Concepts
Discoverod Concopts
ppttve/ beeed
Euppy object
corted Jow
objests drsepung
prone barking amerage spon years devoted to its family
logs coverod in sdart hair
live averge 10-12 yoar
short, level back
always ceads to play
reluttvely weighing cet pounds
object short
berwis
may have slght curl
loven plry//nnds exerciinn
month roudd
bead/geinted object
small Dimpact heeed
good companion dog
inelligent breed
dart brown
small speniel
peens some bealth groblems/otgen infections idfelti
black anse daakk boows eyea
good leced foe
childhen
few major bealth concems
innelligent quick leamners
(desee chestat
jove play and nuz simply stunning
level copline
Figure 2: Original and additional discovered concepts for the Sussex Spaniel ImageNet class. By green, denote the concepts retained from the original low-level pertaining to the class, by maroon, concepts removed via the binary indicators Z, and by purple, the newly discovered concepts.
Original Concegte Set
(Black body /witee teathers hard revered and respected
Found wathands Amsraln/NZ very elepant hind
Black festbers /wite stipes mnshationn persed 92-34
National lcon magmifional bind beoutted black festbers Black w/ while wingtpp become aggressive il itrtatened koppt in zoes ball red very peníle wingspan Decing Eoom preditors iswim up to 30ammh perform accobutic water
2.5
enjoyed by birdwanchers
gracefull bad that can swam and objest gond swimmer
Discovered Conceuts Puches
light brown small wodiigg bad hack/mron
head delic/ songhird
head
and a black
shaped with pointy end
smootth wet dkin long /eendee body wath small while sputs;
light brown cost with dorker spots heceds tundea "oort light. color dupp places anwlland blcc. black
A random example from the Black Swan class of ImageNet- Ik validation set. On the upper part, the original concept set corresponding to the class is depicted; on the lower, some of the concepts discovered via our novel CF-CBM.
Figure 3:
In Fig we focus on the wise behavior of the proposed framework. To this end, and for a random image from the ImageNet-IK IK validation set, we illustrate: (i) the original low-level set of attributes describing its class (Black Swan), and (ii) some the low-level attributes discovered by our CF-CBM. We observe that the original concept pertaining to the class cannot adequately represent the considered example. Indeed, most concepts therein would make the interpretation task difficult even for human annotator. In stark contrast, the proposed framework allows for more interpretable set of concepts, capturing finer information residing in the patches; this can in turn facilitate a more thorough examination of the network's decision making process.
In this context, and since we have access to all the concepts/attribtes discovered on the image and patch level, we can examine and visualize the discovered concepts for each patch and assess their validity. We provide such a visualization in Fig. 4 where for the Black Swan image of Fig we present five of the most contributing discovered attributes the patch-wise level, i.c., attributes that are inferred to be active for this example, encoded in the binary masks Z, and that have large contribution to the classification decision. Therein, observe that our approach is able to exploit the information residing in localized regions of the image, granting significant insights towards the decision-making process of the proposed framework.
Additional qualitative investigations with respect to the inferred concept patterns are provided in the Appendix.

---

itade up satie savvll
dack gey plasage tet itation and deup woers loeg, ihan, me aroster zary bice odse Nark
deld bassng
feld bastag mny the color algay mustecare
betl shallww and day wiler dease and sany color very rish deop tany cake
wading abalive wazm
poy plamage colseed
Jagg groetid body vay trzullgent and cusioss cmadly toown black poaned
molean-sied
bvth shalloe and log videoe desse and wayy racy buu color trauissor.
dafk grey planage black cbject may bls coler sligilly traileces
Mad d'po
decae and wary sare otte Bildd slgbtly tuaabaeee
preee
Figure 4: Five of the most activated concepts per patch for Black Swan image from the ImageNet validation set. After training, we have access to the full set of activated concepts for each patch allowing the examination of the inferred active attributes. This facilitates the examination of their validity, while providing insights for the decision making process of the proposed CBM-based framework.
5 Limitations & Conclusions
potential limitation of the CF-CBM framework is the dependence the vision-language backbone. The final performance and interpretation capacity is tied to its suitability with respect to the task at hand. If the embeddings cannot adequately capture the relation (in terms of similarity) between images/patches-coneeppp
there is currently no mechanism to mitigate this issue. However, our construction easily accommodates adapting the backbone. Concerning the complexity of the proposed CF-CBM framework, by precomputing all the required embeddings, the resulting complexity is orders magnitude lower than training a conventional backbone. more thorough discussion on the limitations and complexity of our approach is presented in the Appendix.
In this work, proposed an innovative framework in the context of ante-hoc interpretability based on a novel hierarchical construction. We introduced the notion of concept hierarchy, which, high-level concepts characterized by number of lower-level attributes. In this context, we leveraged recent advances in CBMss and Bayesian arguments to construct an end-to-end coarse-to-fim network that exploit these distinct concept representations, by considering both the whole image, well its individual patches; this facilitated the discovery and exploitation of finer information residing in patch- specific regions of the image. We validated our paradigm both in terms of classification performance, while considering new metric for evaluating the network's 's capacity towards interpretability. As we experimentally showed, we yielded networks that retain or even improve classification accuracy, while allowing for more fine-grained investigation of their decision process.
Acknowledgements
This work was supported by ANR project OBTEA
10

---

References
[1] David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proc. CVPR, 2017.
[2] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proc. ICML,, 2020.
[3] Jia Deng. Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proc. CVPR, 2009.
[4] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerehner. beta- VAE: Learning basic visual concepts with a constrained variational framework. In Proc. ICLR, 2017.
[5] Matthew D. Hoffman, David M. Blei, Chong Wang, and John Paisley. Stochastic variational inference. Journal of Machine Learning Research, 2013.
[6] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparametrization with gumbel-softmax. In Proc. ICLR, 2017.
[7] Eunji Kim, Dahuin Jung, Sangha Park, Siwon Kim, and Sungroh Yoon. Probabilistic concept bottleneck models. In Proc. ICML, 2023.
[8] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Proc. ICLR, 2014.
[9] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Proc. ICML, 2020.
[10] Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. In Proc. ICLR, 2017.
[11] Dhruv Mahajan, Sundararajan Sellamanickam, and Vinod Nair. A joint learning framework for attribute models and object descriptions. In Proc. ICCV, 2011.
[12] Diego Marcos, Ruth Fong, Sylvain Lobry, Rémi Flamary, Nicolas Courty, and Devis Tuia. Contextual semantic interpretability. In Proc. ACCV, 2020.
[13] Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, and Tsui-Wei Weng. Label-free concept bottleneck models. In Proc. ICLR, 2023.
[14] Tuomas Oikarinen and Tsui- Weng. CisPedissect Automatic description of neuron repre- sentations in deep vision networks. In Proc. ICLR, 2023.
[15] Konstantinos Panousis and Sotirios Chatzis. Discover: Making vision networks interpretable via competition and dissection. In Proc. NeurlPS, 2023.
[16] Konstantinos Panousis, Sotirios Chatzis, and Sergios Theodoridis. Nonparametric Bayesian deep networks with local competition. In Proc. ICML, 2019.
[17] Konstantinos P. Panousis, Dino Ienco, and Diego Marcos. Sparse linear concept discovery models. In Proc. ICCCW CLVL, 2023.
[18] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal. Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In Proc. ICML, 2021.
[19] Vikram V. Ramaswamy, Sunnie S. Y. Kim, Ruth Fong, and Olga Russakovsky. Overlooked fac- tors in concept- explanations: Dataset choice, concept learnability, and human capability. In Proc. CVPR, 2023.
[20] Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. In Proc. NeurIPS, 2016.
11

---

[21] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The caltech-uesd birds-200-2011 dataset. Technical report, 2011.
[22] Eric Wong, Shibani Santurkar, and Aleksander Madry, Leveraging sparse linear layers for debuggable deep networks. In Proc. ICML, 2021.
[23] Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In Proc. CVPR, 2010.
[24] Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Bure Buriso and Mark Yatskar. Language in bottle: Language model guided concept bottlenecks for interpretable image classification. In Proc. CVPR, 2023.
[25] Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. In ICLR 2022 Workshop on PAIR=2Struct: Privacy, Accountability, Interpretability, Rohustness, Reasoning on Structured Data, 2022.
[26] Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region- basedd language-image pretraining. In Proc. CVPR, 2022.
12

---

Appendix
A Training and Experimental Details
Considering dataset D = {(X,, n: Dn))i=n we employ the standard cross-entropy loss, denoted by CE(Ú.. f(X,, A)), where f(X,,, n: A) [oftmax([Y]] are the class probabilities. For the simple concept-based model, i.e., without any discovery mechanism, the logits [У], correspond to either [Y (Eq. (4)) (Y z.1 (Eq.(8), depending the considered level. In this context, the only trainable parameters are the classification matrices for each level, i.e., W He or W Le-
For the full model, the presence of the indicator variables, i.e., ZH н and/or L, necessitates different treatment of the objective. To this end, we turn to the Variational Bayesian (VB) framework, and specifically to Stochastic Gradient Variational Bayes (SGVB) [8.
In the following, we consider the case where the levels are linked together. Obtaining the objective for a single level is trivial; one only needs to remove the other level's terms. Since the network comprises two outputs, and the loss function consists of two distinct CE terms: (i) one for the high-level, and (ii) one for the low-level. The final objective function takes the form of an Evidence Lower Bound (ELBO) S
CELBO = Σ CF( An. ZBlw) CF( AZ.(Z(m))
(14)
+
where augmented the CE notation reflect the dependence on the binary indicators. 3 is a scaling factor 4 to avert the KL term from dominating the downstream task. The KL term encourages the posterior to be close the prior; setting (g ,z to very small value "pushes" the posterior to sparser solutions. Through training, we aim to learn which of these components effectively contribute to the downstream task.
For computing Eq. (14), we turn to Monte Carlo (MC) sampling using a single reparameterized sample for each latent variable. Since, the Bernoulli is not amenable to the reparameterization trick we turn its continuous relaxation using the Gumbel-Softmax trick [10 6].
A.1 Bernoulli Relaxation
Let us denote by 2,, the probabilities of g(2;), i= I....N, N. We can directly draw reparameterized samples & E (0, 1)M from the continuous relaxation as:
2 exp (log 2, + L)/T)
(15)
where L E R denotes samples from the Logistic function, such that:
L log U log(1 U), U ~ Uniform(O., 1)
(16)
where is called the temperature parameter; this controls the degree of the approximation: the higher the value the more uniform the produced samples and vice versa. We set r to 0.1 in all the experimental evaluations. During inference, we can use the Bernoulli distribution to draw samples and directly compute the binary indicators.
A.2 Experimental Details
For our experiments, we set Ohz a2 = 3 we select the best performing learning rate among (10-1 4, 10-3, 3,5- 10-3 3, (0-3,10-2) 10 for the linear classification layer. We set a higher learning rate for W H. and W L. (10x) to facilitate learning of the discovery mechanism.
For all our experiments, we use the Adam optimizer without any complicated learning rate annealing schemes. We trained our models using single NVIDIA A5000 GPU with no data parallelization. For all our experiments, we split the images into extracted by resizing first and passing through CLIP to match the standard CLIP input size. To
16 patches. The patch level CLIP embeddings
13

---

this end, we first resize the patch to 224. For SUN and CUB, we train the model for maximum of 1000 epochs, while for ImageNet, we only train for 100 epochs.
For the baseline non-interpretable backbone, we follow the setup of LF-CBM 13 Thus, for CUB and SUN we consider RN18 model and for ImageNet, a RN50.
Complexity. For SUN and CUB, training each configuration for 1000 epochs, takes approximately 10 minutes (wall time measurement), while for ImageNet, 100 epochs require approximately 4 hours.
Variability Investigation. Given the probabilistic nature of our proposed framework, we investigate the factors of variability in the experimental results. In the context of CF-CBMs, these factors pertain to both the initialization of the parameters of the network and the random drawing of samples for the concept presence indicators throughout training. Thus, by performing multiple runs under given experimental conditions, while utilizing different seeds, we can confidently evaluate the variability, We consider the CUB dataset with P = 16 patches and perform ten different runs, each with different random seed. In T the obtained results are depicted; therein, the mean classification accuracy and mean sparsity rate across the ten different runs are reported, along with the computed standard deviation among them. We report both the high and low level values. As we observe, the results are highly consistent, resulting in negligible standard deviation for the classification rates and relatively small sparsity standard deviations 1.00 and 1.40 for the high and low level respectively.
Table 4: Mean accuracy, Mean Sparsity and their standard deviations for both the high and the low
level of our CF-CBM framework. We performed ten different runs under different seeds to assess the effect of initialization and random sampling.
Level Mean Accuracy (%) Standard Deviation High Low
Mean Sparsity (%) Standard Deviation
79.50
0.15
52.00
1.00
73.60
0.22
31.60
1.40
Sparsity Effect Ablation. To evaluate the effect of the sparsity inducing mechanism on the obtained classification and interpretability results, we consider ablation study, where we vary the Bernoulli prior probabilities an, 0. that directly affect the obtained sparsity. The higher the value of each respective a, the less sparse are the obtained results; this allows for assessing the impact of the sparsity inducing behavior the proposed model. The results are depicted Table in the performed study, we consider the same value for az and or,, denoted a. Therein, we observe that the sparser the representation, the better the attribute matching capabilities of the model, while at the same time the classification performance is retained even improves compared to less sparse setting. These results also highlight the necessity of another metric apart from the classification accuracy to assess the interpretation capabilities of the resulting models, since all settings exhibit similar performance.
Table 5: An ablation study the impact of the sparsity level in the resulting accuracy and Jaccard Similarity. We vary the values a a, of the Bernoulli priors (using the same value denoted as a); lower values lead to sparser representations, while values close concept sets.
lead denser discovered
Accuracy (%)
Sparsity (%)
Jaccard Similarity
High Level
Low Level
Class Wise
Example Wise
0.0001
79.50
50.00
73.20
29.80
32.50
17.60
0.001
79.00
55.50
72.70
31.10
29.20
16.60
0.01
78.75
58.20
73.00
32.20
26.70
16.60
0.1
79.00
62.00
73.00
37.00
26.10
16.50
1
77.60
83.70
69.15
43.70
25.00
15.60
Alternative linkage formulation. In the setting where high level concepts share some sub- characteristics, we can devise more efficient way to mask the low-level concepts and reduce the computational complexity of the approach. Specifically, we begin by considering concepts in A₁ and total of I.all low level concepts in the Ar, concept set, which now comprises concatenation of the sub-characteristies of all high level concepts. Evidently, if no sub-characteristics shared by the high level concepts, get اall
high level
H x L as in the general case presented in the
14

---

main text, otherwise, L < H X L. This formulation can also accomodate concepts sets where all concepts are shared and each class has a different number of potential sub-characteristics, as the case for CUB and SUN, where T, is 312 and 102 respectively.
In both cases though, since we know which sub-characteristies characterize each high-level concept h, we can consider a fixed L-sized binary vector bg E {0, _ that encodes this relationship: these are concatenated form the matrix B E [0, _ Each entry (, h therein, denotes if the low-level attribute characterizes the high-level concept h; if so, [B], 1, otherwise [B 0.
To exploit this representation to formalize the linkage between the High and the Low level, first consider which high-level concepts are active via Z H. and use B to uncover which low-level attributes should be considered in the final decision; this is computed via mean operation, averaging over the high-level dimension H. Then, we use the indicators Z r. to further mask the remaining low-level attributes. This yields:
Z x [nnB")
(17)
Thus, by replacing the indicators Z in Eq with Z, the two levels are linked together and can be trained on end-to-end fashion before.
B Discussion: Limitations
Generalizability: The focus on VLMs might limit the generalizability to other domains and data.
To the best of our knowledge, most (if not all) CBM models deal with the same type of data, and we could say that they share similar shortfalls. Evidently, our approach is centered around vision and language models. However, we posit that the consideration of two (or more) different kind of representations//iees through our construction, instead of limiting the generalizability of the approach, it improves it since we can now exploit this structure to address cases where CBMs were lacking. This includes for example cases Remote Sensing data, where one could have both satellite and ground level images, along with textual descriptions for each. This could potentially be generalized to all kinds of data, considering that multimodal models for multiple modalities available. As long as can project the data in common embedding space and compute the similaritics between modalities, we posit that our CF-CBM method is more expressive than other conventional CBM methods due to their structured construction. Nevertheless, assessing the applicability of our approach to other domains is indeed an important part of our future work.
Complexity: How the computationally heavy is the proposed framework?
Compared other approaches that train or fine-tune conventional backbones or resort to complicated training schemes, our approach is relatively lightweight. Specifically, a large amount of the overhead relates to the computation of the CLIP embeddings, a process that takes couple of minutes for each dataset. These are calculated once and from then on, we can use these embeddings as is; at the same time. training comprises learning just the linear layers present in the construction. The computational complexity in this setting is limited, considering we can run 1000 epochs for CUB/SUN in approximately 7 minutes on single GPU for P 4 and P = 9, 20 minutes for P= 64, and 30 minutes for P = 256 patches. We did not use any parallelization for our computations; this could help speeding up the process in case of even larger grids. Inference time negligible, considering that our approach essentially comprises just inner products in a small number of layers.
During inference and for real-time projection into the common embedding space, there is substantial work that is taking place with respect to reducing the overhead in Multimodal and Large Language Models via quantization and other techniques. This would easily allow for on-line projection and estimation of the corresponding values even on low-power commodity devices such as smartphones.
Dependence on the Pretrained Models: The issue of the dependence to the pretrained models is indeed an important aspect of the approach.
As we briefly touch upon in the limitations section, the dependence on the considered lisionelanguage backbone is very important for the final performance of the model. Mitigating this issue a very challenging task; we need to somehow be able to compute the relation between the considered modalities terms of either similarity or some other metric. Thus, we are tied to the usage of a backbone that projects the data in common embedding space or where the representations are
15

---

aligned. CLIP and other contrastively-traine multimodal models at this point constitute the most easy apply and intuitive frameworks. To enhance performance in a case where the backbone is not suitable for the considered task, one potential solution is to finetune the pretrained model on the task and the other to train such a cross-modal model from scratch either deterministically or probabilistically (that could potentially allow for some uncertainty analysis and insights into the predictions). However, both solutions require ground truth data that may be difficult to obtain depending on the considered task. We aim to explore these possibilities in the recent future.
C The Top-down and Bottom-up view of concept design
As noted in the main text, in this work, we consider both high and low level concepts. Within this frame of reference, one design choice that needed to be addressed is the interpretation of the connection between these two sets. In our view, there two ways approach this setting: (i) top-down (which we consider), and (ii) bottom-up. We posit that a combined botton-up-thhe-tto-- down approach is what would most closely follow human-like behavior when analysing an object. However, it is the second step that is more of a conscious process: we first become aware of the whole object, e.g., a bird dog, even if we have subconsciously perceived a lot of low-level cues to reach that conclusion, and then, based on this high-level knowledge, we can draw further conclusions about the nature of the lower-level image characteristics, e.g. interpreting furry texture as either feathers or fur.
In purely bottom-up approach, would first analyse the low-level characteristics, such shapes and textures, and we would then try to reason about the whole context in order to assign them semantics, e.g. ears, tail, fur. In our opinion, there isn't single right approach for solving this problem in the context of interpretability. We posit however, that the information exchange that takes places between the high and the low levels via the learning process of the binary indicators does indeed allows for context information sharing between both levels (in the forward pass only from high to low, but also the inverse during training).
One of the motivations of this work was to be able to examine not only the high level concepts but mainly the low level ones. This could potentially allow for drawing conclusions about the high level concept in terms of the uncovered low level attributes. In this context, we can focus on the discovered low-level attributes themselves and reason the high-level concepts. In our opinion, this is somewhat captured in the proposed framework. Indeed, in the qualitative analyses, we observed that, many times, the discovered low level concepts revealed attributes that semantically connected to various high level concepts.
Future work. In our setting, we assume that there exists known hicrarchyyrelationship between the concepts. However, it may very well be the case that there exists some hidden/latent hierarchy in the ground truth attributes that is not explicitly captured via the construction of the concepts sets. In this case, an interesting extension to our proposed framework would be a compositional bottom-up approach with no priori known hierarchies. Within this context, we could potentially devise method that explicitly integrates the aforementioned bottom-up view, aiming to uncover the hidden hierarchies.
Finally, in this work, we opted for the simplest split of the image into non-overlapping regions, namely non-overlapping fixed-size patches. This modeling decision allowed for demonstrating the impact of the hierarchical construction towards finer concept discovery, while the same time facilitated the investigation of the contribution of each individual level when considering a single general purpose vision language model. To augment the modeling capacity the proposed framework, more involved splitting procedures can be considered for the low level, such as superpixels other specific vision-language backbones such as Region-CI.. 26 We aim to explore these avenues in the near future.
16

---

D Further Investigations and Qualitative Analyses
D.1 Alignment between CLIP similarities and Concept Presence Indicators
To further investigate the behavior of the proposed framework, we cxamine the alignment between
CLIP similarities and the inferred concept presence indicators; this allows for assessing if the inferred indicators only activate the most similar (in the CLIP sense) concepts.
CLIP Simi aemilrriy--ooo Active Indicators Aligument:
High Level
077
03
00
080
5.5 2.0
615 9-30 33HN00 A0 S0OP
1813
CHIaawibbyy Value
Figure 5: Alignment between the inferred concept presence indicators and CLIP similarities on the High Level of the CF-CBM framework. We split the CLIP similarities into bins of size 0.05; in each bin we count the number of concepts assigned therein (according their CLIP similarity and denoted by #Conc) and we compute the fraction of inferred active concepts to said number. We observe that in this case, the higher the similarity, concepts with high similarity value exhibit largest percentage of activation.
For this exploration, we focus on the CUB dataset, split the CLIP similarities into bins of size 0.05 and compute the ratio of activated concepts to the total number of concepts across all the examples of the validation set in each bin. The obtained statistics for the High Level are depicted in Fig. We observe that average, concepts with high CLIP similarity exhibit larger percentage of activation in this level. Investigating this behavior on individual examples, e.g.. Fig reveals the same trend; however, though overall observed the same trend in multiple examples, the same time, can also observe the flexibility of the per-instance selection, where in several examples, concepts that have lower CLIP similarity values also be active, even having examples with low CLIP similarity values more active than high CLIP similarity concepts as in Fig.
The corresponding results for the Low level depicted in Table In contrast to the high level, we observe that on average, the concept activations with intermediate CLIP similarity values are more active. We posit that the high level can potentially exploit the information provided via the high
CLIP Semilarity Active Indicators AliireeAAlllli Alignmentt Esample 50 High Lavel
07.
05
03
C'nc
#Cne
#Cwc
KOWE
Figure 6: Alignment between the inferred concept presence indicators and CLIP similarities on the High Level of the CF-CBM framework for Example 50 the CUB validation set. this instance, we observe the same pattern as in Fig. 5
17

---

CLIP Sintlariy Alignment:
Example Level
04
ENITAE
CLI Sanikatyy Valce
Figure 7: Alignment between the inferred concept presence indicators and CLIP similarities on the High Level of the CF-CBM framework for Example 5720 in the CUB validation set. In this instance, and contrary to the previous illustrations, we observe different trend, where concepts with low-siiilarity value exhibit largest percentage of activation compared to concepts with high similarity.
CLIP Simillritty Active Indicaters Alignment:
0.29
0.55
035
CLI
Figure 8: Alignment between the inferred concept presence indicators and CLIP similarities on the Low Level of the CF-CBM framework. We split the CLIP similarities into bins of size 0.05; in each bin we count the number of concepts assigned therein (according to their CLIP similarity and denoted by #Conc) and we compute the fraction of inferred active concepts said number. We observe that in this case, concepts with average similarity values have a higher percentage of activation compared the concepts with the highest similarity.
level concepts, using the most similar ones, i.e., the ones that have high similarity with each instance, towards the classification task. On the contrary, on the low-level, the given concepts may very well describe several different patches; thus, the discovery mechanism aims to compensate the lack of meaningful classification signal through the diversification of the used concepts; this results in the utilization of concepts that have a lower -CLIP based- similarity in order to achieve classification. Nevertheless, the flexibility the proposed framework allows specific examples to deviate from this trend, and individually exhibit distinct concept activation patterns to achieve the downstream task, e.g., Fig.
D.2 Concept Activation Patterns
As already discussed, one important aspect of the proposed CF-CBM framework concerns the flexibility of the per-example presence indication mechanism. In stark contrast to other commonly used pparsity-inducing approaches, the considered concept discovery mechanism allows for inferring essential number and combination of active concepts towards achieving the downstream task. In turn, this leads to unique patterns of activations on both the individual examples but consequently on class-wise level. To explore the emerging patterns, we plot the sum of active indicators of all examples in class with respect to the considered concepts. The results for the High Level for
18

---

CLIP Sinclanty Indicators Alignment:
Prample S12 Level
032
0.5
0.35
0.22
EUVO LIN
Sariaara Valce
Figure 9: Alignment between the inferred concept presence indicators and CLIP similarities on the Lowe Level of the CF-CBM framework for Example 512 in the CUB validation set. In this instance, and contrary to the average pattern in the low level, we observe that the concepts with the highest similarity value exhibit the largest percentage of activation.
four random classes for the CUB dataset are illustrated in Fig. 10 We readily observe that each class yields distinct pattern of concept activation. Within each class, there are several concepts are shared by a number of examples as one would expect; at the same time however, we observe how the data-driven concept discovery mechanism allows individuals examples yield different activation patterns. This leads to concepts that are only active in one or two examples of the class, which were inferred nevertheless essential for their representation. We observe the same behavior in the low-level of the CF-CBM framework, depicted in Fig 11 Despite yielding a highly sparse representation, observe distinct patterns of concept activations per class, retaining nevertheless the flexibility of modeling individual examples.
D.3 Discussion: Differences with & sparsity.
One of the most commonly used sparsity inducing methods is 4, However, in view, it is very restrictive in the context of interpretability. This regularization typically imposed on the weights of the linear layer of concept-based model; this would result into the uunanteed- effect of turning off concepts completely for all images or on a per class basis using more complicated solvers. At the same time, it typically requires some kind of ad-hoe and unintuitive sparsity-accuracy trade-off, while the solvers are typically applied in post-hoc manner, c.g. [22] [3). This greatly limits the flexibility of the approach, since either all images in general or all images particular class must follow the "global" or "class" pattern found by the solver, potentially omitting important information present in each individual image. On the other hand, we consider a per-instance concept discovery mechanism that is based on simple data-driven Bernoulli distribution, which can explicitly denote concept relevance without confining the results in class or global representation, while training can be performed end-to-end using Stochastic Gradient Variational Bayes.
In this context, and discussed in Section D.2 there many cases where concepts do not contribute to a specific class at all, i.e., no instance of the class activated said concepts, potentially yielding similar conclusions to based methods. However, we also observed cases where a single instance in the class was using particular concept. The same was also true for concepts active for only two or three instances. Within this context, expect that the use of the conventional é fssspasyy loss in a post hoc manner, along with the requirement for ad-hoc sparsity/performance thresholds would eliminate this within-class information; at the same time, applying such an l based approach to framework comprising two different connected levels and subsequently two distinct classification losses, at least in principled way, is at this point, not clear.
D.4 Further Qualitative Analysis
19

---

High Concepts Stack Sorood Albatrone
Hligh Conceptss: Ankket
10
20
15-
Active High Level Concepts
Active High Level Conceptis
Homed Goobe
La Cuntee Sparton:
30
10
100
200
100
150
250
Cooopet
Concept
Figure 10: Summary of Activation of High Level concepts for Classes: Black Footed Albatross, Least Auklet, Horned Grebe and Le Conte Sparrow. We readily observe that each class yields a different pattern of concept activation. Within each class. there are several concepts shared by number of examples as one would expect; at the same time however, we observe how the data-driven concept discovery mechanism allows individuals examples yield different activation patterns. This leads to concepts that are only active in one or two examples of the class, which were inferred nevertheless essential for their representation.
20

---

Level Concepts
25
20-
20
13
10
10
50
200
130 Conrgh
Conupe
Active Low Level Concopts For Class
Actire Low Level Concopis Class:
Homed Geobo
Common Wellwwhnutt
30
30
25-
25.
50
100
180
200
250
300
0 50
100
130
200
250
300
CreupeNuun
CanegrNaas
Figure 11: Summary of Activation of Low Level concepts for Classes: Groove Billed Ani, Eastern Towhee, Horned Grebe and Common Yellowthroat. In this setting, we once again observe very different patterns of concept activations for each class, albeit yielding a highly sparser representation. Similar to the High Level, within each class, there are some concepts examples as one would expect; at the same time however, we observe how the low-level concept discovery mechanism allows individuals examples to yield different activation patterns even this highly sparse setting. This allows for the utilization of concepts that are only active in one or two examples of the class.
shared by almost all class
Original Concegts
Diseavered Cynoepts
long uil helps balance ioe
stay their panents your
small böack
troiical
claws see sharp
kill pryy times
shads cost twioe your
yellowish-breww onlor
adupted hasse esvironment
very agile dog
long hush
jump very high
light brown
small object
small head pointed snout
stunning
lifegen up years
cost gra color almond shape and yellowish brown
often soen
long hair along
dens the now
finds food even snow
resouscetul
Jump up to 30 in the ziz
call Is long and bushy
short, blumi objoct
fiesce
agile crenture
thick fur
budhy lkeess warm
thick
black eyes
asound the wild shoet anubby tail
Figure 12: Original and additional discovered concepts for the Arctic Fox ImageNet class. By green, denote the concepts retained from the original low-level set pertaining to the class, by maroon, concepts removed via the binary indicators Z. and by purple the newly discovered concepts.
21

---

Original Concept
live shallow
monuiceed sdiztiat
pospiltr srunce
deferd Haelf
asyliing
pandy creaitre
tips
beowa
Fan watch
eyes iocated on the top
unteidal pooks
Earryy lorge crah blood nod
end bege dows
contilrcc delicacy
not comalered endagerod
mid avent flooor
Discovered Concapua Patches
mike shalle
vital the acreystem
popular hiking spor majestic inspiring Inturioss
beaumiful place to relax
build
Figure 13: random example from the Rock Crab class of ImageNet- Ik validation set. On the upper part. the original concept set corresponding to the class is depicted, while the lower, some of the concepts discovered via our novel patch-spes speccfic formulation.
Original Concepts
Discovered Concents
essentizl the health the plane
full of color
vast and varted ccocystom
lough priskly skin
repion souuad river
destination worth pootzeting for Jutare generatio
great divenity of life on the obyost
uoed as decoration distinctively bumpy extector
many endagesed spectos
trae wonder of the natural woeld long thin stip of land
home to some the most bevutiul
natuaa heiitage
long bush
versatile
yollow flesh
torms with Hife everything to grotect
import marine ccnsystem
found in warm waters
flresh pale
ouler
dnffeeent colote beautiful and csotic
introdaced Fumpe the 16th century
made by individual pieces
Ialeideozoge of codoes
".perfecct snack
Figure 14: Original and additional discovered concepts for the Corul Reef ImageNet class. By green, we denote the concepts retained from the original low-level set pertaining to the class, by maroon, concepts removed via the binary indicators Z, and by purple the newly discovered concepts.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We claim that the consideration of two-level coarse-to fine representation can increase the effectiveness and interpretability in the context of concept- based classification. Our experimental results validate this claim; we obtain on par and even better classification performance in all the considered settings, while our novel quantitative evaluation based the Jaccard similarity, yields an absolute improvement up 15% compared to related methods.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have included a limitations section in the main text, while discussing other potential issues and how these can be mitigated in the context of our proposed framework in the Appendix. We tested our approach using a diverse range of datasets and have included discussion about complexity.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
23

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
We do not introduce any theorems and proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We include the code implementation of the proposed framework along with the submission; nevertheless, all the necessary details concerning the architecture, processing, hyperparameters, and other settings are clearly described in the main text and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
24

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: The code is available at the provided link with all the necessary instructions, checkpoints and other information.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https SonddeSubmisssooPPoon
/ wttppssccpppbllc/////ccccccpttpt
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The Experimental details are clearly outlined both the main text but also in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We performed a variability analysis considering ten different runs under given experimental conditions with different seeds. We did not observe any significant variability between runs with different seeds. We report the results in the appendix.
25

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report the used computational resources in the appendix. We additionally include wall-time measurements for training the proposed framework for each considered dataset.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted for this paper conforms with every aspect of the NeurlPS Code of Ethics. It did not involve human participants, we used publicly available benchmarks datasets that are not deprecated and there is potential societal harm.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
26

---

Question:
Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This paper aims to address the issue of interpretability of modern deep architectures. Evidently, interpretability is a highly important aspect towards safe and robust deployment of deep networks in real-world settings discussed in the main text. There no obvious negative societal impacts at this stage.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This work does include any high risk data or models, thus no safeguards were necessary.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
27

---

No other assets apart from publicly available ones are used in this work and these are credited.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com /daaaset has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The model and the code introduced in the context this paper are fully described.
Justification:
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We did not use crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
28

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
29


Reviews:
[{"review_id": "JzSudWkPms", "reviewer": "Reviewer_6WPf", "summary": "The authors propose coarse-to-fine concept selection in Concept Bottleneck Models (CBMs). They introduce a concept hierarchy that identifies low-level concepts in local patches of input images, as well as high-level concepts in the overall images.  Additionally, the authors enhance interpretability by considering sparsity in concept predictions. Their proposed model, CF-CBM, achieves high classification performance while maintaining interpretability.", "questions": "1. I would like to gain a clearer understanding of the high-level concepts as defined by the authors. The authors used the class name as the high-level concept set.\n\n1-a. If the object's class is the high-level concept, can we argue that predicting the high-level concept (class) from the low-level concepts in the existing CBM does not establish a hierarchical structure between concepts?\n\n1-b. If the object's class is indeed the high-level concept, it seems that referring to class as high-level concept may lead to confusion for readers.\n\n1-c. When seeking low-level concepts, as the authors propose in local patches, can we always expect to find them? For instance, the characteristic \"small/delicate songbird\" shown in the Discovered Concepts Patches of Figure 3 does not seem to be a property that can be defined at the patch level.\n\n2. Why are there multiple concept discovery blocks for low-level concepts?\n\n3. Where can the discovery mentioned in lines 266-267 be found in the tables?\n\n4. Where can the discussion in lines 341-342 be verified?\n\n5. If low-level concepts are identified at the patch level, it seems possible to trace which patch each concept was found in. Demonstrating that the image patch where the concept was discovered aligns with the concept itself could enhance the interpretability of the proposed method.", "limitations": "The authors have adequately addressed limitations in the Limitations & Conclusions section. Since the proposed method uses a frozen pretrained CLIP as its backbone, the ability to discover concepts may be constrained by the limitations inherent in CLIP's training.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The authors propose a novel evaluation metric based on Jaccard similarity to evaluate concept predictions.\n\n2. The proposed method enhances both classification accuracy and concept prediction accuracy by making predictions from local patches in a sparse manner.", "weaknesses": "1. While the Jaccard similarity metric effectively assesses the alignment between the model's predicted concepts and the actual concepts, it serves only as one aspect of interpretability evaluation. Notably, Table 2 indicates that the Jaccard index is quite low, raising questions about whether such a score sufficiently demonstrates the model's interpretability. Additionally, it would be helpful to clarify what factors contribute to the low Jaccard index.\n\n2. Moreover, the concepts described in the paper appear somewhat ambiguous. I encourage the authors to refer to the Questions section for further clarification.\n\n3. A clearer explanation is needed regarding how the authors' proposed approach enhances classification accuracy and interpretability. Specifically, it would be helpful to understand whether predicting concepts from local patches is effective, if learning class predictions aids in identifying low-level concepts, and how the application of sparsity contributes to these improvements. Additionally, an ablation study is necessary to support these claims. This also includes an explanation of why this approach can be referred to as coarse-to-fine.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful feedback and suggestions.\n\n- **Weakness 1**: *The use of Jaccard similarity.* \n\n> We thank the reviewer for raising this point that can help clarify the importance of using a different metric compared to the standard binary accuracy typically considered in the interpretability domain. We agree with the reviewer that the Jaccard similarity only serves as a partial investigation of a model’s interpretability evaluation. However, the emerging low score in this setting does not reduce the impact of considering this metric in the context of interpretability but instead provides a more realistic measure of the model’s capacity.\n\n> Indeed, attribute prediction is a hard multi-label problem, even for humans. Consider for instance the SUN attributes annotations; therein, every image has been independently annotated by three annotators. Out of all attributes that have been labeled as positive by at least one annotator, only 16.5% have also been labeled as positive by the other two. Evidently, considering this mismatch even between human annotators, we expect that concept-based models will exhibit difficulties in discovering concepts with or without the use of any ground truth data. This is especially true for models that additionally depend on the attribute detection capabilities of VLMs. Nevertheless,  the performance we obtain is as expected, and indeed in line with other recent work [A]. If we compare the concept prediction accuracy of our approach to [B], where the mean average precision (AP) and AUC are reported, we see that we obtain comparable results, although, as expected, the supervised method performs better than our approach by a small margin. Specifically,  for the example-wise setting, we yield an AP of 26.90 and AUC of 67.60 The corresponding results of [B], are AP of 28.35 and AUC of 76.22 for the seen classes and AP of 25.31 and AUC of 72.10 for the unseen ones. Thus, the experimental results suggest that our framework is able to provide similar performance without the use of concept supervision. Yet, we agree that this has potential implications for the interpretability of any CBM approach, although studying this falls beyond the scope of this work. \n\n>[A] Rao, S., et al. \"Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery.\" arXiv preprint arXiv:2407.14499 (2024).\n\n>[B] Marcos et al., Attribute Prediction as Multiple Instance Learning, TMLR, 2022\n\n- **Weakness 2**: *Ablations and clarifications*\n\n> We thank the reviewer for their comments that will help clarify the performed ablation studies, while improving the work with additional results. We’ll clearly state the settings in the experimental section to avoid any potential confusion.\n\n> **Effectiveness of predicting concepts from local patches.**\nPredicting low-level concepts from the patches themselves (without first considering a high level concept detection) is denoted in Table 1 as $CDM^L$, with and without sparsity, while in Table 2, $CDM^L$ denotes using the patches and the low-level concepts with the sparsity mechanism (and again without high level concept detection). In all cases, we observe that predicting concepts solely using patches and low-level concepts, and without any other mechanism, yields subpar performance both classification-wise but also in terms of attribute matching. In contrast, when we add the high-level concept detection process and tie the levels together, the classification and attribute matching performance significantly improve.\n\n> **Learning class predictions (High-level Concepts) aids in identifying the low-level concepts.**\nThis setting indeed corresponds to our CF-CBM method, where we discover the high-level concepts, and then discover the low level ones. From Table 2, we observe that our method significantly improves the concept prediction capabilities of CBMs, improving both on the $CDM^L$ method discussed previously and the CDM method that in this setting considers the whole image and the low-level concepts. Thus the experimental results suggest that learning the high level predictions and propagating this information to the patch level, significantly improves the attribute prediction capabilities of the emerging model. \n\n> **How the application of sparsity contributes to this improvement.**\nWe thank the reviewer for this suggestion, since it will help enrich the results of the work. In Table 1 in the main text, we present $CDM^H$ and $CDM^L$ with and without the sparsity mechanism. Therein, we observe that when using the sparsity inducing mechanism, we obtain on par or even improved classification performance while using only a subset of the concepts. \nAs far as the concept detection capabilities are concerned, if we remove the sparsity mechanism, we are essentially using all the concepts, and we thus lose the ability to explicitly assess which concepts are active to assess the results. \n\n> However, following the reviewer’s proposal, we introduce a novel ablation study, where we vary the Bernoulli prior probability $\\alpha$, that directly affects the obtained sparsity. The higher the value of alpha, the less sparse are the obtained results; this allows for assessing the impact of the sparsity inducing behavior of the proposed model as per the reviewer’s suggestion. The obtained results are depicted in Table 1 in the included PDF. In summary, we observe that the sparser the representation, the better the attribute matching capabilities of the model. These results also highlight the necessity of another metric apart from the classification accuracy to assess the interpretation capabilities of the resulting models, since all settings exhibit similar classification performance. We’ll add these results in the camera-ready.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5OkEfWjXi0", "reviewer": "Reviewer_pbhW", "summary": "This work introduces a novel framework that leverages recent advances in vision-language models and a Bayesian approach for coarse-to-fine concept selection. It introduces the notion of concept hierarchy, allowing high-level concepts to be characterized by lower-level attributes and exploiting granular information in image patches.", "questions": "How can it be ensured that the patches assigned to each image accurately correspond to low-level concepts?", "limitations": "The author mention in the Limitations of the dependence on the vision-language backbone.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The writing is fluent and easily comprehensible.\n- Propose a novel way of assessing the interpretation capacity of CF-CBMs based on the Jaccard index between ground truth concepts and learned data-driven binary indicators. \n- Extensive experiments were conducted to demonstrate that the proposed CF-CBMs outperform other state-of-the-art methods in terms of classification accuracy and interpretability.", "weaknesses": "- Over-reliance on the vision-language backbone's capability might result in poor performance for images from uncommon datasets.\n- There is a lack of experiments on test-time concept interventions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing the qualities of our approach concerning the originality, the significance, the quality and the clarity. \n\n- **Weakness 1**: *Over-reliance on the vision-language backbone's capability might result in poor performance for images from uncommon datasets.*\n\n> This is indeed a limitation that our approach shares with all methods employing VLMs, which could be alleviated by fine-tuning the VLM on a custom dataset. We will explicitly mention this in the related section.\n\n- **Weakness 2**: *There is a lack of experiments on test-time concept interventions.*\n\n> We thank the reviewer for their suggestion. Indeed, examining the concept intervention capabilities of our framework is already part of a future work. Nevertheless, following the reviewer’s suggestion, we performed a preliminary analysis on potential interventions. In our framework, the most appropriate type of intervention corresponds to inaccurate concept activations (similar to “Type 3: Incorrect concept activations” of Label-Free CBMs). \n\n>In this setting, and for each example/patch, we can readily examine and alter the concept activation patterns. To this end, we consider the example provided in Fig. 3 in the main text (and Fig. 1 in the included PDF). For this example, we intervened on the discovered concepts by randomly turning on 10 concepts that contain the words white and feathers, while randomly switching off 10 concepts that contain the word black. We performed this process ten times and observed the changes in the classification decision. Most times, i.e, 7 out of 10, we yielded classes such as quill, african grey, oystercatcher, black stork and penguin and the rest, irrelevant classes such as picket fence, can opener and geyser. We also explored a “by-hand” approach, where we switched off some concepts that were not relevant to the example in consideration, e.g., very strong structure and color depends on the type of spider, and observed no change in the classification decision. These results serve as an initial investigation in the efficiency of intervention in our framework.  We’ll add more qualitative examples in the camera-ready.\n\n- **Question 1**: *How can it be ensured that the patches assigned to each image accurately correspond to low-level concepts?*\n\n> We thank the reviewer for raising this point. Indeed, during inference, we have full access to the per-example discovered concepts encoded in the binary masks $\\mathbf{Z}$; this includes both the image-level and patch-level concepts. Thus, we can visualize the results on every level and examine the behavior of the approach. Such a visualization is provided in Fig. 1 in the included PDF file.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4UiTC7XZYH", "reviewer": "Reviewer_yu5m", "summary": "The author introduced a novel Concept Bottleneck Model (CBM) that facilitates hierarchical concept learning. Specifically, the proposed Concept Discovery Block (CDB) plays a pivotal role in uncovering concepts from preprocessed image-text similarity embeddings by employing a variational Bayesian framework to learn a binary mask. Additionally, by applying the CDB module to each patch-level image to detect low-level concepts and the entire image for high-level concept discovery, information propagation between the two levels leads to robust classification performance through sparse concept learning.", "questions": "Please check out the Weakness section. I listed additional questions as follows:\n- Q1: For the inference phase, the author mentioned two ways to address the stochastic nature of the drawing process for the binary masks (Line 235), and the proposed method leveraged the latter one. Is there any specific reason for this? Have you compared the difference between both?", "limitations": "Please check out the Weakness section.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- S1: As an ante-hoc interpretable CBM, the proposed method is intuitive and requires lightweight computation, which is desirable. \n- S2: The performance of the proposed method is superior to other multimodal CBM baselines.", "weaknesses": "- W1: The class/label designated at the high level and its attributes at the low level are strong hierarchical constraints. So, the proposed method was limited to showing its applicability only in cases with a transparent hierarchical relationship between attributes and classes by specifying the pool of low-level concepts corresponding to each class. This may require burdensome human inspection to configure.\n- W2: Another concern is the fixation of the interpretable threshold to all CDB modules as 0.05. The author described it as the probability value used to determine whether the specific concept is active. However, even if some image patches have the same concept, it is evident that the concept may contribute to each patch to a different extent. Therefore, dynamically adjusting or learning the threshold may perform better than a fixed threshold.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time, consideration and their helpful insights that will help improve the manuscript.\n\n- **Weakness 1**: *The class/label designated at the high level and its attributes at the low level are strong hierarchical constraints.*\n\n> It is true that such a hierarchical relation between concepts is needed by our approach. We have so far performed experiments with existing benchmarks where these relations were given. In the future, we plan on expanding this to concept hierarchies that are already compiled by domain experts: for instance, organ ecosystem->plant species->traits type or disease->symptoms. Note that constructing these types of hierarchies is already common in many fields and is much less burdensome than needing to generate a dataset with per-image concept annotations. In addition, it is worth noting that, although the concept hierarchies in CUB and SUN were manually designed, in the case of the ImageNet dataset, the concepts were automatically derived via an LLM, showing that this process can be automated to some extent.\n\n- **Weakness 2**: *Another concern is the fixation of the interpretable threshold to all CDB modules as 0.05.*\n\n> We thank the reviewer for raising this interesting point that can open the way for a potential follow-up of our current approach. In this context, we would like to note that this threshold is used only during inference, while during training the model is free to adjust the “probabilities” of each concept (on either the high or the low level). Thus, during training, the model decides how useful the concept is and adjusts its probability accordingly. If during inference, the probability of activation is very small, e.g. less than 0.05 (translating to the concept being active 5 out of a 100 times), we could assume that its presence is not essential for modeling the image/patch in consideration. However, we fully agree with the reviewer that this threshold could be adjusted or learned during the process, thus resulting in a more contextualized decision in order to activate(or deactivate) concepts. For this reason, we will discuss this point as a perspective to explore in the revised version of the paper. Please also see our response to Q1 regarding an ablation study when not using a threshold. \n \n- **Question 1**: *Addressing the stochastic nature.*\n\n> The motivation behind this decision was two-fold: (i) complexity of the inference process and (ii) interpretability. Indeed, to obtain better estimates of the active concepts using sampling, we need to draw multiple samples from the learned posteriors and average the results. This increases the complexity of the approach relative to the number of inference samples. At the same time, this process renders the interpretation of the active concepts more laborious, since potentially different sets of concepts may be active per run and by averaging the results it may be more difficult for a practitioner to interpret the results. In contrast, by deterministically deciding which concepts are active via thresholding allows one to readily examine which concepts are active per example. Please also see Table 2 in the included PDF. Therein, we show that we need multiple samples to reach the performance, thus showcasing the increase in complexity and examination.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rRZvUZl0rD", "reviewer": "Reviewer_JnxP", "summary": "This paper presents a type of label free concept bottleneck model for ante-hoc interpretability that incorporates a hierarchichal concept representation. The concepts are represented in a two-level hierarchy with high-level concepts denoting scenes/objects and lower level concepts denoting more specific attributes at a patch-level. Experimentally, the authors show better accuracy and better concept prediction (on datasets with GT annotations) while proposing a metric based on Jaccard index to judge quality of concept prediciton.", "questions": "Q.1 One key question I have is how prone is the model to detect some concept accurately but in an uninterpretable way? In other words how can we be sure the concept detection process itself is well grounded? \nFor eg. For a given image, is the detection of \"red feather\" actually because the model detects a red feather or just because of \"red-colored head\" of a bird? Did you try to investigate this qualitatively via some tools (maybe saliency maps or activation maximization)? How easy is it to find a poorly grounded concept detector? \nConcept detection accuracy is possibly one way to evaluate this. However, a user would still need a qualitative tool for the same when the ground-truth concept annotations are not available.\n\nQ.2 In the qualitative examples I frequently see concepts such as \"become aggressive when threatened\", \"quick learners\", which are not grounded in any visual information but our biological/behavioral/social understanding of the respective objects/classes. How do you view the use of these concepts for the purpose of image classification? \n\nQ.3 Are the patch level CLIP embeddings extracted by resizing the patches and passing through CLIP or using patch-based embeddings of the original input image?", "limitations": "The authors do discuss them separately and clearly (partly in main paper and partly in appendix).", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. I really like the idea of hierarchichal representation of concepts. It is natural and intuitive and novel in the context of concept representations for interpretability. It's specific instantiation here is also reasonable.\n2. Experiments are comprehensive in in terms of multiple, diverse large-scale datasets and baselines generally.\n3. The presentation in general is strong and motivations are clear.", "weaknesses": "1. I have some important concerns about interpretability and soundness of label-free CBMs in general that rely on CLIP embedding similarity for concept prediction. Please see Q.1, 2 in Questions tab.\n\n2. The only baseline I would suggest adding would be standard CBMs, specially for concept prediction accuracy (Tab. 2). \nAlthough I expect standard CBM to perform better given its supervised training, but it'd be interesting to see how much is the performance gap if there is any.\n\n3. It'd be interesting to see (even if qualitatively) how the system behaves with concept intervention like the original CBMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for highlighting the strengths of our work and for raising some interesting questions.\n\n- **Weakness 2**: *Comparison to supervised baseline.*\n\n> Following the reviewer’s suggestion, and given the limited time in the rebuttal, we compare the concept prediction accuracy of our approach to [1], since they already report the performance in terms of average precision (AP) and AUC. As expected, the supervised method performs better than our approach, but not by a great margin. Specifically,  for the example-wise setting, we yield an AP of 26.90 and AUC of 67.60 The corresponding results of [1], are AP of 28.35 and AUC of 76.22 for the seen classes and AP of 25.31 and AUC of 72.10 for the unseen ones. Thus, the experimental results suggest that our framework is able to provide similar performance without the use of concept supervision.\n\n> [1] Marcos et al., Attribute Prediction as Multiple Instance Learning, TMLR, 2022\n\n- **Weakness 3**: *Test-time Interventions.*\n\n> We thank the reviewer for their suggestion. Indeed, examining the concept intervention capabilities of our framework is already part of a future work. Nevertheless, following the reviewer’s suggestion, we performed a preliminary analysis on potential interventions. In our framework, the most appropriate type of intervention corresponds to inaccurate concept activations (similar to “Type 3: Incorrect concept activations” of Label-Free CBMs). In this setting, and for each example/patch, we can readily examine and alter the concept activation patterns. To this end, we consider the example provided in Fig. 3 in the main text (and Fig. 1 in the included PDF). For this example, we intervened on the discovered concepts by randomly turning on 10 concepts that contain the words white and feathers, while randomly switching off 10 concepts that contain the word black. We performed this process ten times and observed the changes in the classification decision. Most times, i.e, 7 out of 10, we yielded classes such as quill, african grey, oystercatcher, black stork and penguin and the rest, irrelevant classes such as picket fence, can opener and geyser. We also explored a “by-hand” approach, where we switched off some concepts that were not relevant to the example in consideration, e.g., very strong structure and color depends on the type of spider, and observed no change in the classification decision. These results serve as an initial investigation in the efficiency of intervention in our framework.  We’ll add more qualitative examples in the camera-ready.\n\n- **Question 1**: *how prone is the model to detect some concept accurately but in an uninterpretable way?* \n\n> We thank the reviewer for raising this point. Indeed, one of the main motivations of this work was to avoid this kind of misdetection (L53-61) of concepts based on other information present in the image. In our work, we have access to all the concepts discovered both on the image level, but also on the patch-level. This means that we can access and visualize the discovered concepts on these two levels and examine their validity. We provide such a visualization in the included PDF. We have yet to explore other qualitative tools such as activation maximization. This is, however, a very important avenue for our future research which includes the exploration of such methods, along with the consideration of even more isolated low-level information stemming from part detection algorithms or superpixel-based approaches.\n\n- **Question 2**: *Usage of non-visual concepts.*\n\n> This is a very interesting point that the reviewer is making. This is a fair question and highlights a limitation in the existing concept datasets and particularly automatically created ones (via LLMs or other methods). Those concepts are not always strictly visual, although they may be correlated to visual concepts, making them useful for image-based classification. In the future, we plan to invest some effort in the creation of datasets with exclusively visual low-level concepts.\n\n- **Question 3**: *Patch Resizing*\n\n> The patch level CLIP embeddings are extracted by resizing first and passing through CLIP to match the standard CLIP input size. To this end, we first resize the patch to 224. We’ll include this information in the camera ready.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for taking the time to review our manuscript and for their insightful comments. 

We carefully considered all the comments that the reviewers raised and addressed them diligently. To this end, we respond to each question individually and we also include a PDF with some novel investigations, both qualitative and quantitative, based on the feedback and comments of the reviewers.

Meta-review: Initially, this paper received mixed ratings: 4, 7, 5 and 5. After post-rebuttal discussion, the ratings were changed to 5, 6, 6, and 6: three upgraded and one downgraded but still positive. Three reviewers upgrading the ratings were satisified with rebuttal. The reviewer downgrading the rating did not provide detailed comments to the rebuttal, but still kept positive. The AC read the comments and the paper carefully.
Decision: Accept (poster)