Cross-Modal Conceptualization in Bottleneck Models
Authors: Danis Alukaev
Semen Kiselev
Ilya Pershin
Bulat Ibragimov
Vladimir V. Ivanov
Alexey Kornaev
Ivan Titov
Venue: EMNLP 2023 Main (2023)
Keywords: interpretability
cross-modal learning
concept-based models
cross-attention mechanism
robustness

Abstract: Concept Bottleneck Models (CBMs) assume that training examples (e.g., x-ray images) are annotated with high-level concepts (e.g., types of abnormalities), and perform classification by first predicting the concepts, followed by predicting the label relying on these concepts. However, the primary challenge in employing CBMs lies in the requirement of defining concepts predictive of the label and annotating training examples with these concepts. In our approach, we adopt a more moderate assumption and instead use text descriptions (e.g., radiology reports), accompanying the images, to guide the induction of concepts.  Our crossmodal approach treats concepts as discrete latent variables and promotes concepts that (1) are predictive of the label, and (2)  can be predicted reliably from both the image and text. Through experiments conducted on datasets ranging from synthetic datasets (e.g., synthetic images with generated descriptions) to realistic medical imaging datasets, we demonstrate that crossmodal learning encourages the induction of interpretable concepts while also facilitating disentanglement.

Cross-Modal Conceptualization in Bottleneck Models
Danis Alukaev'i Semen Kiselev', Ilya Pershin',
Bulat Ibragimov 26 Vladimir Ivanov', Alexey Kornaev', Ivan Titov Reseerrh Center for AI, Innopolis University 2DCS, University of Copenhagen $ILCC, University of Edinburgh dd wlukaeeeinnooollss.uu ietitov@inf.eem wd. uc.uk
ty $bulat@di ku. dk
Abstract
Concept Bottleneck Models (CBMs) (Koh al., 2020) assume that training examples (e.g.. x-ray images) are annotated with high-level concepts (e.g., types of abnormalities), and per- form classification by first predicting the con- cepts, followed by predicting these concepts. The main difficulty in using CBMs comes from having to choose concepts that are predictive of the label and then having label training examples with these concepts. In our approach, we adopt more moderate assumption and instead use text descriptions (e.g.. radiology reports), accompanying the im- ages training, to guide the induction con- cepts. Our cross-modal approach treats con- cepts discrete latent variables and promotes concepts that (1) are predictive the label, and (2) can be predicted reliably from both the im- age and text. Through experiments conducted on datasets ranging from synthetic datasets (c.g.. synthetic images with generated descrip- tions) to realistic medical imaging datasets, we demonstrate that cross-modal learning encour- ages the induction of interpretable concepts while also facilitating disentanglement. Our results also suggest that this guidance leads to increased robustness by suppressing the re- liance on shortcut features.
this bird has pointed wings, yollow forehead, all- purpose beak
label relying
conceot extractor
extractor
agreemeett
Gumbee Sigmoid
Gumbel Sigmo
P predictor
Ps predicter
yellow-heaced blackbird
the XCB framework, during training we promote agreement between the text and visual mod- els' discrete latent representations. Moreover, we intro- duce sparsity regularizers the text model to encourage disentangled and human- uuman-interpretable latent representa- tions. At inference time, only the visual model is used.
Figure 1:
objectives, for example, aiming to generate human- like plausible rationales or extract faithful explana- tion from the model (Jacovi and Goldberg, 2020; De Young et al., 2020).
1 Introduction
The limited interpretability of modern deep learn- ing poses a significant barrier, hindering their prac- tical application in many scenarios. In addition to enhancing user trust, interpretations can assist in identifying data and model limitations, as well facilitating comprehension of the causes behind model errors. Explainable ML has become a ma- jor field, with various methods developed over the year to offer different types of explanations, such input attribution (Sundararajan al., 2017; Si- monyan al., 2014), 4), free-text rationales (Camburu et al., 2018) or training data attribution (Koh and Liang, 2017). Moreover, methods pursue diverse
this work, our specific focus is on a line of research that aims to develop neural models ca- pable of making predictions via an (semi-) inter- pretable intermediate layer (Andreas et al., 2016; Bastings et al., 2019; Rocktäschel and Riedel, 2016; Koh et al., 2020). This class of models is appeal- ing because the interpretable component creates "bottleneck" in the computation, ensuring that all the information regarding the input conveyed through discrete variables. As result, these vari- ables play a causal mediation role in the prediction

---

F extracter
C
F
concept
a5 agreement
Sigmoid
Gumbel Sigmoid
Gumbee Sigmaid
Pr predictor
Pc predictor
Pr predictor
Df
Dc
Of
Figure 2: Standard computer vision model w/o interpretability of latent representation f (left), XCB with imposed agreement in visual latent representation f and textual latent representation (right). Notations in the image: visual modality, / - latent representation for æ, 0y prediction visual component, 8 - textual modality, c- latent representation for s, and De prediction textual component.
process (Vig et al., 2020). One notable represen- tative of this model class is Concept Bottleneck Models (Koh et al., 2020). CBMs, for a given input image, first predict which high-level con- cepts are activated, and then infer the class label based on these concepts. This simple and elegant paradigm is advantageous from practical stand- point as the intermediate layer can be examined, validated and intervened on by an expert user. For instance, in medical diagnosties, the concepts could represent types of abnormalities detected from an X-ray, which makes the model predictions easier to serutinize and verify for a specialist.
are linked to texts (such as radiology reports) which serve as guidance for concept induction. Note that the text is not used at test time in our experiments. Specifically, in training, we use an additional CBM model trained to predict the same target la- bel, but this time using textual data. A regularizer is used to encourage prediction of the same set of concepts in both visual and text CBMs on each training example. In this way, XCB ensures that the concepts can be predicted from both text and images, discouraging the image classifier from re- lying on features that cannot expressed through language and, thus, injecting domain knowledge from text into the visual model. Furthermore, an additional regularizer promotes the alignment of concepts with distinct text fragments, thereby en- hancing their disentanglement and interpretability.
Despite their interpretability advantages, CBMs have several critical limitations. Firstly, they re- quire manual creation and annotation of set of high-level concepts for each training example. which be challenging or unattainable in real-world scenarios. Secondly, concepts designed by domain specialists might not possess sufficient predictive power for classification or could be dif- ficult to reliably detect from the examples (e.g., images).
We conducted experiments involving XCBs on various datasets, including synthetic text-image pairs as well as a more realistic medical image dataset. Our experiments demonstrate that XCB tends to surpass the alternative in terms of disen- tanglement and alignment with ground-truth (or (umaan-annotate) concepts. Additionally, we in- corporated synthetic shorteuts into the images and observed that the inclusion of textual guidance re- duced the model's tendency to overly rely on these shortcuts and, thus, made the model more robust.
This work aims to address these shortcomings of CBMs. It considers concepts discrete binary latent variables, eliminating the need for an ex- pert to predefine the concepts. Rather than relying on the annotation of examples with concepts our approach Cross-Modal Conceptualization in Bot- tleneck Models (XCBs)' makes weaker and arguably more realistic assumption that examples
Related Work
Concept Bottleneck Models (CBMs) (Koh and Liang, 2017) can regarded as a representative
"Released at https:/lgthhh..oo///aaiisaaaaka

---

of a broader category of self-explainable archi- tectures (Alvarez Melis and Jaakkola, 2018). In essence, CBMs incorporate a bottleneck composed of manually-defined concepts, which provides a way of understanding 'reasons' behind the model predictions. Examples must be annotated with cor- responding concepts, thus, facilitating the training of two distinct segments of the model: the input-to- concepts and concepts-to-label parts.
tegrate additional human expertise in such models. We consider LCBM as an alternative to XCB in our experiments.
Yuksekgonul et al. (2023) introduced a method that posthoc transforms any neural classifier into a Concept-Based Model (CBM). In contrast to their approach, we leverage text to shape the representa- tion inside the neural classifier, potentially making them more disentangled.
In addition to efforts focused on crafting meth- ods for training CBMs, there is an intriguing branch of research dedicated to exploring the utilization of concepts within CBMs to intervene and modify the models (Shin et al., 2022: Steinmann et al., 2023). While this strand of research appears somewhat divergent from our primary interest, underscores another significant application of CBMs - extend- ing beyond mere prediction interpretation.
While argued that the bottleneck should be discrete to limit the amount of information it can convey, CBMs do not strictly adhere this prin- ciple. In CBMs, even though the concepts them- selves are discrete, their representation within the bottleneck layer is not. Specifically, each concept is represented as a logit from a classifier predicting that concept. This deviation from complete dis- creteness is non-problematic when the two parts of CBMs (input-to-concepts and concepts-to-bl -o--aalll are trained either independently or sequentially. Under such conditions, the logit would not be in- centivized to retain input details unrelated to the concepts. Yet, the problem arises when these model parts are trained simultaneously (Margeloiu et al.. 2021). Mahinpei et al.. 2021 have shown that this information leakage can be alleviated by adopting strictly discrete variables and promoting disentan- glement. These insights drove our modeling deci- sions, prompting us to employ discrete variables and use text (as well as specific forms of regular- ization) to encourage disentanglement.
Proposed Framework
In this section we introduce a novel method for cross-modal conceptualization in bottleneck mod- els (XCBs). We start with a high-level overview of the framework and introduce its core components. Further, we consider models used to process visual and textual modalitics. Finally, we describe tech- niques that were used to encourage cross-modal agreement and disentanglement between the con- cepts.
3.1 Overview
A recent approach titled label-free CBM (LCBM) (Oikarinen et al., 2023) focuses on ad- dressing some of the same challenges we address with XCBs, specifically CBM's dependence on pre- defined concepts and the necessity for manual an- notation of examples with such concepts. This study prompts GPT-3 (Radford et al., 2018) pro- duce a of concepts appropriate for the domain. Following this, the CLIP model (Radford et al., 2021) is applied label the images with concepts. In a parallel development, Yang et al. (2023) intro- duced a similar framework named Language model guided concept Bottleneck model (LaBo), which also incorporates GPT-3 and CLIP as key ingre- dients. One distinction in their approach is the employment of submodular optimization gen- erate set of diverse and discriminative concepts. Although these methods show promise on large vi- sual benchmarks, is unclear how effectively they would work in specialized domains and how to in-
The design of the framework (Fig. encourages each element of the discrete latent representation produced by visual encoder to correspond to a sin- gle high-level abstraction, which can be expressed by a text span. Accordingly, we use an additional textual model that identifies concepts expressed in natural language and determines the correspon- dences to visual features.
XCB can tentatively be divided into symmet- ric visual and textual components (Fig. 2. right). The former consists of computer vision model F x -> ʃ (feature extractor) producing latent representation f for an input image and predic- tor P3 f Us solving the classification task. Similarly, the textual component includes natural language processing model C - (concept extractor) producing latent representation e for a sequence of tokens and predictor P. c ->> -c- The discreteness of latent variables is imposed by a straight-through version of the Gumbel sigmoid

---

such model with latent sigmoid bottleneck as a baseline and do not impose any pressure make the representation interpretable or discrete.
Input Embedding
Positional Encoding
3.3 Textual Model
The concept extractor is an integral part of XCBs used to process textual descriptions and derive set of concepts. It has been observed that many heads in multi-head attention specialize (Voita et al., 2019), i.e. become receptive to narrow and unique set of words. Accordingly, tention mechanism will reinforce disentanglement in the framework and, thus, use in the concept extractor C. We will discuss below how we can further encourage disentanglement by modifying the attention computation.
WR
Sequence Embeddingss
K
keys
hope that the at-
normaliatioo
W
Fig. 3 demonstrates the concept extractor adapt- ing cross-attention mechanism for semantic token clustering. Token embeddings are augmented with a sinusoidal positional encoding that provides the notion of ordering to model (Vaswani al., 2017). The weight matrices W and WV are used to transform input embeddings into matrices of keys K and values V respectively. We compose train- able query matrix Q aimed to capture semantics specific correspondent factor in latent represen- tation c. Averaging matrix of values V with respect to cross-attention scores yields the contextualized embeddings 7;, which are further used to estimate elements of c through correspondent predictors Pr.-
P
Figure Concept extractor based on cross- atssnttttntionn mechanism. Each query captures semantic of correspon- dent latent factor in c. Tokens with highest average cross-aftention score regarded concepts.
Since we established one-to-one correspondence between set of queries and latent factors, the matrix of cross-attention scores could be considered relevance of each input token variables in repre- sentation Concept extractor aggregates average cross- cross-attention scores for each token vocabulary. The concept candidates are, thus, selected by an average attention score for a correspondent latent factor.
(Jang et al., 2017) applied to bottlenecks. To reinforce agreement in feature and concept extractors at training time latent representations and are tied up using similarity metric. The regularizer encourages model to generate set of latent factors that could reliably be predicted by both visual and textual encoders.
The design of concept extractor also accounts for two special cases, First, some token in the input sequence s might not align with any query (i.e. not correspond to any concept). Second, the opposite. some query might not align with any tokens the input sequence s. We thus introduce additional "dummy" query to capture cross-attention scores from irrelevant tokens, and "dummy" tokens (one
3.2 Unsupervised Approach
The standard computer vision model for classifi- cation (Fig. 2, left) is generally composed of the feature extractor F and predictor Py. The latent representation is output of non non-linear ac- tivation function (e.g., sigmoid). Each factor of f could, in principle, correspond single high- level abstraction. Thus, latent representation is not a concept bottleneck per se, but arguably could emulate such behavior, In our experiments we use
II our preliminary experiments, an unsupervised model with a discrete (Gumbel throug)) layer was not par- ticularly effective, trailing the continuous version according to disentanglement metries. Thus, we opted to utilize the stronger continuous version as a baseline in our experiments.

---

per each query) for the model to indicate that all tokens from the input sequence are irrelevant.
Parameter
Shapes
CUR-200
MIMIC
No. examples
2,700
11.788
15,000
No. attributes
14
No. labels
3.4 Cross-Modal Agreement
Vocahulary size
8,983
8,651
No. tokens, j
11.2
181.2
63..0
To ensure alignment between the visual and tex- tual components XCB, we propose the inclusion of an additional tying loss function that assesses the similarity between the latent representations, denoted as f and c. Since these representations are discrete and stochastic, we employ the Jensen- Shannon divergence to tie the distributions of the visual and textual representations. Given that each representation's distribution be factored into Bernoulli distributions for individual features, the calculation of the divergence is straightforward.
No. tokens, SD
0.8
26.4
24.8
No. of tokens, max
373
378
'able 1:
Description of datasets used in research pro- cess. Captions for Shapes and CUB-200 are rule-based generated, whereas in MIMIC captions are structured reports written by radiologists in free form.
Note that both the promotion of discreteness and disentanglement have been recognized to decrease the inclusion of irrelevant information into con- cepts, known concept leakage (Mahinpei et al., 2021). This offers an additional rationale for their application.
Because of imposed similarity between represen- tations f and с the set of concepts retrieved by tex- tual component tends to also correspond to visual latent factors, and thereby the visual part. Otherwise, on unseen data, XCB is virtually identical to CBMs: for given sample a it infers a binary set of human-interpretable at- tributes f. which further used to predict the target variable 0y. Note that training with cross-modal agreement yields an additional computational over- head of 15-25% varying by dataset (App. A).
be used explain
Empirical Evaluation
This section describes the datasets, introduces eval- uation metries for interpretability, and describes the procedure used to measure robustness of models.
4.1 Datasets
For our experiments we have used three classifica- tion datasets (Tab. 1). Each data point takes a form of (Ir, 8, 3,))), where x stands for a visual modality (e.g., chest x-ray image), 8 textual modality (e.g., structured medical report), - binary set of attributes (e.g., types of abnormalities), and label of class (e.g., whether pathology has been detected).
3.5 Encouraging Disentanglement
To intensify disentanglement between concepts we borrow ideas from slot attention (Locatello et al., 2020). Instead of normalizing wross-attention scores exclusively along the dimension of tokens, we firstly normalize them along the concepts, and only then rescale scores along the tokens. This method forces concepts (i.e. queries) to compete for tokens, and thus discourages the same token from being associated with multiple concepts. This effect is further amplified by the usage of entmax (Correia et al., 2019) activation function that pro- duces sparse distributions of concepts (for each token).
Shapes. Primarily for benchmarking we propose a synthetic dataset of primitive shapes (Fig. 4, top).² The image æ is white canvas with random shape (i.e., square, triangle, circle) of random color (i.e., red, green, blue), and random size (i.e.. small, medium, large). Caption c is generated from visual parameters of a shape, its location, and words sampled from the vocabulary, using hand-written grammar. The vector of attributes encodes shape and color, their combination represents a target class y.
We also introduce an additional training objec- tive measuring an average pairwise cosine similar- ity of contextualized embeddings (App. B). This regularization further encourages concepts in the textual component to focus semantically differ- ent parts of an input sequence and, thereby, capture separate factors of variation. Since this procedure is relatively computationally demanding, on each training step the loss computed based on a subset sampled from Tü-
CUB-200. Similar to CBMs evaluate XCB Caltech-UCSD Birds-200-2011 (CUB-200) (Wah et al., 2011). Due incoherent intra-class attribute labeling we performed a majority
Relcased at http:://ghhaah.cmm//aaaaattlllt

---

alignment between the latent representation f and vector of ground-truth attributes a.
triargulan Tax cundratie
Elmosler
exteson turn no bocner tmage bothon right
fore convas transpazent
gower
The relationship between latent representation and underlying factors of variation can be quanti- fied in terms of disentanglement,, completeness, and informativeness (DCT) (Eastwood and Williams, 2018). Disentanglement indicates degree to which a latent factor captures at most one attribute, com- pleteness reflects degree to which each attribute is captured by a single latent factor, and informative- ness accounts for amount of information captured by latent representation. For each attribute k, East- wood and Williams (2018) suggest training linear regressor RĄ + 0g with associated feature importance scores R Disentanglement and com- pleteness are entropic measures derived from R whereas informativeness is a normalized root mean squared error of R₂ computed on the testing subset.
Aluw
Alb-purpese beskc counded nangl
this defindrely
coggez
stll-urppte
peosubly has wings...
Black
cardionegaly
Edacion
lydecducency wnideee tenithorax without cleat sorponlogical storibute
Texton
stalectania
edema -6
Iracture -0
moderate suctlion syjaly.........................
Figure 4: Examples of data points (x visual modal- ity, set of attributes, textual modality) from datasets used in research process: Shapes with label "red circle" (top), CUB-200 with label "yellow headed blackbird" (middle), MIMIC-CXR with label "pathol- ogy" (bottom).
One downside of the DCI metric is the require- ment of annotated attributes, which are typically available only for benchmark datasets. Conse- quently, in real-world scenario, quality of in- terpretations could be assessed based on human evaluation. Namely, we provide human experts with a set of images and pieces of text represent- ing underlying concept in the framework. One of the text pieces is sampled from a different con- cept. The goal of domain expert is to determine which option is irrelevant. For instance, interpreta- tions of XCB on MIMIC-CXR were evaluated by a practicing radiologist through a set of ten ques- tions. A sample question consists of five X-ray images, with which our framework associated to- kens "pneumothoraxx, "effusions", and "atelecta- sis" (corresponding scaled confidence scores th; were left unknown to the expert). As an additional option, we add "pneumonia" and set its confidence score to 3/2 - 0. For a selected option j we further define a metric XScore = Φ5j- Higher values of XScore indicate that model interpretation with the least confidence score is more relevant to provided studies than the random one.
voting as suggested by Koh et al. (2020), Since CUB-200 does not contain captions, description was generated as provided by the vector of attributes (Fig. 4, middle): each attribute a, with heading in form of "has_
::<t>" is represented as "this bird has <t> 
", c.g., saa__eaaaaaaaea::_aaeeaaaaa dagger" becomes "this bird has dagger beak shape". The caption for sample is thereby accumulated from all descriptions of ac- tive attributes and tokens sampled from vocabulary.
MIMIC. To assess XCB in realistic environment we use subset from MIMIC-CXR (Johnson et al.. 2019). The structured reports are written by ra- diologists in a free form and without the use of templates. The set of attributes (Fig. 4, bottom) is comprised of characteristics used by radiologists to differentiate pathological cases, which thereby might be considered as underlying latent factors of a certain generative process and used to evaluate latent representations.
4.2 Measuring Interpretability
4.3 Exploring Robustness
Quantitative evaluation of model interpretability is not entirely standardized. The formulation of a metric highly depends on the specifics of the model architecture and available annotations. Our frame- work belongs to of concept-based models, where structural components are guided by a set of symbolic attributes. In the context of XCBs, we thus define the level of interpretability through
Models often learn to rely on non-robust (i.e. spuri- ous) features, which are easy to identify in training but which will not be present or will be misleading at usage time (Ilyas et al., 2019). For example, in the medical domain training data typically col- lected from both regular and emergency hospitals. Since most such machines put their serial number

---

Modification
Fl-score
A Disent.
A Compl. 1
A Inform.
Sigmoid - Gumbel Sigmoid
-0.04 0.00
0.14 0.02
0.06
0.01 0.0100000
Softmax - Entmax
-..16
-0.04
0.05
0.01 0.00
Regular norm. - Slot attention norm.
-.04
-0.07
-0.03
0.01 0.00
w/o "dammy" query & tokens + w/
-0.03
0.03
0,08 = 0.02
40,02 0.00
Reg. via pairwise similaritics of
0.01
-0.01 + 0.00
0.01 = 0.00
0.00
Dus( (us(f"llc) + (F'lc))
-0.31 + 0.05
-0.11 + 0.02
-0.03 2: 0.00
-0.02 = 0.00
Dis(f"ll) (xz(c(1f")
-0.27 # 0.04
-0,09 # 0.02
-0.02 + 0.00
-0.02 0.00
Table 2: Ablation study of XCB Shapes Dataset.
in the corner of the image and emergency hospi- tals tend to handle harder cases, this ID number becomes a distinctive yet spurious feature (Ribeiro et al., 2016). Reliance of such features is clearly highly undesirable.
posed of feature extractor F followed by sigmoid function and predictor Py. The size of latent rep- resentations was set to 10, 320, and 20 for Shapes, CUB-200, and MIMIC-CXR datasets respectively (Tab. 1). Textual embeddings were randomly ini- tialized as trainable parameters of length 50, The CBM size of bottleneck matched the number of attributes in a dataset (Tab. 1). To run experiments with CBM and LCBM the official implementa- tions were used. Weights of all models were initial- ized using Xavier uniform distribution.
To explore the robustness of the XCB model we incorporate non-robust feature in training subsets. As a shortcut, we use numerical class label located in the upper left corner of the image. The performance and interpretability of standard and proposed models are further evaluated on samples without distractors. The higher the value of the metric is, the higher robustness is expected.
The feature extractor F and predictor Pf were optimized jointly using AdaDelta optimizer (Zeiler, 2012) with learning rate 0.25 and rho 0.95. The concept extractor C' and predictor Pa were optimized jointly via Adam optimizer (Loshchilov and Hutter, 2019) with learning rate 0.001, betas of 0.9 and 0.999, and weight decay 0.01. The learning rates in AdamW and AdaDelta were adjusted using OneCycleL.R scheduler (Smith and Topin, 2019) with maximal learning rates set 0.001 and 0.25, respectively. Temperature of the Gumbel sigmoid followed the exponential schedule (Jang et al., 2017). The experiments were conducted using early stopping policy on validation loss with patience of 5 epochs. Five random seeds were used: 42, 0, 17,9, 9, and 3. Mean and standard deviation for the obtained results were computed over five training runs.
5 Results and Discussion
In this work, compare the proposed models with the standard 'black-box' model, as well as CBMs and LCBMs (Oikarinen et al., 2023). Due to similarity of LCBM and LaBo (Yang et al., 2023) models, we include in comparison only the former (see Section 2 for their discussion). The experi- ments were conducted using synthetic and public datasets highlighted in Section 3.1 of the paper. The obtained results are discussed with respect to their performance, interpretability, and robustness.
The ratios between training, validation, and test splits are 75%, 15% and 10%, respectively. For all datasets image size was set to 299 with a batch size of 64. At each epoch, images were normalized and the batches randomly shuffled.. The CUB-200 follows data processing pipeline described in Cui al. (2018) with additional augmentations such horizontal flip, random cropping, and color jit- tering. The hyperparameters were optimized using Tree-structured Parzen Estimator sampling algo- rithm (Bergstra et al., 201
Ablation study. Components introduced in XCB that aim to improve interpretability and disentanglement of representation cause drop in performance (Tab. 2). Nevertheless, dis- entanglement and completeness metrics were substantially improved by discretization of latent representations via Gumbel sigmoid and using "dummy" tokens in slot normalisation respectively, Remarkably, using Kullback-
To concentrate expressivity of concept ex- tractor C, we restricted our feature extractor F to be fine-tuned Inception model (Szegedy al.. 2016) pre-trained on the ImageNet dataset (Deng et al., 2009), with multi-layered perceptron of depth of one for Py and P.. The baseline model was com-
divergence
"Available online at https//githuh..cm/yywsaia ConeeplBouleneck
Avaallabl online at http:///ghhuhhcom/////wwwrtttt Lab/l abel-free-CBM

---

Dataset
Model
F11
Disent. 1
Compl.
Inform.
Standard
1.00 000
.60 005
.64 141.10
.07 .01
CBM
1.00
.50
.47 4.7
.10
Shapes
LCBM
.55
.99
.08 :01
XCB
.6 01
.78 .03
274
.07 =
Standard
80
.18 02
172.002
81
CBM
.78
.16 .0
.16
.82 :33
CUB-200
LCBM
.74
.19 = .02
.15
84 .02
XCB
.15
212
.2
.78
Standard
.07
103
00 .00
1.00 +000
CBM
275
.02
.01 .00
1,00 = 000
MIMIC
LCBM
-3
.01 =
.01 + .00
1.00 + .00
XCB
.4 .00
.04
.03 .00
1,00 = 00
Table 3: Evaluation results on Shapes, CUB-200 and MIMIC-CXR datasets. XCB model outperforms standard
model, CBM, LCBM in terms of interpretabilltty (DCI metrics) maintaining comparable level of performance.
as a tie loss worsens both performance and DCI, which could indicate that neither modality fully guides induction of implicit high-level abstractions.
a better quality of interpretations terms of XS- core compared to LCBM: 0.72 vs. 0.66.
The framework associates each factor in latent representation with set of textual interpretations (Fig. 5). For example, the images corresponding to the fourth neuron activations cluster in terms shape, i.c., larger value of logit correspond to squares, whereas smaller represent triangles. Explanations of our model are indeed referring to the shape of figure: highest relevance is achieved by tokens "triangular" and "quadratic".
The standard model outperforms CBMs on all datasets (Tab. 3), which can be attributed to a trade-off between performance and level of interpretability. Another possible reason is that manually derived set of attributes might not be predictive enough for classification. On the medical dataset LCBM is inferior to other models indicating current limitations in the domain knowledge of large language models. XCB achieves comparable to other variations CBMs performance.
Performance.
Robustness. Following the procedure presented in Section 3.3 of the paper, the performance and interpretability on non-robust Shapes Dataset were measured. Performance of the standard model on testing subset is lower than for XCB on all datasets (Tab. 4). The DCI of the proposed framework are also superior compared to the standard model. Method of integrated gradients reveals that the base- line mainly focuses on non-robust feature, whereas the proposed framework has higher attribution to the shape of the object (Fig. 6). We speculate that such an increase in robustness can be associated with textual component of the framework, which regularizes implicit visual abstractions.
Interpretability. The highest DCI scores (see Sec- tion 3.2 of the paper) were achieved with the pro- posed XCB model that could be attributed to do- main knowledge transferred to the model (Tab. 3). High values of both disentanglement and complete- ness indicate that alignment of latent representation and set of attributes tends towards bijective. Al- though absolute values of DCI medical dataset are modest, out survey indicates that XCB possess
Note that while we encourage textual and vi- sual components to align, it is not a hard constraint but rather a soft regularizer, which is used only in
ordangutar:: 1515
LKB
SPIL
AHI
270
LHD
AND
THE
ADS
Metric
Shapes
CUB-200
MIMIC
cuadracto::
Fl f Disent, f
.04 = .00
.01 # .00
.02 + .00
.02 + .00
.02 # .00
.01 # .00
four: 3.045
6. engle: 0.0474
Compl.
.02 + .0
.02 : .00
.02 + .00
Inform.
.03 + .00
.04 + .0
.01 + .0
Figure 5: Interpretations for the fourth latent factor on Shapes dataset. Concept candidates for the latent factor (left) describe variance of observed values of logits on bottleneck (right).
Table 4: Difference F1-score and DCI of XCB model
compared to standard model on test subset w/o non- robust feature.

---

Metric
Shapes
Noisy (10
F1
96
.94
Disent.
=
Compl. f .74
.76
Inform. 4 .07 .02
.02
Table 5: Performance of XCB on Shapes Shapes with 10% of text descriptions replaced by random ones.
Figure 6: Sample with non-robust feature (left), inte- grated gradients for standard model (middle), integrated gradients for XCB model (right). XCB shows higher attribution the shape of an object (robust feature).
on information leakage (sce section 4 in Mahinpei et al. (2021)).
Domain dependence. Intepretation of neurons is known to not generalize across domains (Sajjad et al., 2022): the same neuron may encode different phenomena in two different domains. It is likely be the case for latent concepts. Our studies were restricted to fairly narrow domains.
training. Still, systematic errors (e.g., vital con- cepts consistently omitted in text) could be detri- mental. However, moderate amount of random noise might be less problematic. We tested this hypothesis on the Shapes dataset by taking 10% of the training examples and replacing their text descriptions with random ones. The performance did not change significantly (Tab. 5) confirming our hypothesis.
User studies. The most reliable way to guarantee the interpretability and usefulness of representa- tions is through user studies. However, conducting these studies for evaluating interpretability costly and challenging design. In this work, we primarily focused on automatic metrics instead. Nevertheless, it is important note that these met- rics are not flawless and constrained us to datasets where ground-truth semantic concepts are avail- able.
Conclusion
be
The objective of this study is to enhance the prac- ticality of Concept Bottleneck Models by elimi- nating the need for predefined concepts provided by experts and annotating each training example. Instead, our approach utilizes textual descriptions to guide the induction of interpretable concepts. Through experiments, we demonstrate the advan- tages of using textual guidance compared in terms of interpretability in range of set-ups. We be- lieve there are numerous ways to expand on this work, particularly by incorporating large lan- guage model into the textual component. We con- tend that the broader exploration of leveraging guid- ance from text or language models to shape abstrac- tions in models for various domains remains largely unexplored. This direction holds great promise for enhancing the transparency and robustness of models utilized in decision-making applications. Our study serves an illustration of one potential method for providing such guidance.
Ethics Statement
Like any other machine learning models, the po- tential for misuse of XCB technology should not be overlooked. Users of XCB, along with other interpretability technologies, need to be aware that lacks guarantees of faithfulness. Moreover, being new model architecture, it runs the risk of intro- ducing harmful biases that are not present in more established architectures. Consequently, its appli- cation in practical settings necessitates thorough user studies and scrutiny.
Although our experiments employ a public med- ical dataset, and we hope that this line of research will ultimately yield systems capable of effectively supporting decision-making in healthcare, it is cru- cial to emphasize that, at the current stage of de- velopment, such unproven technology cannot be employed in any critical application.
Limitations
Information leakage. A concept has the potential reveal additional information about the input, which may not be immediately evident upon inspecting the concept itself. To address this concern, we employ discrete representations and promote disentanglement between concepts, drawing on conclusions from a prior investigation
Acknowledgements
Danis Alukacv, Semen Kiselev, Ilya Pershin, and Alexey Kornev were financially supported by The

---

Analytical Center for the Government of the Rus- sian Federation (Agreement No. 70-2021-00143 dd. 01. 1021, IGK 000000D73052 IPSS000)))))))
Cian Eastwood and Christopher KI Williams. 2018. A framework for the quantitative evaluation of disen- tangled representations. In International Conference on Learning Representations.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras. Lo- gan Engstrom, Brandon Tran, and Aleksander Madry. 2019. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32.
References
David Alvarez Melis and Tommi Jaakkola. 2018. To- wards robust interpretability with self-explaining nou- ral networks. Advances in neural information pro- cessing systems, 31.
Alon Jacovi and Yoav Goldberg. 2020. Towards faith- fully interpretable NI.P systems: How should we define and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4198-4205, On- line. Association for Computational Linguistics.
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein, 2016. Neural module networks. In Pro- ceedings of the IEEÉ conference on computer vision and pattern recognition, pages 39-48.
Jasmijn Bastings, Wilker Aziz, and Ivan Titov. 2019. Interpretable neural predictions with differentiable binary variables. In Proceedings of the 57th Annual Meeting of the Association for Computational Lin- guistics, pages 2963-2977, Florence, Italy, Associa- tion for Computational Linguisties.
Eric Jang, Shixiang Gu, and Ben Poole. 2017. Cate- gorical reparameterizatiion with gumbel-softmax, International Conference on Learning Representa- tions.
Alistair EW Johnson, Tom J Pollard, Seth Berkowitz. Nathaniel R Greenbaum, Matthew P Lungren, Chih- ying Deng, Roger G Mark, and Steven Horng. 2019. Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):317.
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for syper-parameter optimization. Advances in neural information pro- cessing systems, 24.
Pang Wei Koh and Percy Liang, 2017. Understanding black-box predictions via influence functions. In International conference machine learning, pages 1885-1894, PMLR.
Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natu- ral language inference with natural language expla- nations. Advances in Neural Information Processing Systems, 31.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. 2020. Concept bottleneck models. In Inter- national Conference on Machine Learning, pages 5338-5348 PMLR.
Gonçalo M. Correia, Vlad Niculac, and André F. T. Martins. 2019. Adaptively sparse transformers. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP)), pages 2174- 2184, Hong Kong, China. Association for Computa- tional Linguistics.
Francesco Locatello, Dirk Weissenborn, Thomas Un- terthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. 2020. Object-centric learning with slot atten- tion. Advances in Neural Information Processing Systems, 33:1
Yin Cui, Yang Song, Chen Sun, Andrew Howard, and Serge Belongie. 2018. Large scale fine-grained cate- gorization and comain-specifii transfer learning. In Proceedings of the IEEÉ conference on computer vision and pattern recognition, pages 4109-4118 18.
Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization, In International Confer- ence on Learning Representations.
Jia Deng. Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hier- archical image database. In 2009 IEEÉ conference on computer vision and pattern recognition, pages 248-255. leee.
Anita Mahinpei. Justin Clark, Isaac Lage. Finale Doshi- Velez, and Weiwei Pan. 2021. Promises and pitfalls of black-box concept learning models. proceeding at the International Conference Machine Learning: Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable Al, 1:1-13.
Jay De Young, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2020. ERASER: A benchmark to evaluate rationalized NI. models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4443-4458, Online. Association for Computational Linguistics.
Andrei Margeloiu. Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller. 2021. Do concept bottleneck models learn as in- tended? proceeding at the International Conference on Learning Representations: Workshop on Respon- sible AI, 1.

---

Tuomas Oikarinen, Subhro Das. Lam Nguyen, and Tsui-Wos Wei Weng, 2023. Label-free concept bottleneck models. In International Conference Learning Represenations.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances neural information processing systems, 30.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- ur. Amanda Askell, Pamela Mishkin, Jack Clark, al. 2021. Learning transferable visual models from natural language supervision. In International confer- ence on machine learning, pages 9748-8763. PMLR.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Simas Sakenis, Jason Huang, Yaron Singer, and Stuart Shieber. 2020. Causal mediation analysis for interpreting neural nlp: The case of gender bias. arXiv preprint arXiv:2004.12265.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language under- standing by generative pre-training
Elena Voita, David Talbot, Fedor Moiseev, Rico Sen- nrich, and Ivan Titov. 2019. Analyzing multi-head self-attention: Specialized heads do the heavy lift- ing, the rest can pruned. In Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics, pages 5797-5808, Florence, Italy. Association for Computational Linguisties.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. why should trust you?" explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135- 1144.
C. Wah, Branson, P. Welinder, Perona, and S. Be- longie. 201 Technical Report CNS-TR-201 1-001, California Institute of Technology.
Tim Rocktäschel and Sebastian Riedel. 2016. Learn- ing knowledge base inference with neural theorem provers. In Proceedings of the 5th workshop on auto- mated knowledge base construction, pages 45 40.
Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch and Mark Yatskar. 2023. Language in a bottle: Language model guided concept bottlenecks for interpretable image classifi- cation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19187-19197.
Hassan Sajjad, Nadir Durrani, and Fahim Dalvi. 2022. Neuron-level interpretation of deep nlp models: A survey. Transactions of the Association for Computa- tional Linguistics, 10: 10:1255-303.
Sungbin Shin, Yohan Jo. Sungsoo Ahn, and Namhoon Lee. 2022. A closer look at the intervention pro- cedure of concept bottleneck models. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurlPS 2022.
Mert Yuksekgonul, Maggie Wang, and James Zou. 2023. Post-hoc concept bottleneck models. ICIR.
Matthew D Zeiler. 2012. Adadelta: an adaptive learning rate method, arXiv preprint arXiv:1212.5700
Karen Simonyan, Andrea Vedaldi, and Andrew Zis- serman. 2014. Deep inside convolutional networks: Visualising image classification models and saliency maps. In 2nd International Conference on Learn- ing Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Workshop Track Proceedings.
Leslie N Smith and Nicholay Topin. 2019. Super- convergence: Very fast training of neural networks using large learning rates. In Artificial intelligncee and machine learning for mudri-domain operations applications, volume 1106, pages 369-386. SPIE.
David Steinmann, Wolfgang Stammer, Felix Friedrich, and Kristian Kersting. 2023. Learning in- tervene on concept bottlenecks. arXiv preprint arXiv:2308.11 13453.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Interna- tional conference on machine learning, pages 3319- 3328. PMLR.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe. Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 28 2826-.82.

---

A Computational Requirements
Dataset
Standard
XCB
Shapes
30.5 0.0
0.1
The primary source computational overhead in XCB (compared to standard models) tends to be the cross-modal agreement of latent representations on training time, which in turn is majorly influenced by the size of bottleneck. Keeping bottlenecks of reasonable size (approximately equal to the number of factors of underlying generative process) results 15-25% increase in training time (Tab. 6). Note that another possible source of computational com- plexity could be an average pairwise cosine simi- larity of contextualised embeddings Although, in practice was applied to random subset of 5% of training examples and did not affect the runtime substantially.
CUB-200
85.2
106.4
MIMIC
134.5 0.4
1.5
Table 6: Training time averaged over five training runs for Standard vs. XCB models seconds on Tesla- 16Gb.
pre-train visual component and with decrease its learning rate gradually add gradients from tex- tual component, (b) associate each block of the model with particular frequency of weights up- dating so that fuisual < fteztual.
C Examples
Shapes (Fig. 7, top): retrieved concepts include description of color ("crimson", "red") and shape ("three", "angles"), Note that concept "circular" corresponds to a negative logit in f, so we can speculate that model regards object in the image the opposite to "circular" one.
B Optimization
Training objective for XCB models is generally consists of four terms:
classification loss functions for visual and tex- tual components of XCB, e.g, for all our se- tups we used regular cross-entropy loss;
CUB-200 (Fig. 7, middle): retrieved concepts in-
clude description of plumage texture ("buff", "eyer- ing"), beak shape ("hooked", "seabird"),, and color of upperparts ("grey").
tying loss that ensures cross-modal alignment of two latent representations and f activated by sigmoid function, e.g., Jensen-Shannon divergence (1);
3
Dss(d'1l") m' = 1 (c')))
pred: "zed triangle"
CB "crinson"
10.3
"three"
log
+
C1 "circular'
-6.4
m
C2 "angles"
4.3
C7 'rad"
4.1
1
(1 c) log
(1 (1-
())
N
Σ J log
f ml
+
pred: blaack-footed al babtrssss
"butr"
C53
"hoaked"
4.8
- l ml)
[1
log
"seabird"
4.7
N 100
C109 "grey"
4.4
CA2
"eyering"
4.4
sparsity regularizer for contextualised embed- dings ri, e.g., pairwise cosine similarity (2).
pred: 'pathology"
C3 "pneunothorax"
3.0
lossvey (r)
n1?
1=0 I=
rrllry
(2)
C17
'volume'
"focal"
2.8
2.2
C5
'low"
1.5
C10 "eclema"
-1.4
In practice, visual modality often tends be more complicated than textual, which makes model favor updating visual component rather than tex- tual one. Empirically, we found that there are two straightforward ways to improve convergence: (a)
Figure Interpretations of logits in latent representa- tion ff on inference time for Shapes dataset (top), CUB- 200 (middle), and MIMIC-CO CXC (bottom) datasets.

---

MIMIC-CXR (Fig. bottom): retrieved concepts include name of observed pathologies (" ("pneumoth- orax", "edema"), and some indirect factors ("focal", "volume", "low") used in similar context.
D Textual Redundancy
One of the possible applications for XCB models could be medical imaging, where structured reports produced by radiologists often possess redundant data, e.g., in impression and findings sections. To examine how effectively the model could handle and process such data we simulated redundancy by concatenating paraphrased description to the descriptions in the Shapes dataset. We observed discernible difference in results (Tab. 7) and attribute it the fact that our sparsity regularizer ensures each text fragment corresponds a single concept, but does not penalize a concept being inferred from multiple text segments.
Metric
Original
Redundant
F
.95
Disent. f
.78 .03
.79
Compl.
.74
.80 .05
Inform.
_07 .20
.07 = .02
Table 7: Performance of XCB original Shapes vs. Shapes with redundant text descriptions.


Reviews:
[{"review_id": "GRJ82cyhuT", "reviewer": "Reviewer_UqZr", "summary": "", "questions": "1. How does the proposed model address ambiguities or inconsistencies present in the text?\n\n2. Can the authors shed light on the computational requirements for the execution of the proposed method?\n\n3. In scenarios where one modality is of inferior quality, how does the model ensure that it doesn't negatively affect the quality of representations?\n\n4. Given that textual descriptions, particularly in medical datasets, may possess redundant data, how does the model effectively handle and process such data?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the interesting questions and acknowledgement of our detailed experimentation and the clarity of our motivation.\n\n\n**Q1: How does the proposed model address ambiguities or inconsistencies present in the text?**\n\n**Regarding inconsistencies:** Note that while we encourage textual and visual components to align, it's not a hard constraint but rather a soft regularizer, used only in training. Still, systematic errors (e.g., vital concepts consistently omitted in text) could be detrimental (see also our reply to your Q3). However, a moderate amount of `random’ noise might be less problematic. We tested this hypothesis on the Shapes dataset by taking 10% of the training examples and replacing their text descriptions with random ones. The performance did not change that much (XCB on noisy data vs XCB on original data: $0.94$ vs $0.96$ in F1; $0.75$ vs $0.78$ in Disentanglement; $0.76$ vs $0.74$ in Completeness; $0.09$ vs $0.07$ in Informativeness). These are average across 5 runs and we kept the same hyperparameters as the ones used on clean data. We'll describe this experiment in the revised paper.  \n\n**On ambiguity:** Specific types, such as lexical ambiguity, are likely less problematic in specialized domains which motivated this work (e.g., medical records).  Nonetheless, other types (e.g., anaphoric ambiguity) could be challenging for our shallow non-contextualized text encoder. We posit that even if the textual component is wrong on some examples during training, the overall impact may be minimal as long as the majority of cases are straightforward for the text component (similar to random ‘noise’ above).  Still, we agree that it is an interesting question deserving more exploration.  In principle, the concept prediction architecture (Fig 2) can be layered on top of an LLM. This may make textual interpretation of concepts less faithful but could aid in resolving ambiguities.\n\nNote that we experimented with the MIMIC dataset; textual descriptions in MIMIC are written by human experts, so come with ambiguities and inconsistencies. \n\n\n**Q2: Can the authors shed light on the computational requirements for the execution of the proposed method?**\n\nThe overhead is only in training,  since the cross-modal agreement isn't utilized during testing.  Training with cross-modal agreement is 15-25% slower than without it, varying by dataset.  We'll include run times in the revised paper.\n\n**Q3: In scenarios where one modality is of inferior quality, how does the model ensure that it doesn't negatively affect the quality of representations?**\n\nIn this study, our primary focus is on scenarios where textual descriptions (only available during training) offer guidance to the visual module, see the introduction. The intent is for textual descriptions to instill beneficial biases into the visual component, thereby enhancing its robustness and interpretability. However, if textual data is deceptive – emphasizing irrelevant shortcuts while overlooking crucial concepts – it could be counterproductive. For instance, in Figure 6 and Table 6, the guidance encourages the model to concentrate on the sprite (a presumed robust feature) rather than the background (a presumed distractor). If the background were actually the vital feature and the sprite were a distractor, then that guidance would be misleading. As we discussed above (Q1), if the inferior quality refers to being noisy (rather than deceptive / adversarial), we expect this to be less of a problem. There may be ways to detect these issues. E.g., a substantial dip when using cross-modal agreement (i.e., F1 of XCB << Standard) should be a cause for concern. This could indicate shortcuts in the visual data or issues with textual descriptions, warranting a more in-depth investigation.\n\n**Q4: Given that textual descriptions, particularly in medical datasets, may possess redundant data, how does the model effectively handle and process such data?**\n\nWe don’t believe redundancy poses challenges for our method. For example, our sparsity regularizer ensures each text fragment corresponds to a single concept but doesn't penalize a concept being inferred from multiple text segments. To empirically verify this, we simulated redundancy by concatenating a 'paraphrased' description to the descriptions in the Shapes dataset and observed no discernible difference in results (XCB on redundant data  vs XCB on original data: $0.95$ vs. $0.96$ in F1; $0.79$ vs $0.78$ in Disentanglement, $0.80$ vs. $0.74$ in Completeness; $0.07$ vs. $0.07$ in Informativeness; all averages across 5 runs). We will add the experiment to the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The motivation is sound. The approach is well-presented.\n\n2. The experiments and analysis are extensive and show that the proposed approach can boost the Interpretability and robustness of the models. ", "reasons_to_reject": "1. The efficacy of the proposed method is contingent upon the availability and quality of textual descriptions accompanying images. The model's performance might deteriorate if the accompanying text is ambiguous, incomplete, or of substandard quality.\n\n2. There might be scenarios where allowing the vision encoder to learn concepts from both modalities could adversely impact the quality of representations, especially if the quality of one modality is compromised.\n\n3. The textual data, especially in medical contexts, might contain redundant information. It's unclear how the model would effectively manage this redundancy.", "questions_to_authors": "1. How does the proposed model address ambiguities or inconsistencies present in the text?\n\n2. Can the authors shed light on the computational requirements for the execution of the proposed method?\n\n3. In scenarios where one modality is of inferior quality, how does the model ensure that it doesn't negatively affect the quality of representations?\n\n4. Given that textual descriptions, particularly in medical datasets, may possess redundant data, how does the model effectively handle and process such data?", "paper_topic_and_main_contributions": "The paper introduces a novel technique named Cross-Modal Conceptualization in Bottleneck Models (XCBs) to augment the traditional Concept Bottleneck Models (CBMs). By harnessing text descriptions accompanying images, XCBs automate the concept induction process, thus avoiding manual annotation. The authors propose a mechanism to align concepts between visual and textual data. The experiments conducted confirm that their methodology offers advancements in interpretability, disentanglement of concepts, and model robustness over traditional CBMs.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "w5yYWzJ8Tw", "reviewer": "Reviewer_K3Jd", "summary": "", "questions": "A: How to handle the ambiguities or inconsistencies in the text?\n\nB: What are the computational demands of implementing the proposed method?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive feedback, and interesting questions. \n\n**Q1: How to handle the ambiguities or inconsistencies in the text?**\n\n**Regarding inconsistencies:** Note that while we encourage textual and visual components to align, it's not a hard constraint but rather a soft regularizer, used only in training. Still, systematic errors (e.g., vital concepts consistently omitted in text) could be detrimental. However, a moderate amount of `random’ noise might be less problematic. We tested this hypothesis on the Shapes dataset by taking 10% of the training examples and replacing their text descriptions with random ones. The performance did not change that much (XCB on noisy data vs XCB on original data: $0.94$ vs $0.96$ in F1; $0.75$ vs $0.78$ in Disentanglement; $0.76$ vs $0.74$ in Completeness; $0.09$ vs $0.07$ in Informativeness). These are average across 5 runs and we kept the same hyperparameters as the ones used on clean data. We'll describe this experiment in the revised paper.  \n\n**On ambiguity:** Specific types, such as lexical ambiguity, are likely less problematic in specialized domains which motivated this work (e.g., medical records).  Nonetheless, other types (e.g., anaphoric ambiguity) could be challenging for our shallow non-contextualized text encoder. We posit that even if the textual component is wrong on some examples during training, the overall impact may be minimal as long as the majority of cases are straightforward for the text component (similar to random ‘noise’ above).  Still, we agree that it is an interesting question deserving more exploration.  In principle, the concept prediction architecture (Fig 2) can be layered on top of an LLM. This may make textual interpretation of concepts less faithful but could aid in resolving ambiguities.\n\n**Q2: What are the computational demands of implementing the proposed method?**\n\nThe overhead is only in training,  since the cross-modal agreement isn't utilized during testing.  Training with cross-modal agreement is 15-25% slower than without it, varying by dataset.  We'll include run times in the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The novel XCB for cross-modal learning leveraging both visual and text data to address the significant limitations of conventional conceptual bottleneck models (CBM). By incorporating textual descriptions to guide concept induction, it overcomes the challenges associated with manual annotation and concept definition.\n\nXCB reduces the model's dependence on image shortcuts, making them more robust. This contribution has important implications for building models that are more reliable and less prone to bias or misinterpretation.\n\nThis work provides insights into the field of interpretable machine learning. By facilitating the alignment of concepts to different text fragments, the proposed method encourages disentanglement and interpretability of learned concepts.", "reasons_to_reject": "The success of the proposed method relies on the availability and quality of textual descriptions accompanying images. In cases where such text is incomplete, ambiguous, or of low quality, the performance of the model may suffer.", "questions_to_authors": "A: How to handle the ambiguities or inconsistencies in the text?\n\nB: What are the computational demands of implementing the proposed method?", "paper_topic_and_main_contributions": "The paper presents a novel method called Cross-Modal Conceptualization in Bottleneck Models (XCBs) to enhance traditional Concept Bottleneck Models (CBMs). By utilizing text descriptions accompanying images, XCBs automate the concept induction process, avoiding manual annotation. The authors devise a way to synchronize concepts between visual and text data, conducting experiments to show that their approach offers improvements in interpretability, disentanglement of concepts, and model robustness compared to conventional CBMs.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QLOSqNTcxW", "reviewer": "Reviewer_RnA7", "summary": "", "questions": "1. I'm wondering whether we could see some performance degradation in some scenarios because we are letting the concepts from the vision encoder be learned from both modalities. could this harm the quality of representations? what if the text quality is not good enough or the text is noisy? what if one modality doesn't have good quality? \n2. the text may have redundant information, especially in medical datasets. how the model would deal with it?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive feedback, and interesting questions. \n\n**Q1: what if the text quality is not good enough or the text is noisy? what if one modality doesn't have good quality?**\n\nNote that while we encourage textual and visual components to align, it's not a hard constraint but rather a soft regularizer, used only in training. Still, systematic errors (e.g., vital concepts consistently omitted in text) could be detrimental (see also our reply to your Q3). However, a moderate amount of `random’ noise might be less problematic. We tested this hypothesis on the Shapes dataset by taking 10% of the training examples and replacing their text descriptions with random ones. The performance did not change that much (XCB on noisy data vs XCB on original data: $0.94$ vs $0.96$ in F1; $0.75$ vs $0.78$ in Disentanglement; $0.76$ vs $0.74$ in Completeness; $0.09$ vs $0.07$ in Informativeness). These are average across 5 runs and we kept the same hyperparameters as the ones used on clean data. We'll describe this experiment in the revised paper. Note also that we experimented with the MIMIC dataset; textual descriptions in MIMIC are written by human experts, so come with ambiguities and inconsistencies. \n\n\n\nThat being said, our primary focus is on scenarios where textual descriptions (only available during training) offer guidance to the visual module, see the introduction. The intent is for textual descriptions to instill beneficial biases into the visual component, thereby enhancing its robustness and interpretability. However, if textual data is deceptive – emphasizing irrelevant shortcuts while overlooking crucial concepts – it could be counterproductive. For instance, in Figure 6 and Table 6, the guidance encourages the model to concentrate on the sprite (a presumed robust feature) rather than the background (a presumed distractor). If the background were actually the vital feature and the sprite were a distractor, then that guidance would be misleading. \n\n**Q2: the text may have redundant information, especially in medical datasets. how the model would deal with it?**\n\nWe don’t believe redundancy poses challenges for our method. For example, our sparsity regularizer ensures each text fragment corresponds to a single concept but doesn't penalize a concept being inferred from multiple text segments. To empirically verify this, we simulated redundancy by concatenating a 'paraphrased' description to the descriptions in the Shapes dataset and observed no discernible difference in results (XCB on redundant data  vs XCB on original data: $0.95$ vs. $0.96$ in F1; $0.79$ vs $0.78$ in Disentanglement, $0.80$ vs. $0.74$ in Completeness; $0.07$ vs. $0.07$ in Informativeness; all averages across 5 runs). We will add the experiment to the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. the novelty of matching two modalities' latent representation for CBS models to help have more explainable models.\n2. The paper presents a comprehensive evaluation and discussion \n3. the paper reads pretty well and clearly states the motivation and proposed idea followed by analysis and discussion", "reasons_to_reject": "N/A\nI have a few questions though\n", "questions_to_authors": "1. I'm wondering whether we could see some performance degradation in some scenarios because we are letting the concepts from the vision encoder be learned from both modalities. could this harm the quality of representations? what if the text quality is not good enough or the text is noisy? what if one modality doesn't have good quality? \n2. the text may have redundant information, especially in medical datasets. how the model would deal with it?", "paper_topic_and_main_contributions": "an approach for more explainability of the CBM model.\nthe research paper is about overcoming the limitations of CBM models 1. the need for manual creation and annotation and 2. the fact that concepts designed by domain experts may not be leveraged for prediction tasks. the proposed framework utilizes some guidance from textual data for getting latent representation from the vision encoder. this could encourage the model to get the predicted concepts from both text and images. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a novel method known as Cross-Modal Conceptualization in Bottleneck Models (XCBs), aimed at enhancing traditional Concept Bottleneck Models (CBMs). XCBs leverage textual descriptions accompanying images to automate the concept induction process, eliminating the need for manual annotation. The authors present a mechanism for aligning concepts between visual and textual data.

In general, all reviewers agree that this work is well-motivated and sound. The reviewers concur that it holds significant implications for the field of interpretable machine learning. Therefore, the final decision for the paper is acceptance.
Decision: Accept-Main