Cross-Modal Conceptualization in Bottleneck Models (2023)

Abstract: Concept Bottleneck Models (CBMs) assume that training examples (e.g., x-ray images) are annotated with high-level concepts (e.g., types of abnormalities), and perform classification by first predicting the concepts, followed by predicting the label relying on these concepts. However, the primary challenge in employing CBMs lies in the requirement of defining concepts predictive of the label and annotating training examples with these concepts. In our approach, we adopt a more moderate assumption and instead use text descriptions (e.g., radiology reports), accompanying the images, to guide the induction of concepts. Our crossmodal approach treats concepts as discrete latent variables and promotes concepts that (1) are predictive of the label, and (2) can be predicted reliably from both the image and text. Through experiments conducted on datasets ranging from synthetic datasets (e.g., synthetic images with generated descriptions) to realistic medical imaging datasets, we demonstrate that crossmodal learning encourages the induction of interpretable concepts while also facilitating disentanglement.

## introduction the limited interpretability of modern deep learning poses a significant barrier, hindering their practical application in many scenarios. in addition to enhancing user trust, interpretations can assist in identifying data and model limitations, as well as facilitating comprehension of the causes behind model errors. explainable ml has become a major field, with various methods developed over the year to offer different types of explanations, such as input attribution @xcite , free-text rationales @xcite or training data attribution @xcite . moreover, methods pursue diverse objectives, for example, aiming to generate humanlike plausible rationales or extract faithful explanation from the model @xcite . in this work, our specific focus is on a line of research that aims to develop neural models capable of making predictions via an (semi-) interpretable intermediate layer @xcite @xcite . this class of models is appealing because the interpretable component creates a "bottleneck" in the computation, ensuring that all the information regarding the input is conveyed through discrete variables. as a result, these variables play a causal mediation role in the prediction process @xcite . one notable representative of this model class is concept bottleneck models @xcite . cbms, for a given input image, first predict which high-level concepts are activated, and then infer the class label based on these concepts. this simple and elegant paradigm is advantageous from a practical standpoint as the intermediate layer can be examined, validated and intervened on by an expert user. for instance, in medical diagnostics, the concepts could represent types of abnormalities detected from an x-ray, which makes the model predictions easier to scrutinize and verify for a specialist. despite their interpretability advantages, cbms have several critical limitations. firstly, they require manual creation and annotation of a set of high-level concepts for each training example, which can be challenging or even unattainable in real-world scenarios. secondly, concepts designed by domain specialists might not possess sufficient predictive power for classification or could be difficult to reliably detect from the examples (e.g., images). this work aims to address these shortcomings of cbms. it considers concepts as discrete binary latent variables, eliminating the need for an expert to predefine the concepts. rather than relying on the annotation of examples with concepts our approach -cross-modal conceptualization in bottleneck models (xcbs) foot_0 -makes a weaker and arguably more realistic assumption that examples are linked to texts (such as radiology reports) which serve as guidance for concept induction. note that the text is not used at test time in our experiments. specifically, in training, we use an additional cbm model trained to predict the same target label, but this time using textual data. a regularizer is used to encourage prediction of the same set of concepts in both visual and text cbms on each training example. in this way, xcb ensures that the concepts can be predicted from both text and images, discouraging the image classifier from relying on features that cannot be expressed through language and, thus, injecting domain knowledge from text into the visual model. furthermore, an additional regularizer promotes the alignment of concepts with distinct text fragments, thereby enhancing their disentanglement and interpretability. we conducted experiments involving xcbs on various datasets, including synthetic text-image pairs as well as a more realistic medical image dataset. our experiments demonstrate that xcb tends to surpass the alternative in terms of disentanglement and alignment with ground-truth (or human-annotated) concepts. additionally, we incorporated synthetic shortcuts into the images and observed that the inclusion of textual guidance reduced the model's tendency to overly rely on these shortcuts and, thus, made the model more robust. ## related work concept bottleneck models (cbms) (koh and liang, 2017) can be regarded as a representative of a broader category of self-explainable architectures @xcite . in essence, cbms incorporate a bottleneck composed of manually-defined concepts, which provides a way of understanding 'reasons' behind the model predictions. examples must be annotated with corresponding concepts, thus, facilitating the training of two distinct segments of the model: the input-toconcepts and concepts-to-label parts. while we argued that the bottleneck should be discrete to limit the amount of information it can convey, cbms do not strictly adhere to this principle. in cbms, even though the concepts themselves are discrete, their representation within the bottleneck layer is not. specifically, each concept is represented as a logit from a classifier predicting that concept. this deviation from complete discreteness is non-problematic when the two parts of cbms (input-to-concepts and concepts-to-label) are trained either independently or sequentially. under such conditions, the logit would not be incentivized to retain input details unrelated to the concepts. yet, the problem arises when these model parts are trained simultaneously @xcite . @xcite have shown that this information leakage can be alleviated by adopting strictly discrete variables and promoting disentanglement. these insights drove our modeling decisions, prompting us to employ discrete variables and use text (as well as specific forms of regularization) to encourage disentanglement. a recent approach titled label-free cbm (lcbm) @xcite focuses on addressing some of the same challenges we address with xcbs, specifically cbm's dependence on predefined concepts and the necessity for manual annotation of examples with such concepts. this study prompts gpt-3 @xcite to produce a set of concepts appropriate for the domain. following this, the clip model @xcite is applied to label the images with concepts. in a parallel development, @xcite introduced a similar framework named language model guided concept bottleneck model (labo), which also incorporates gpt-3 and clip as key ingredients. one distinction in their approach is the employment of submodular optimization to generate a set of diverse and discriminative concepts. although these methods show promise on large visual benchmarks, it is unclear how effectively they would work in specialized domains and how to in-tegrate additional human expertise in such models. we consider lcbm as an alternative to xcb in our experiments. @xcite introduced a method that posthoc transforms any neural classifier into a concept-based model (cbm). in contrast to their approach, we leverage text to shape the representation inside the neural classifier, potentially making them more disentangled. in addition to efforts focused on crafting methods for training cbms, there is an intriguing branch of research dedicated to exploring the utilization of concepts within cbms to intervene and modify the models @xcite . while this strand of research appears somewhat divergent from our primary interest, it underscores another significant application of cbms -extending beyond mere prediction interpretation. ## proposed framework in this section we introduce a novel method for cross-modal conceptualization in bottleneck models (xcbs). we start with a high-level overview of the framework and introduce its core components. further, we consider models used to process visual and textual modalities. finally, we describe techniques that were used to encourage cross-modal agreement and disentanglement between the concepts. ## empirical evaluation this section describes the datasets, introduces evaluation metrics for interpretability, and describes the procedure used to measure robustness of models. ## results and discussion in this work, we compare the proposed models with the standard 'black-box' model, as well as cbms and lcbms @xcite . due to similarity of lcbm and labo @xcite ) models, we include in comparison only the former (see section 2 for their discussion). the experiments were conducted using synthetic and public datasets highlighted in section 3.1 of the paper. the obtained results are discussed with respect to their performance, interpretability, and robustness. the ratios between training, validation, and test splits are 75%, 15% and 10%, respectively. for all datasets image size was set to 299 with a batch size of 64. at each epoch, images were normalized and the batches were randomly shuffled. the cub-200 follows data processing pipeline described in @xcite with additional augmentations such as horizontal flip, random cropping, and color jittering. the hyperparameters were optimized using tree-structured parzen estimator sampling algorithm @xcite . to concentrate on expressivity of concept extractor c, we restricted our feature extractor f to be fine-tuned inceptionv3 model @xcite pre-trained on the imagenet dataset @xcite , with multi-layered perceptron of depth of one for p f and p c . the baseline model was com-posed of feature extractor f followed by sigmoid function and predictor p f . the size of latent representations was set to 10, 320, and 20 for shapes, cub-200, and mimic-cxr datasets respectively (tab. 1). textual embeddings were randomly initialized as trainable parameters of length 50. the cbm size of bottleneck matched the number of attributes in a dataset (tab. 1). to run experiments with cbm foot_3 and lcbm foot_4 the official implementations were used. weights of all models were initialized using xavier uniform distribution. the feature extractor f and predictor p f were optimized jointly using adadelta optimizer @xcite with learning rate 0.25 and rho 0.95. the concept extractor c and predictor p c were optimized jointly via adamw optimizer @xcite with learning rate 0.001, betas of 0.9 and 0.999, and weight decay 0.01. the learning rates in adamw and adadelta were adjusted using onecyclelr scheduler (smith and topin, 2019) with maximal learning rates set to 0.001 and 0.25, respectively. temperature of the gumbel sigmoid followed the exponential schedule @xcite . the experiments were conducted using early stopping policy on validation loss with patience of 5 epochs. five random seeds were used: 42, 0, 17, 9, and 3. mean and standard deviation for the obtained results were computed over five training runs. ablation study. components introduced in xcb that aim to improve interpretability and disentanglement of representation cause drop in performance (tab. 2). nevertheless, disentanglement and completeness metrics were substantially improved by discretization of latent representations via gumbel sigmoid and using "dummy" tokens in slot normalisation respectively. remarkably, using kullback-leibler divergence as a tie loss worsens both performance and dci, which could indicate that neither modality fully guides induction of implicit high-level abstractions. performance. the standard model outperforms cbms on all datasets (tab. 3 ), which can be attributed to a trade-off between performance and level of interpretability. another possible reason is that manually derived set of attributes might not be predictive enough for classification. on the medical dataset lcbm is inferior to other models indicating current limitations in the domain knowledge of large language models. xcb achieves comparable to other variations of cbms performance. interpretability. the highest dci scores (see section 3.2 of the paper) were achieved with the proposed xcb model that could be attributed to domain knowledge transferred to the model (tab. 3 ). high values of both disentanglement and completeness indicate that alignment of latent representation and set of attributes tends towards bijective. although absolute values of dci on medical dataset are modest, out survey indicates that xcb possess a better quality of interpretations in terms of xscore compared to lcbm: 0.72 vs. 0.66. the framework associates each factor in latent representation f with a set of textual interpretations (fig. 5 ). for example, the images corresponding to the fourth neuron activations cluster in terms of shape, i.e., larger value of logit correspond to squares, whereas smaller represent triangles. explanations of our model are indeed referring to the shape of a figure: highest relevance is achieved by tokens "triangular" and "quadratic".