Enhancing Concept Localization in CLIP-based Concept Bottleneck Models
Authors: Rémi Kazmierczak
Steve Azzolin
Goran Frehse
Eloïse Berthier
Gianni Franchi
Venue: Accepted by TMLR ()

Abstract: This paper addresses explainable AI (XAI) through the lens of Concept Bottleneck Models (CBMs) that do not require explicit concept annotations, relying instead on concepts extracted using CLIP in a zero-shot manner. We show that CLIP, which is central in these techniques, is prone to concept hallucination—incorrectly predicting the presence or absence of concepts within an image in scenarios used in numerous CBMs, hence undermining the faithfulness of explanations. To mitigate this issue, we introduce Concept Hallucination Inhibition via Localized Interpretability (CHILI), a technique that disentangles image embeddings and localizes pixels corresponding to target concepts. Furthermore, our approach supports the generation of saliency-based explanations that are more interpretable.

Published in Transactions on Machine Learning Research (01/2026)
Enhancing Concept Localization in CLIP-based Concept Bottleneck Models
Rémi Kazmierczak
ren..kcmmirrcakkSSsttepoorrr.
Unité d d'Informatqque et d'Ingénierie des Systèmes
ENSTA Paris, Institut Polytechnique de Poris
Steve Azzolin
Hecve.azzlinnnhmmnn
Department Information Engineering and Computer Science University of Trento
Eloise Berthier
eloise. bettebrrrttiieresttpprrrri..
Unité d d'Informatique d' d'Ingénterie des Systèmes
ENSTA Paris, Institut Polytechnique de Paris
Goran Frehse
goran.frehaseeossto-pprrrs
Unité d' d'Informatique et d'Ingénierie des Systèmes ENSTA Paris, Institut Polytechnique de Paris
Gianni Franchi
giommi.rrncchieenste paris,
Unité d d'Informatique et d' d'Ingénierie des Systèmes ENSTA Paris, Institut Polglechnique de Paris
Reviewed on OpenReview: https: // openreview. net/ forum? id= 2ca01 uw
Abstract
This paper addresses explainable AI (XAI) through the lens of Concept Bottleneck Models (CBMs) that do not require explicit concept annotations, relying instead on concepts ex- tracted using CLIP in a zero-shot manner. We show that CLIP, which is central in these techniques, is prone to concept hallucination incorrectly predieting the presence or absence of concepts within an image in scenarios used in numerous CBMs, hence undermining the faithfulness of explanations. To mitigate this issuc, we introduce Concept Hallucination Inhibition via Localized Interprelability (CHILI), technique that proposes a disentangling of image embeddings. Furthermore, our approach supports the generation of saliency-based explanations that are more interpretable.
Introduction
Deep Neural Networks (DNNs) are now used many areas, including sensitive domains such as medicine and law. In these settings, trust essential. To build trust, the field of Explainable Artificial Intelligence (XAI) provides tools that help users understand how DNNs make decisions. One important family of methods is concept-based explanations. These explanations describe predictions using human-understandabb concepts, often expressed as words. For example, a model that classifies an image as a dog might rely on concepts such as fur, ears, snoul, or pamos. The ability of a model to represent raw data (e.g., images) as concepts called conceptual representation is therefore key to creating models that can provide such explanations.
A popular way use concepts is embed them directly into the model. This creates an interpretable latent space, where each neuron corresponds concept. Models built. this way are known Concept Bottleneck

---

Published in Transactions on Machine Learning Research (01/2026)
Models (CBMs) (Koh et al. 2020 Bennetot al. 2022 While CBMs improve interpretability by design, they usually require concept annotations during training, which are expensive and difficult to collect.
Recently, contrastive language-naage models, such as CLIP Yan et al. 2023a have been widely used for tasks like zero-shot classification and open-world recognition. Because CLIP links images and text, researchers have started using as a free source of concepts for CBMs (Yang et al. 2023 Panousis et al. 2023 Cui al., 2023). This removes the need for manual annotations, but also introduces a new challenge: the concepts extracted by CLIP may not always reflect what is actually in the image.
A particularly critical challenge for CBMs is the phenomenon of concept hallucination (illustrated in Fig- ure D wherein concepts are inferred based on contextual cues rather than their actual presence within the image. This issue undermines the foundational hepothesis of CBMs that the concept bottlenek serves as a faithful conceptual representation of the image content. While prior work has addressed related chal- lenges (Oh Hwang, 2025 Liu al.) 2024b), our approach distinguishes itself in two key aspects. First, we not only mitigate concept hallucinations but also enhance their localization by explicitly addressing the spatial distribution of activation maps. Second, extend the applicability of our method to CBMs, thereby offering tailored solution for improving both the reliability and interpretability of these models.
Concept
Concept
Image
Concept N
CLIP
Soore (Conoept
(Concept
Prediction
Tail in the image
Score (Conscpt
Clasiffir
CLJP Score
Imraee
17.40
20.07
"Tail"
CLIP mo
(a) Principle of CLIP-based concept bottleneck model.
(b) CLIP hallucination issue.
Figure 1: Using the CLIP-score between embedings of input images and predefined concepts, labeling-free concept extraction can be performed, allowing prediction on an interpretable latent space (left). However, CLIP tends to hallucinate the presence of concepts, troubling the localisation of CLIP-based CBMs (right)..
Our contributions are as follows:
We conduct extensive statistical analysis to investigate the relationship between the CLIP-score and the localization of concepts. Notably, our findings demonstrate that CLIP-scorce fail to accu- rately represent the actual location of concepts within images.
Based this observation, propose CHILI (Concept Hallucination Inhibition via Localized In- terpretability), method that aims to disentangle the activations of CLIP, and consequently the CT.rP-score, distinguishing between object representation, which is related to the physical location of the concept, and contextual representation, which pertains to activations associated with features that do not directly represent the concept but suggest its presence.
To demonstrate the efficacy of our discntangling method, we employ it means to perform image segmentation and binary classification of concepts in spurious situations. We showcase that our method achieves slightly superior results compared to concurrent methods in both tasks.
We apply CHILI to real-world use cases to construct new, more interpretable CBMs. Our results demonstrate that such intervention is feasible with only limited accuracy cost.
2

---

Published in Transactions on Machine Learning Research (01/2026)
2
Related Work
Concept Bottleneck models (CBMs) CBMs constitute a class of models that exploit a conceptual representation of input data, termed the "concept bottleneck," " facilitate inference, thereby enhancing interpretability. While certain studies employ custom datasets featuring concept annotations to construct CBMs (ooh al. 2020 Diaz-Rodrigues et al.
2022), the emergence of text-image contrastive foundation
models has signilicantly propelled research in this area, enabling the development of CBMs without explicit
concept annotations (Yan et al.) 2023a Kazmierczak et al. 2024). Notably, CLIP Yan et al. 2023a has
become the predominant choice for crafting these CBMs (Kazmierczak et al. 2025
Neuron interpretation To interpret the behavior of a model post-training, a commonly employed ap- proach involves identifying the role of specific neurons in the process by detecting patterns that induce their activation. Some methods directly display neuron activations response to designed inputs (Gandclsman) statistical analysis on probing datasel. to achieve this goal et al. 2023 More sophisticated techniques Alternatively, optimization techniques can be employed to (Shaham et al. 2024 Kalibhat et al. 2023
determine the input that maximizes the activation of given neuron (Olah et al. 2017
Saliency based explanations To explain image-based decisions, saliency-based explanations- which aim to highlight. the most influential regions according to the model- are widely adopted. Among these, model- agnostic approaches such as SHAP (Lundberg & Lee 2017), LIME (Ribeiro et al. 2016). and RISE (Petsiuk (et al. 2018) are particularly popular due to their versalility. These methods analyze model behavior in response to perturbed versions of the input image.
Additionally, the formal structure of deep learning models has enabled alternative approaches for generating saliency maps by directly examining activation patterns (Zhou el al. 2016 Gandelsman et al. 2023). Another distinct class of deep neural network (DNN)- based saliency methods leverages gradients to weight activations, as seen in Grad-CAM (Selvaraju et al. 2017), FullGrad (Srinivas Flcurct 2019), and HiResCAM (Draclos
& Carin 2020 However, such gradient- based techniques require differentiable computations, a constraint that does not apply to conceptual representations.
3 Evaluating the concept localization abilities of CLIP
3.1 Preliminaries
First, let. us define some notions that we consider essential to describe the experiments we will perform.
The widespread success CLIP has spurred significant. research effort around its inter- pretability, Existing studies primarily focus on two issues: bias and spurious feature reliance, and concept hallucination.
Related studies
The most extensively studied aspect is bias, often examined through image classification tasks. Works such those by Moayeri et al. (2023b); Zhang et al. (2024) demonstrate accuracy drops on biased datasets, revealing CLIP's reliance on spurious features. Furthermore, Birhane et al. (2021): Hall et al. (2023) show that these biases extend to societal concerns, including gender and racial discrimination. Mitigation approaches include fino-tuning (Alabdulmohsin et al.] 2024 Gerych al. 2024) and activation decomposition (Yco et al., 2025). Another key challenge is CLIP's ss lendency to hallucinate text or objects during text-image similarity computations (Oh & Hwang. 2025 Liu et al., 2024b), a phenomenon attributed to the modality gap, where one modality contains more information than the other (Schrodi et al. 2024).
While these studies address general seltings, we focus on the specific context of CBMs. This setup presents unique challenges, as concept sets are often highly correlated by design. To our knowledge, the literature lacks a comprehensive evaluation of CLIP's relevance in CBMs, except for the pioneering work by Debole et al.) 2202),, which assesses the quality of embeddings derived from foundation models. Our study distinguishes itself by addressing concept. hallucination in CBMs.
3

---

Published in Transactions on Machine Learning Research (01/2026)
Another underexplored aspect is localization. CBMs implicitly assume that concept representations should not only detect the presence of concepts but also locate them within images. Pre-CI.IP CBMss achieved
this through backbones trained with localization-aware loss functions Diaz-Rodriguez et al. 2022
Bennetot
cct al. 2022 Regarding mitigation methods, Srivastava et al. (2024): Huang & Huang 2024 propose
line-Luning the CLIP backbone to improve localization. The closest work to ours Yeo et al.
20025 which
identifies attention heads responsible for hallucination. However, our method differs in both the identification
approach and its application to CBMs. Another noteworthy work is Zhang et al. (0026), which proposes model-agnostic method based spatially-aware saliency maps by design.
Class / concept In the context of CBMs, classes refer to the target labels intended for prediction, which are inherently determined by the dataset. Concepts, by contrast, represent a set of interpretable enti- ties-most commonly textual descriptions-that serve as proxies for performing inference. Within CBMs applied image classification, an image first. represented in terms of these concepts, after which the class prediction is derived from this conceptual representation. In most implementations, concepts correspond to subcomponents of the target label. Two predominant approaches have emerged for defining these concepts. The first one involves prompting large language models: for instance, Yang et al. (2023) extract concepts by querying GPT-3 with prompts such as "describe what the [CLASS NAME] looks like." The second approach leverages dedicated datasets that annotate specific attributes or subparts the output class present in the image Didr-goor al. 2022).
3.2 Probing CLIP for Concept Hallucination
To study potential limitations of CLIP-based Concept Bottleneck Models (CBMs), we first need to under- stand what drives high CLIP score. In this subsection, we design an experiment test whether CLIP embeddings reliably reflect the physical presence of concepts in images, whether they are influenced by contextual or semantic cues.
Datasets We perform experiments on three different datasets: ImageNet Deng et al. 2009 Momu-
mAI (Lamaa et al. 2021 and CUB (Wah et al. 2011).
ImageNet: A large-scale object classification dataset. where classes correspond to everyday objects (e.g,, kit for), and concepts refer object parts (e.g., head, tail, paw). Since ImageNet lacks fine- grained part annotations, we extended it with + [Li et al. 2024
MonumAI: A dataset focused on monument style classification, where classes are architectural styles and concepts correspond to structural elements such as arches, columns, or domes.
CUB: A Ane-grained bird classification dataset, where concepts are visual parts such wings, beak, or tail.
These datasets cover wide range of tasks, from generic object recognition to fine-grained classification, making them suitable for evaluating concept detection. A complete list of the concepts used in each dataset is provided in Appendix B
CLIP Score as a Mcasure of Concept Detection Given an image and a text T, CLIP uses an image encoder Ming()) and texl encoder Mixxt() (+) project them into shared embedding space. The similarity between image and text is computed by the cosine similarity:
S(I,)) = (Ming (T), (T), Mtext (T))
This score allows CLIP match images and text without explicit training on the target datasct, which is why it has become a standard tool for zero-shot classification and concept detection. However, the score is high even when the concept is absent from the image, it indicates hallucination problem.
4

---

Published in Transactions on Machine Learning Research (01/2026)
Experimental Setup We construct three subsets of data given two classes and o2, and concept k that is strongly linked to ej but absent from 02:
Images of class ej where concept present.
Images of class a where concept k is absent.
Images of class 02, where concept k is always absent by design.
For each triplet (c1, c2 k), we sample images randomly and repeat the process 10 times. The full list of triplets is given in the appendix.
The goal of this setup to test whether CLIP tell apart the true presence of a concept from its semantic association with class. Concretely, compute the average CLIP score of each subset with respect to the concept k. We also compute the failure rate, defined as the fraction of cases where the subset without the concept receives a higher score than the subset where the concept is actually present.
Results and Analysis Table | reports the average CLIP score across all three datasets. The results show that CLIP does not reliably separate the true presence of concept from its absence. For example, in both MonumAl and CUB, the scores for k-present and k-absent subsets of class с₁ are almost identical, indicating that CLIP relies heavily on class-level associations rather than visual evidence. The high failure rates (40 (40-50%) further confirm that CLIP often assigns higher scores images without the concept than to those containing it. This demonstrates significant risk of concept hallucination, raising concerns about using CLiP-based embeddings as faithful representations in CBMs.
Foil. Rate
k present
k absent
(k absent)
MonumAl
18.16 + 2.45
18.09 + 2.38
16.71 + 2.75
0.40
CUB
15.58 + 1.74
15.61 + 1.65
12.73 + 2.06
0.50
ImageNet
19.35 + 1.93 19.18 + 1.37
14.82 + 1.80
0.40
Table 1: Average CLIP score on different. setups. In the first column, & is present in the images. Tn the second and third ones, & is not present. Fail. Rate presents the failure rate, iee, the fraction of cases where the subset of images that do not possess the desired concept induces higher score.
4 Disentangling concept representations
4.1 Preliminaries
To address this challenge, we introduce a novel method, CHILI, for disentangling concept localization from concept suggestion within the conceptual representation images. The primary objective of this approach is provide users with a conceptual representation that decomposcs into distinct components: one related to the object of interest and another one to its surrounding context. By selectively retaining only the object-related component, method aims to produce conceptual representation that reduces concept hallucinations.
Notations We now describe in more detail how the image encoder Ming of CLIP computes its representa- tions. The encoder is a Vision Transformer (VIT) consisting of L stacked transformer layers, each containing a multi-head selt-allention (MSA) block and a multi-layer perceptron (MLP) block. The input image is first split into a sequence of patches, linearly projected into tokens, and augmented with a special class token (denoted by index cls). These tokens are processod layer by layer through the transformer. Formally, we denote by E [1, H]: the attention heads, 1 e []: the transformer layers, i [1,N]] the patch tokens, and Z the residual stream (intermediate representation) at layer 1.
5

---

Published in Transactions on Machine Learning Research (01/2026)
The image encoder produces single vector representation of the image by applying learned projection P to the final embedding of the class token. In CLIP, this is written as:
(1) P. [zL] cls
where [z*] denotes the class token at the final layer.
Unrolling the Transformer. Each layer of the transformer updates the residual stream by combining the outputs of the MSA and MLP blocks:
z' z'= + +MA''' (SAA(A MT.P' (z),
where Z denotes the normalized activations after the attention block.
By expanding this recursion, we can express the final class token representation sum of contributions from all layers:
Mang (T) P [2
P
P MLP' (
(1)
cls
cls
Decomposition into Attention Heads. Following the analysis of Elhage et al. (2021): Gandelsman el al. (0023), the image embedding M1) can be written as the sum of contributions from each transformer layer each attention head h, and each image token Intuitively, let consider that transformer layer has output linear map (usually denoted Wo) that mixes the heads and produces the final MSA vector. Applying W/ and then the projection P to the cls MSA output gives
P(MSA' p P(W's (Headr .... Head
For our decomposition it is convenient to view the effect of w as linear map applied to each head and then summed. Thus we may write
" E.wwwo
P[MSA'ZZ-)]J.
(Z
(a (a )
Am 1m0
where w.th denotes the linear map that extracts the contribution of head h after the usual output projection, I.h u. = (z,.(ZZ--11 is the value vector produced for token by head at layer 1, and els, is the attention weight from the class query to token i in head h, layer
We now define the vector contribution coming from token i, head h, layer I after all linear projections:
miLk = PW (o L. (Oels, vi l.).
(2)
Each miya is vector in the same embedding space MIn((I. With this definition can rewrite the sum of all MSA contributions compactly:
PMMA (2
mi.ik-
I=1
Using equation 2 and the expansion above, equation becomes
H
N
Mang (I) = P|Z +
maLkh +
P MMLP' (2
h= t=0

---

Published in Transactions on Machine Learning Research (01/2026)
Decomposition of the activation map
Decomposition of the score
Activation map related to the concept "a lagge cone-shaped bill"
Image
Σ
I
A
A Context
SContest
Σ
A Objoct
SObject
Figure 2: Decomposition of the activation map
The first term PiZe]as Jcls the projected initial class token; the last sum collects the MLP contributions. Since, by definition, the CLIP score is
S(1,)) = (Ming()), Meexe (T))
inserting the expression for Ming(1) (T) gives
H
N
- Maext (T))
(mi,ar Maext (T)) +
([MMPP (2'))ss Mrex(
Let us define
(T) = (mi,a,, Meex(()) (T)),
which is the scalar aligmment between the bead/token contribution and the text embedding. If we collect the small terms (initial class-loken projection and the MLP outputs) into a residual e, we obtain the compact decomposition proposed by Elhage et al. (2021):
Gandelsman et al. (2023
S(1,T)
ΣΣΣ
(T) +
(3)
1=1 1m1 i=0
Here we have (P(Z") Icls, Muxa Mixx(TT) (T)) E (PIMLP'(2")
Miext (T)).
Notably, can represent the contribution of a specific MSA head h and layer I to the score attention map by grouping terms AI,i, by position, which we denote by Ar, = [AL When reshaped, At, can represent the heatmap illustrating the patch-wise contributions as tensor of size N. Finally, we denote by A the summed attention map:
A
4.2 Concept Hallucination Inhibition via Localized Interpretability (CHILI) our method
Our goal to find a decomposition of Aya into two terms, respectively representing the activations related to the effective presence of the object in the image, and the activations related to the suggestions of the presence of the concept (see Figure 2
Context ALE = I,
+ A
Object
(4)
A Context: is linked with all the content in the image that is not the concept, i.c., locations that do not overlap with the segmentation of the concept.

---

Published in Transactions on Machine Learning Research (01/2026)
Object is linked with all the content in the image that is the concept, i.c., locations that do overlap ALA
with the segmentation of the concept.
Filtering Pseist-Register Artifacts The first step of our method is to remove high-norm tokens, which are known artifacts of Vision Transformers trained on large datasets (Darcet et al. 2024 Lu et al. 2025). We refer to these artifacts as pseado-registery they tend store global information, but their spatial localization on the activation map is not meaningful for interpretability. Following this intuition, we separate this component and denote it the pseudo-register part. Formally, for each attention map Ar, al layer I and head h, we define the pseudo-register part as the residual after applying median filter:
A
P. register i
At,h (m)))))
where fm a median filter with kernel size 3. This operation removes localized noise while isolating the global, non-informative artifacts.
Remaining activations. After filtering, assume that the remaining activations in At represent the spalially-dependenll part of the conceptual representation. In other words:
some neurons locus on delecting palterns that directly correspond the object (concept) described by the text T,
while other neurons detect contextual features that indirectly suggest the presence of T.
Weighting heads and layers. To search for such a decomposition, we assign weight wi,s each pair (I, h) according to score based on the Intersection over Union IoU(:--) between a pseudo mask based on activations and ground truth segmentations obtained using a probing dataset:
[1 11- ToU((... (Aca).0)))
GE,,
with D Probee the probing dataset containing activations Ath and ground truth segmentations G that segment the concept, a is temperature scaling hy peprraaraeeter, and
{
fm(Aun) > mean(fml (At,)) fm((A,)) S mean(ff(A))) (A).)).
hm (AI.))
In our experiments, the probing dataset corresponds to, for each data point of the training set, the selection of a random concept present in the image, and the corresponding mask. Note that for each test dataset, we considered different probing set.
Decomposition of activations. Once the decomposition is performed for each head and layer, using these weights, we split each activation map into two parts:
Object Wh,h fm (A())))) A
A Context = =(1 - w(,) Sm (ArN))
Here:
Object captures features directly aligned with the concept (object-related),
A I.
Context captures features not aligned with the concept, ie., contextual cues. I.
Siuce previous works (Darect al. 2024) demonstrate that incorporating registers into the model architecture effectively mitigates these artifacts, adopt the term "pscudo-registens" for clarity and convenience. However, emphasize that this interpretation while strongly supported by empirical evidence--rmmann working hypothesis rather than definitive conclusion.
8

---

Published in Transactions on Machine Learning Research (01/2026)
Monumai
ImageNet
CUB
LTC (Yeo et al. 2025)
0.536 (0.020)
0.566 (0.016)
0.532 (0.013)
Concept Attention (Gandelsman et al. 2023
0.516 (0.018)
0.495 (0.022)
0.485 (0.014)
Register
0.548 (0.018)
0.495 (0.039)
0.487 (0.019)
CHILI (ours)
0.565 (0.015)
0.596 (0.017)
0.533 (0.020)
Performance comparison across datasets on concept detection. Results are shown for different methods (LTC, CHILI, Register, and Concept Attention) on three datasets: Monumai, ImageNet, and CUB. Values represent the mean AUC with variance averaged over different runs.
Table 2:
Score decomposition.
By combining equation
equation
and the resummations among tokens:
SObject
Σ
A
Object
i=0
N
gContext
A Context
==0
we can also disentangle the CLIP score into two interpretable contributions:
S(I,T) SObject &Context +
(5)
The activations Objectt = E A and Context = Ete Σ Context can thus be interpreted as the token-level contributions to the scores gObject and SContext, respectively, for the given image and text.
For further details, we invite the reader to look at Appendix C that provides observations about the pscudo- register filtering method, the isolated contribution of each component, the temperature parameter, and the probing dataset.
5
Experiments
5.1
Concept detection
The most straightforward way evaluate the efficiency of our method evaluate on a binary object detection task. To do so, compute the AUROC score in the case where the class present in the image is intended to (column 1 vs column 2 of Table D using the different components S (refered as Concept Attention)
the same setup the statistical analysis in Section 3.2 From this setup,
(refered as Prregister from the decomposition of equation and locate-then-correc (LTC) (Yeo et al. 2025 The results are displayed in
CHILI) gObject (refered CHILI in the table), 5Context, SPrreiitter -> 1= Σ Aith
Table 2
First, we observe that the bascline-which corresponds to using the raw CLIP score struugglles to detect the presence of the concept, thereby reinforcing the findings Scction 3 Regarding the disentangled components, the 5Context component also exhibits poor detection performance. In contrast, the 5Object component demonstrates significantly higher detection capability, as intended by design.
The use of the pseudo register component &P.rogister showcases similar performances to using the raw CLIP score, indicating that pseudo registers contain non-localised, hallucination-prone information.
5.2 Object segmentation
To test whether our method can localize concepts images, we adapt into a scgmentation module. The task is to segment both classes and concepts from ImageNet across the entire test set. In practice, we evaluate how well the activation maps highlight the relevant, pixels using three standard metrics:
9

---

Published in Transactions on Machine Learning Research (01/2026)
Pixel accuracy (Acc.) percentage of correctly classified pixels,
mean Intersection over Union (mloU) overlap between prediction and ground truth masks,
mean Average Precision (mAP) - quality of the predicted mask in terms of precision.
We compare method with several post-hoc interpretability approaches (i.e., without fine-tuning the model): LRP (Binder et. al. 2016), partial-.RP (Voita et al. 2019), rollout (Abnar & Zuidema, 2020). raw attention, Grad-CAM (Selvaraju et al. 2017p, Chefer et al. (Chefer et al.) 2021), and Concept Atten-
tion Gandelsman et al. 2023 Among them, two are natural baselines for the concept-level segmentation:
Raw attention: the penultimate layer the vision transformer (Ming (?))
Concept Attention: the original, non-disentangled activation map A.
Table reports the results.
We highlight the scores of our Object component (CHILI) A Abbeet
Method
Pixel Acc. t
mloU
mAP
Closs-level segmentation
LRP
52.81
33.57
54.37
partial-LRP
61.49
40.71
72.29
rollout
60.63
40.64
74.47
raw ablention
65.67
43.83
76.05
GradCAM
70.27
44.50
70.30
Chefer et al.
69.21
47.47
78.29
Concept Attention
76.78
57.14
82.89
CHILI (ours)
78.79
60.22
84.86
Concept-level segmentation
Concept Atlention
70.86
46.17
87.65
CHILI (ours)
71.76
47.74
88.38
Table 3: Segmentation performance on ImageNet. concept-level segmentation (bottom). Higher is better.
Results for class-level segmentation (top) and
Analysis. From the class-level results, we observe that our method outperforms all baselines across all three metrics. In particular:
Compared Concept Attention, our discntangled Object map improves pixel accuracy by +2.0 points (76.78 78.79), mloU by +3.1 points (57.14 - 60.22), and mAP by +2.0 points (82.89 84.86).
The improvement other classical methods such Grad-CAM (+8.5 mloU) rollout (+19.6 mAP) is even more pronounced.
At the concept level, we also see consistent but smaller gains: about +1 point in accuracy, +1.5 in mloU, and +0.7 in mAP. These results confirm that isolating the Object component leads to cleaner and more accurate localization than using the full, entangled activation map. This demonstrates the interest of CHILL.
Remark. From an XAI perspective, simply providing accurate object segmentations not enough build reliable Concept Bottleneck Model (CBM). In the next section, we show how CHILI can be leveraged construct CBM that is not only accurate but also trustworthy.
10

---

Published in Transactions on Machine Learning Research (01/2026)
6 Applying CHILI to CBMs
6.1 Method
In the previous sections, we showed how CHILI allows us to mitigate the localisation issue of CBMs, leading to more interpretable concept extraction. We now turn to the question of how this disentanglement impacts the performance of CRMs.
Baseline. As baseline, we consider a standard CLIP-based CBM (Yan et al. 2023b), which relies directly the full CLIP similarity score S.), In contrast, approach replaces this score with the disentangled object-only component 5Ôbject, introduced in Section
Since 5Object is less alfected by contextual bias and concept hallucination, we hypothesize that it can yield a more interpretable CBM, albeit at the possible cost of predictive accuracy.
Evaluation. For each dataset, we compare the classification accuracy of the baseline CBM (using S) with the proposed CBM (using 5Object). The results are reported in Table
Method
Monamai
ImageNet
CUB
Baseline CBM (S)
74.67
73.55
65.05
CHILI (ours, 5Object)
74.34
72.80
64.90
Table 4: Classification accuracy CBMs with and without CHILL Results are shown for three datasets. Bascline CBM uses the full similarity score S, while our approach uses the disentangled score gObject.
Analysis. Our results show that applying CHILI leads to only minor decrease in accuracy across datasets, suggesting that it can be viable strategy for improving CBM interpretability without severely compromising performance. The effect depends on the dataset:
CUB: There almost no accuracy loss, which suggests that object-related signals dominate the decision process in this dataset.
Monumai and ImageNet: small drop in accuracy occurs, likely because contextual features (e.g,, background or environment) play a role classification. By removing them, the model becomes less biased but also loses some useful cues.
Discussion. Overall, these findings highlight trade-off:
On the positive side, CHILI reduces bias and mitigates errors caused by spurious correlations, especially in fine-grained datasets such CUB.
On the negative side, filtering out contextual information inevitably discards some predictive fea- tures, which can slightly reduce accuracy.
Importantly, there is no guarantee that reducing hallucinations will improve accuracy. However, from an XAI perspective, prioritizing faithfulness and interpretability is crucial, and our results suggest that CHILI can achieve this while keeping performance reasonably close to the baseline.
6.2 Explanations
Once the model is trained, we can leverage the activations produced to build visual explanations that gather the name of the most important concept and their location on the image. We explain the process below.
We extract the concept representation using CHILL We then apply DeepSHAP {Lundberg & Lee 2017) the model, with key distinction: rather than computing SHAP values at the image level, as conventional
11

---

Published in Transactions on Machine Learning Research (01/2026)
large, hawk-liik body
dark brown black color
rusty brown color
a large, seabird
brown and black feachers
largo, hawk-l ke body
-14 .9
+10.33
rusty color
a large, seabird
9.03
brown and black feathers
88.00
1681 other features
+82.95
EE0O0]
-5.415
Figure Example of explanation produced by the intervention of CHILI in a CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: Orchard Oriole
for image classification tasks, we perform the computation the concept level. This approach allows us quantify the importance of each concept in the inference of the target label. Finally, for the top five concepts identified by DeepSHAP, we visualize their contributions heatmaps derived from their corresponding activations, A0bject, Examples of these explanations, including failures cases, are presented in Figure 3 and Appendix
7 Limitations and discussion
In this work, we studied the ability of CLIP to focus on patterns located the object designated by the textual encoding to produce an inference and proposed a way to disentangle the activations of the model without fine-tuning. We want to discuss here the limitations of such procedure.
Layer/head dccomposition We base our approach on a layor/head docompasition to achieve the disen- (angling. Such an assumption, as noted by Gandelsman al. (2023), neglects indirect effects, i.e., potential interactions from previous layers on deeper ones. Additionally, assume that each position in the layer/heor pairs can be attributed to pseudo-register, context, object (or at least can be more easily separated by doing so).
Other factors of high CLIP score We voluntarily focus on the impact of the object's presence the increase or decrease of the CLIP score. However, being a complex model, the factors that influence high CLIP scores are multifactorial. For example, the proportion of salient features @Darcet et al.) 2024h or the text prompt (Zhou et al. 2022) also inffuences its output. Such factors are a notable reason why our discntangling does not completely eliminate the failures in the experiments of Section 3.2 For example, the images from the case c C1; l absent have by construction many more close-ups than the case c == c1; present, inducing perturbation towards higher scores. is also evident that CLIP suffers from numerous biases (Moayeri al. 2023b) that can influence the score in either a decreasing increasing manner.
Role of pscudo registers One aspect of our disentangling is the presence of high-norm artifacts (Darcet et al. 2024h, which we refer to as pseudo-registers. The reason dedicated special part to them in
12

---

Published in Transactions on Machine Learning Research (01/2026)
decomposition is that their role is ambignous: since they do not seem to exhibit spatial coherence with the image, it is difficult to determine whether they store information about the object or the concept.
8 Conclusion
In this paper, we examined the limitations of using CLIP as concept extractor. Through statistical analysis, we identified challenges associated with correlating high scores with the localization of concepts in images, particularly in cases where the presence of concept is merely suggested. To address this issue, we propose method that factorizes the embedding space into components related to the object, the context, and pseudo-registers Empiriaal results demonstrate that our disentangling approach can partially eliminate the contextual aspects of conceptual representation thereby advancing towards more bocalizationfoouusee CLIP- based concept bottleneck models. In addition, while a probing dataset is required to compute the calibration performed in our method, our approach does not require any additional training the model.
However, several limitations are present our study. Primarily, we were unable achieve complete disen- langlement of the activations. This shortcoming can be attributed to multiple hypotheses we adopted in our experimental setup. Notably, neglected second-order effects and assumed that attention heads are not polysemantic, an assumption that is somewhat reductive.
In this paper, we limit ourselves to CLIP based on VIT backbones. This restriction is motivated by the fact that this paradigm has become the gold standard for most zero-shot CBMs released in recent years (Yang et al. 2023 Yan et al., 2023a Kazmierczak et al. 2024). However, we plan to extend these analyses other overlooked backboncs, such as those based on ResNet, and models, such as LLaVa. The goal is twofold: first, this opens the way to post-hoc disentangling on other CBMs; secondly, it allows to st study the similarities and differences across various embedding networks, potentially leading to more interpretable zero-shot CBMs.
9 Acknowledgments
This work was performed using HPC resources from GENCI-DDRIS (Grant 2025 AD011014675R2)
References
Samira Abnar and Willem Zuidema. arXiv:2005.0003 2020.
Quantifying attention flow in transformers. arXiv preprint
Tbrahim Alabdulmohsin, Xiao Wang, Andreas Steiner, Priya Goyal, Alexander D' D'Amour, and Xiaohua Zhai Clip the bias: How useful is balancing data in multimodal learning? orXiv preprint ar arXiv:2403.0433 2024.
Adrien Bennetot, Gianni Franchi, Javier Del Ser, Raja Chatila, and Natalia Diaz-Rodriee Rodriguez. Greybox xai: A neural-symbolic learning framework to produce interpretable predictions for image classification. Boonleddg-BBssed Systems, 258:109947, 2022.
Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio Calmon, and Himabindu Lakkaraju. Interpreting clip with sparse linear concept embeddings (splice). Advances in Neural Information Processing Systems, 37: 84298 84328, 2024.
Yequan Bic, Luyang Luo, Zhixuan Chen, and Hao Chen. Xcoop: Explainable prompt learning for computer- aided diagnosis via concept-guided context oplimization. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 783-783. Springer, 2024.
Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller, and Wojeiech Samek. Layer-wise relevance propagation for neural networks with local renormalization layers. In International Conference Artificial Newral Networks, pp. 63 71, 2016.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Abeba. Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: misogyny, pornog raphy, and malignant stereolypes. arxiv preprint 0rXiv:2110009900 2021.
Hila Chefer, Shir Gur, and Lior Wolf. Generic attention-model explainability for interpreting bi-modal and encoder-decoder transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 397 406, 2021.
Chen Chen, Bowen Zhang, Liangliang Cao, Jiguang Shen, Tom Gunter, Albin Madappally Jose, Alexan- der Toshev, Jonathon Shlens, Ruoming Pang, and Yinfei Yang. Stair: Learning sparse text and image representation in grounded tokens. arXiv preprint (rXf::30111108// 2023.
Mia Chiquier, Utkarsh Mall, and Carl Vondrick. Evolving interpretable visual classifiers with large language models. In European Conference on Computer Vision, pp. 183-201. Springer, 2024.
Jihye Choi, Jayaram Raghuram, Yixuan Li, Suman Banerjee, and Somesh Jha. Adaptive concept bottleneck for foundation models. In ICML 2024 Workshop on Foundation Models in the Wild, 2024.
Yan Cui, Shuhong Liu, Liuzhuozheng Li, and Zhiyuan Yuan. Ceir: Concept-based explainable image repre- sentation learning. orXiv preprint arXiv:2312. 10747, 2023.
Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In The Twelfth International Conference Learning Representations, 2024.
Nicola. Debole, Pietro Barbiero, Francesco Giannini, Andrea Passerini, Stefano Teso, and Emanuele Mar- conato. If concept bottlenecks are the question, are foundation models the answer? arXio preprint arXiv:250..19774 2025.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pottern Recognition, pp. 248 255, 2009.
Natalia Díaz- Rodriiozr,, Alberto Lamas, Jules Sanchez, Gianni Franchi, Ivan Donadello, Siham Tabik, David Filliat, Policarpo Cruz, Rosana. Montes, and Francisco Herrera. Explainable neural-symbolic learning (x- nesyl) methodology fuse deep learning representations with expert knowledge graphs: The monumai cultural heritage use case. Information Fusion, 79:58-88 83, 2022.
Rachel Lea Draelos and Lawrence Carin. Use hirescam instead of grad-cam for faithful explanations of convolutional neural networks. arXie preprint (xXi::2011.08891 2020.
Jessica Echterhoff, An Yan, Kyungtae Han, Amr Abdelraouf, Rohit Gupla, and Julian McAuley. Driving through the concept gridlock: Unraveling explainability bottlenecks in automated driving. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 7346 7355, 2024.
Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, et al. A mathematical framework for transformer circuits. Trans- former Circuits Thread, 111):12, 2021.
Javier Fumanall-ddocn, Javier Andreu-Perees Oscar Cordón, Hani Hagras, and Humberto Bustince. Artxai: Explainable artificial intelligence curates deep representation learning for artistic images using fuzzy tech- niques. IEEE Transactions on Fuzzy Systems, 32(4):1915-1926, 2023.
Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip's image representation via text- based decomposition. arXiv preprint arXiv:2310.05916 2023.
Walter Gerych, Haoran Zhang, Kimia Hamidieh, Eileen Pan, Maanas K Sharma, Tom Hartvigsen, and Marzych Ghassemi. Bendvim: Test-time debiasing of vision-language embeddings. Advances in Neural Information Processing Systems, 37:62480-62518 2024.
Mclissa Hall, Laura Gustafson, Aaron Adcock, Ishan Misra, and Candace Ross. Vision-language models performing zero-shot tasks exhibit gender-based disparities. Xiv preprint arXio:2301.11 11100, 2023.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Lijie Hu, Songning Lai, Wenshuo Chen, Hongru Xiao, Hongbin Lin, Lu Yu, Jingfeng Zhang, and Di Wang. Towards multi-dimensional explanation alignment for medical classification. Advances in Neural Informa- tion Processing Systems, 37:129640 -129674, 2024.
Xinyi Huang and Long-Kai Huang. Model editing for clip with unknown spurious correlations in visual encoder. openreciew, 2024.
Neha Kalibhat, Shweta Bhardwaj, C. Bayan Bruss, Hamed Firooz, Maziar Sanjabi, and Soheil Feizi. Identi- fying interpretable subspaces in image representations. In International Conference on Machine Learning. pp. 15623 15638, 2023.
Rémi Kazmierczak, Eloïse Berthier, Goran Frehse, and Gianni Franchi. CLIP-QDA: An explainable concept bottleneck model. Transactions on Machine Learning Research Journal, 2024.
Rémi Kazmierczak, Eloise Berthier, Goran Frehse, and Gianni Franchi. Explainability and vision foundation models: A survey. Information Fusion, 122:103184, 2025.
Injae Kim, Jongha Kim, Joonmyung Choi, and Hyunwoo J Kim. Concept bottleneck with visual con- cept filtering for explainable medical image classification. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 225-233 Springer, 2023.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning, pp. 5338-5338 5348, 2020.
Alberto Lamas, Siham Tabik, Policarpo Cruz, Rosana Montes, Álvaro Martine--Sevilla, Teresa Cruz, and Francisco Herrera. Monumai: Dataset, deep learning pipeline and citizen science based app for monumental heritage taxonomy and classification. Neurocomputing. 420:266-280, 280, 2021.
Xiao Li, Yining Liu, Na Dong, Sitian Qin, and Xiaolin Hu. Partimagenet++ dataset: Scaling part-based models for robust recognition. In Europeun Conference on Compuler Vision, pp. 396-414, 2024.
Jiaxiang Liu, Tianxiang Hu, Yan Zhang, Xiaotang Gai, Yang Feng, and Zuozhu Liu. A chatgpt aided explainable framework for zero-shot medical image diagnosis. arXie preprint arXiv:2307.0198) 2023.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, pp. 38-55. Springer, 2024a.
Yufang Liu, Tao Ji, Changzhi Sun, Yuanbin Wu, and Aimin Zhon. Investigating and mitigating object hallucinations in pretrained vision-language (clip) models. arXiv preprint ar Xiv:2410.03116 2024b.
Andrew Lu, Wentinn Liao, Liuhui Wang, Huzheng Yang, and Jianbo Shi. Artifacts and attention sinks: Structured approximations for efficient vision transformers. arXiv preprint arXiv:2507.16018, 2025.
Scott M Lundberg and Su-In Lee. A unified approach interpreting model predictions. Advances in newral information processing systems, 30, 2017.
Mazda Moayeri, Keivan Rezaci, Maziar Sanjabi, and Soheil Feizi. Text-to-concept (and back) via cross model alignment. In International Conference on Machine Learning, pp. 25037-25060. PMI.R, 2023a.
Mazda Moayeri, Wenxiao Wang, Sahil Singla, and Soheil Feizi. Spuriosity rankings: sorting data to measure and mitigate biases. Advances in Neural Information Processing Systems, 36:41572-41600, 2023b.
Thomas Norrenbrock, Marco Rudolph, and Bodo Rosenhahn. Q-senn: Quantized self-explaining neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 21482 21491, 2024.
Hongscok Oh and Wonseok Hwang. Vision-encoders (already) know what they see: Mitigating object hallucination via simple fine-grained elipscore. arXiv preprint arXiv:2502 20034, 2025.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui- Wei Weng. Label-free concept bottleneck models. arXiv preprint arXiv:23040061 2023.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11):e7, 2017.
Konstantinos P Panousis, Dino lenco, and Diego Marcos. Coarse-to- concept bottleneck models. Advances in Neural Information Processing Systems, 37:105171-105199, 2024.
Konstantinos Panagiotis Panousis, Dino lenco, and Diego Marcos. Sparse linear concept discovery models. In Proceedings of the icee/cut international conference on computer vision, pp. 2767-2771, 2023.
Cristiano Patricio, Luis Teixcira, and João C Neves. Towards concept-based interpretability of skin lesion diagnosis using vision-language models. In 2024 IEEE internalional symposium on biomedical imaging (ISBI), pp. 1-5. IEEE, 2024.
Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint 1rXiv:1806.0711 07421, 2018.
Zhiyuan Ren, Yiyang Su, and Xiaoming Liu. Chatgpt-powered hierarchical comparisons for image classifi- cation. Advances in neural information processing systems, 36:69706 69718, 2023.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135 1144, 2016.
Simon Schrodi, David T Hoffmann, Max Argus, Volker Fischer, and Thomas Brox. Two effects, trigger: On the modality gap, object bias, and information imbalance in contrastive vision-language models. arXiv preprint ar Xix::2204. 07983, 2024.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishha Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 618-626, 2017.
Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Jacob An- dreas, and Antonio Torralba. A multimodal automated interpretability agent. In Forty-first International Conference on Machine Learning, 2024.
Suraj Srinivas and François Fleuret. Full-gradient representation for neural network visualization. Advances in neural information processing systems, 32, 2019.
Divyansh Srivastava, Ge Yan, and Lily Weng. Vlg-cbm: Training concept bottleneck models with vision-
language guidance. Advances in Neural Information Processing Systems, 37:79057-79001 -79044, 2024.
Abhinav Kumar Thakur, Filip Nliesski, Hông-An Sandlin, Zhivar Sourati, Luca Luceri, Riccardo Tom- masini, and Alain Mermoud. Multimodal and explainable internet meme classification. arXiv preprint arXiv:2212.05612, 2022.
Moritz Vandenhirzz, Sonia Laguna, Ričards Marcinkevičs, and Julia Vogt. Stochastic concept bottlenock models. Advances in Neural Information Processing Systems, 37:51787-5i 51810, 2024.
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self- attention: Specialized heads do the heavy lifting, the rest can be pruned. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5797-5808, 2019.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. Caltech-uesd 20002-200-2011. Technical Report COSS-RR-0011000 California Institute of Technology, 2011.
Shijie Wang. Qi Zhao, Minh Quan Do, Nakul Agarwal, Kwonjoon Lee, and Chen Sun. Vamos: Versatile action models for video understanding. Tn Furopean Conference on Computer Vision, pp. Springer, 2024.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Yifan Wu, Yang Liu, Yue Yang, Michael S Yao, Wenli Yang, Xuchmi Shi, Lihong Yang, Dongjun Li, Yueming Läu, Shiyi Yin, et al. A concept-based interpretable model for the diagnosis of choroid neoplasias using multimodal data. Nature communications, 16(1),3504, 2025.
Jiaqi Xu, Cuiling Lan, Wenxuan Xie, Xuejin Chen, and Yan Lu. Relrieval-based video language model for efficient long video question answering. arXiv preprint orXiv:2312.04 2023.
An Yan, Yu Wang, Yiwu Zhong, Chengyu Dong, Zexue He, Yujie Lu, William Yang Wang, Jingbo Shang, and Julian McAnley. Learning concise and descriptive attributes for visual recognition. In Proceedings of the TEEE/CW International Conference Computer Vision, pp. 3090-3100, 2023a.
An Yan, Yu Wang, Yiwu Zhong, Zexue Hc, Petros Karypis, Zihan Wang, Chengyu Dong, Amilcare Gentili, Chun-Nan Hsu, Jingbo Shang, al. Robust and interpretable medical image classifiers via concept bottleneck models. arXiv preprint arxiu:2310. 03182, 2023b.
Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEFE/CVF conference on computer vision and pattern recognition, pp. 19187-19197, 2023.
Wei Jie Yeo, Rui Mao, Moloud Abdar, Erik Cambria, and Ranjan Satapathy, Debiasing clip: Interpreting and correcting bias in attention heads. arXiv preprint arXiv:2505.17422 2025.
Yow, Neha Priyadarshini Garg, Manoj Ramanathan, Wei Tech Ang, et al. Extract explainable trajectory corrections from language imputs using textual description of features. arXiv preprint arXiv:2401.081 000.00701, 2024.
Hanwei Zhang, Luo Cheng, Rui Wen, Yang Zhang, Lijun Zhang, and Holger Hermanns. S1-cbm: En- hancing concept. bottleneck models with semantic locality for better interpretability. arXiv preprinl arXiv:2601.124 12804, 2026.
Miao Zhang, Ben Colman, Ali Shahriyari, Gaurav Bharaj, et al. Common-sense bias discovery and mitigation for classification tasks. ai preprint 2r11:2201.132111 2024.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2921 2929, 2016.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwci Liu. Conditional prompt lcarning for vision- language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 16816-16825, 2022.
A Use of CLIP in CBMs
Table
presents the foundation models used in the CBMs highlighted in the study of Kazmierezak et al. Note that the list is not exhaustive.
2025
B Datasets
ImageNet The first dataset we use ImageNet (Deng et al. 2009), that provides annotation of images into 1000 classes. The datasct having not concept-level annotations natively, we used the PartImagoNett dataset (Li et al. 2024), which provides semantic segmentation annotations for different images of ImageNet. The scenarios used in study are detailed in Table 0
17

---

Published in Transactions on Machine Learning Research (01/2026)
Title
PFM used
STAIR (Che el al.
2023
CLIP
Chat GPT XAI Liu et al. 2023
CLIP
ARTxAI (Fumanal-
2023
CLIP
Explanable meme classilication
Thakur et al.
2022
CLIP
Label free CBM (Oikarinen et al.
2023
CLIP
LaBo Yangg et al. 2023
CLIP
Learning Concise Yan et al. 2023a
CLIP
Sparse CBM (Panousis et al. 2023
CLIP
CBM with filtering (Kim et a.. 2023
CLIP
Robust CBM Yan et al. 20236
CLIP
Hierarchichal CBM Pannuuss al.
2024
CLIP
ChatGPT CBM (Ren et al.
2023
CLIP
Skin lesion CBM
Patricio et al.
2024
CLIP
R-VLM Xu et al.
2023
CLIP
CEIR CCui et al.
2023)
CLIP
Concopt Gridlock Eccherhoff et al.
2024
CLIP
SpLiCE Bhalla al.
2024
CLIP
MMCBM (wu et al.
2025
CLIP
XCoOp Bie et al.
2024
CLIP
CLIP-QDA ADA-QQAA Kazmierczaa al.
2024
CLIP
Text-To-Ee Co-Concppt (Moayeri et al.
2023a
CLIP
LLM-Mutate (Chiquier et al. 2024
Llama2+CLIP
VAMOS Wang et al. 2024
BLIP-2
Q-SENN Norrenbrock
a
2024
CLIP
ExTraCT Yow et al.
2024
CLIP+BERT
Adaptative CBM [Choi al.
2024
CLIP
Stochastic CBM (Vandenhirzz et al.
2024
CLIP
Med-MICN Hu et al.] 2024
CLIP
VLG-CBM Srivastava et al.
2024
CLIP
Table 5: PFM usage in CBMs.
Scenario
C1
C2
Tiger
Gondola
tail
2
Bolete
Stole
lamellae
3
LesserPanda
BlackSwan
paw
4
ModelT
Turnstile
wheel
5
Plunger
Commonlguana
handle
6
AnalogClock
Goldfish
dial
7
Fly
Strawberry
wing
8
Barracouta
Barbell
fin
9
CompulerKeyboard
Convertible
key
10
FountainPen
HowlerMonkey ink _catridge
Table 6: List of ImageNet runs with respective triplets classes C₁, C22 and concepts k.
Monumai Monumai (Lamas et al. 2021 a specialized dataset containing images of monuments. is composed of 908 images. Each image is annotated accordingly the overall structure that corresponds the class, and the architectural features that corresponds the concepts. There are 15 concepts and 4 classes available. The scenarios used in study detailed in Table
18

---

Published in Transactions on Machine Learning Research (01/2026)
Sconario
C2
hispanic-muslin
baroque
lobed arch
baroque
renaissance
porthole
baroque
gothic
broken podiment
broken_podiment
4
baroque
renaissance
solomonic column
5
gothic
hispanic-musiing
pointed arch serliana
6
renaissance
baroque
7
gothic
baroque
trefoil_arch
8
baroque
renaissance
rounded arch ogee_an arch
9
gothic
renaissance
Table 7: List of Monumai runs with respective triplets classes C2 and concepts k.
CUB CUB Wah et al. 2011 is a dataset dedicated to the classification of birds, with 200 classes cor- responding to species. Concept level, localised annotations are also not available natively. To obtained such annotation, we used the procedure of VLG-CBM (Sivastava et al. 2024) that uses Grounding Liu et al. 2024a) to localize concepts as bounding boxes. The scenarios used in our study are detailed in Table
Scenario
C1
Orchard Oriole
Least Auklet Bay breasted Warbler
long tail long pointed beak yellowish belly brown grayish body brown streaks on the chest black wingtips white body with gray wings white front white belly
Red headed Woodpecker
Worm eating Warbler
Chuck will Widow
Whip poor Will
Rock Wren
House Sparrow
Belted Kingfisher
Herring Gull
Worm eating Warbler
Ring billed Gull
Red breasted Merganser
Red bellied Woodpecker Golden winged Warbler
Red breasted Merganser
Geococcyx
10
Pied Kingfisher
Vermilion Flycatcher
black back
Table List of CUB runs with respective triplets classes cı, C22 and concepts k.
Ablation studies
C.1
Comparative performance analysis
Table g
presents results for CHILI, presented in the format of Table 1 comparability, all values have been normalized
are reported below. To ensure
Dataset
Base
= c1; in (present)
c c1; k not in cl (absent)
Failure rate
ImageNet
Baseline
0.853 (0.005)
0.846 (0.005)
0.4
Chili
0.713 (0.035)
0.668 (0.040)
0.2
MonumAI
Baseline
0.828 (0.025)
0.768 (0.021)
0.4
Chili
0.680 (0.035)
0.712 (0.039)
0.3
CUB
Baseline
0.779 (0.029)
0.794 (0.030)
0.5
Chili
0.507 (0.070)
0.548 (0.082)
0.3
Table 9: Score comparison between original CLIP-scores and CHILI Normalized values. Variances in parentheses.
19

---

Published in Transactions on Machine Learning Research (01/2026)
The values in parentheses represent standard variances. Consistent with the findings in Section 5 results indicate that CHILI yields modest yet measurable improvement, reflected in both the relative score reduction and failure rate decrease.
C.2 Artifact Mitigation Methods
To address pseudo-register artifacts, we systematically evaluated multiple approaches: erosion erosion/dilation /diaaiion, Prin- cipal Component Analysis (PCA), thresholding, and median filtering. Our decision to adopt median filtering was informed by an ablation study, detailed Table 10 assessing their impact on object segmentation per-
formance.
Method
pixAcc
mIoU
mAP
Bascline
0.7678
0.5714
0.8289
+ FrosionnDiilation
0.7474
0.4795
0.7070
+ PCA
0.6627
0.4503
0.7525
Thresholding
0.7700
0.5775
0.8295
Median Filtering (Ours)
0.7827
0.5945
0.8460
Table 10: Performance comparison of pseudo-register artifacts mitigation methods
Median filtering and thresholding emerged the most effective methods, likely duc the high-norm, localized nature of the artifacts. We ultimately selected median filtering for its superior performance and avoid introducing additional hyperparameter (as required by thresholding).
C.3 Individual Components
We propose here to measure the apport of pseudo-register filtering of the weighting scheme by isolating them individually. Table I1 echoos the segmentation experiment of Table while Table 12 echoes the segmentation experiment of Table
Filter Register
Weighting
pixAcc
mloU
mAP
FALSE TRUE
FALSE
0.7678
0.5714
0.8289
FALSE
0.7827
0.5945
0.8460
FALSE
TRUE
0.7735
0.5827
0.8333
TRUE
TRUE
0.7879
0.6022
0.8486
Table ImageNet class segmentation performance comparison with different configurations
Filter Register
Weighting
AUROC
FALSE
FALSE
0.495
TRUE
FALSE
0.566
FALSE
TRUE
0.570
TRUE
TRUE 0.596
Table 12: ImageNet binary classification performance comparison with different configurations
In both cases, each component individually improves performance, with the combined approach yielding the best results.
C.4 Temperature parameter
We analyse here the sensitivity of our method the hyperparameter a, which controls the weighting scheme, in Table 13
20

---

Published in Transactions on Machine Learning Research (01/2026)
AUROC
0.586
0.582
10
0.596
50
0.587
100
0.587
Table 13: AUROC values for different Temperature parameter a.
The optimal value of 10 reflects trade-off: approaches 0, weights become insensitive to IoU, while as approaches infinity, weights approximate an indicator function (ToU > 0).
C.5
Probing dataset
In our experiments, the probing dataset distinct for each dataset. To further explore the robustness of our approach, we investigated the impact applying the weights derived from the ImageNet dataset to perform disentanglement on images from another dataset (MonuMAT) and vice versa. The results of this cross-datasel evaluation are presented in the table below.
Probing dataset
Test dataset
Auroc
Imagnet
Monumai
0.557 (0.015)
Mommai
Monumai
0.565 (0.015)
Monumai
Imagnet
0.589 (0.016)
Imagnet
Imagnet
0.596 (0.017)
Table 14: Auroc values for different probing and test dataset combinations.
While we observe expected decrease in aatoommmncc--ttibuaaabl to the fact that the probing data originates from a different distribution than the inference set this decline is modest. We hypothesize that this robustness stems from similarities in image content across datasets. This suggests that the learned representations retain degree of gencralizability despite differences in annotation gramularity.
Computational costs
All the experiments were conducted on a single V100 GPU with 32GB of RAM. While CHILI introduces additional computational overhead, the memory cost is negligible since the only additional information is the weight matrix. The architecture remains the same. Concerning inference time, illustration, currently average the following times:
Method
Iters / second
Baseline 51.64
CHIT.
1.13
Table 15: Inference time comparison in iterations per second.
While there is significant increase of inference time, we consider reasonable since the architecture allows to precompute CL.P scores in order to train the CBM.
21

---

Published in Transactions on Machine Learning Research (01/2026)
E
Additional samples
In addition of the explanation samples provided in the main paper, we provide here additional samples. The goal is to provide a more global insight, including failures cases of CHILI. These examples correspond to Figures to 12
A recurrent issue is the tendency of heatmaps to overfocus on specific regions of the imago-such as the heads of birds in Figures or t
resulting in the mislocalization of certain features. We hypothesize that this behavior stems from the use of uniform weighting scheme across all concepts. We also observed that our method faces challenges with shape-related concepts, like the concept large, hawk-lik like body on Figure g
or long, thin antenae in Figure 10
long, stout niby-red thmmat
back
brown the sides
feathers
256
long, stout beak
+8.54
ruay
with
streaks
brown and white feathers
1681 other features
-84.82
-90
-80
-70
-60
-50
-40
-30
-20
-10
-60042
Figure 4: Example of explanation produced by the intervention of CHILI in CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: Brown Creeper.
22

---

Published in Transactions on Machine Learning Research (01/2026)
irideccnn blue green back
dark blue-black upper body a purp.e-red need and breast
small, biuu-green body
yellowish--reee sereeisss-ggennnnee legs
462
ridescent blue-green back
dark blue black upper body
+6.37
purple red head and breast
+0.18
small, blue- body
+7.12
yellowish-greet
+5.07
features
+13.55
Figure Example of explanation produced by the intervention of CHILI in a CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: Cape Glossy Starling.
brown reddish
black band across the breast
large, sparrow-lite bird
pala brown upperparts
4660003
dark coloration
+9.53
reddish cap
+7.36
across breast
large, sparrow-ike like bird
pale brown upperparts
1651 other features
-61.55
-60
-50
-40
-30
-20
-10
4[AX)
-6855
Figure Example of explanation produced by the intervention of CHILI in CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: House Sparrow.
23

---

Published in Transactions on Machine Learning Research (01/2026)
thin antenrae
black ring around bill
brown streak cown its back
shart, brown legs
long, straight beald
-75300
long, thin antennae
black ring around the bill
+7.85
brown streak down its back
-7.5
short, brown logs
+7 19.19 moneene
long, straight beal?
1681 other
-90.89
-100
[XXI]
Figure Example of explanation produced by the intervention of CHILI in a CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: House Wren.
purple red head and breast
red belly
spotted conical
purple red head and breast
+13.55
head
+10.95
red belly
110.83
spotted feathors
49:77
1681 features
EUAX) -0.33
Figure 8: Example of explanation produced by the intervention of CHILI in CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: Red Bellied Woodpecker.
24

---

Published in Transactions on Machine Learning Research (01/2026)
swift, acrobatic flight
large, hawk-likke body
grayish white body
white tail
swift, direct nlght
207.05
swift, acrobatic flight
+11.09
large, body
+30.14
grayish-white hody
+9.43
white tail
+8.02
swift, direct flight
1681 other features
+165.59
150
EIAXI]
Figure 9: Example of explanation produced by the intervention of CHILI in CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: Red Legged Kittiwake.
long tail white tip
long black tip
wingspan about
"usty- brown breast
long,
Kxl 26-255..3 915
long
+10.72
long
wingspan of ebout 3 feet
rusty brown breast band
long, thin antannaa
+R
1681 other features
+231.78
100
150
200
250
[[11
-4.401
Figure 10: Example of explanation produced by the intervention of CHILI in CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: Scissor Tailed Flycatcher.
25

---

Published in Transactions on Machine Learning Research (01/2026)
bright green
blue back
white collar around the reck green tail with white edge
black stripe on the head
=0:49.99
bright green back
blue back
+7.51
white collar around the neck
+7.25
tail with white edge
+6.54
black the head
-5.67
[RX)
Figure 11: Example of explanation produced by the intervention of CHILI CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: Tree Swallow.
greenish-ylllow body
yellowish bill
ruby-red throat
a yellowish belly
yallowish eyebrew
4x) 1.44.85
greenish-yellow body
yallowish bill
ruby red throat
yellowish eyebrow
+7.7
yellowich belly
-7.65
1691 other features
+125.68
Figure 12: Example of explanation produced by the intervention of CHILI in CBM. On the bottom left, the input image. On the bottom right, the SHAP values. Target label: Worm Eating Warbler.
26


Reviews:
[{"review_id": "qj18RnStpK", "reviewer": "Reviewer_5ZdL", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents CHILI (Concept Hallucination Inhibition via Localized Interpretability), a method for enhancing concept localization in CLIP-based Concept Bottleneck Models (CBMs). The core idea is to disentangle CLIP’s activations into object-related, context-related, and pseudo-register components, aiming to mitigate concept hallucination and improve interpretability without fine-tuning. Experiments on ImageNet, MonumAI, and CUB datasets demonstrate consistent improvements in localization and segmentation performance, with minimal loss in classification accuracy when integrated into CBMs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors provided good technical details introduction and comprehensive experimental analysis. Specifically:\n\n1. The problem of concept hallucination in CLIP-based CBMs is clearly motivated and relevant for improving the faithfulness of explainable AI.\n2. The method is analytically grounded, decomposing CLIP activations at the layer/head level and proposing a principled separation of object and context signals.\n3. Empirical evaluation is extensive across multiple datasets and tasks, showing consistent interpretability gains.", "audience_yn": "Yes", "audience_explanation": "The paper’s findings address a current and significant gap in explainable AI for vision-language models, a topic that aligns directly with TMLR’s readership. Several reasons support this:\n\n1. Relevance to Concept Bottleneck Models (CBMs): CBMs are a central theme in interpretable learning, and the paper extends them to the CLIP setting, which is widely studied. TMLR readers include researchers in interpretability, representation learning, and multimodal AI.\n\n2. Methodological contribution (CHILI): The proposed disentanglement of CLIP activations into object, context, and pseudo-register components provides a new direction for mitigating hallucination without retraining. Such post-hoc interpretability approaches are of particular interest to TMLR readers focusing on practical, model-agnostic explainability.\n\n3. Empirical insight: The quantitative evaluation across ImageNet, MonumAI, and CUB datasets gives concrete evidence of concept hallucination and its mitigation, which are empirical findings that would interest both application-driven and theoretical researchers.", "requested_changes": "1. The claim regarding pseudo-register tokens (page 7, Section 4.2) lacks analytical or empirical justification. The paper states that “these tokens act like pseudo-registers: they tend to store global information, but their spatial localization on the activation map is not meaningful for interpretability,” and that this component is separated accordingly. However, no quantitative or qualitative evidence is provided to validate this behavior in the CLIP setting. Since this assumption is critical to the decomposition, the authors should provide supporting analysis or visualization demonstrating the behavior of these tokens.\n\n2. Minor typographical and formatting issues should be corrected. For example,  in Table 4, inconsistent commas and decimal separators appear in numerical values (e.g., “73,55” instead of “73.55”). These errors reduce polish and readability.\n\n3. The qualitative results (Figures 3) only present positive examples where CHILI explanations align well with human perception. For a more convincing interpretability assessment, the paper should also include negative samples, cases where the predicted concept or heatmap has little or no visual correspondence with the image. Showing such counterexamples would clarify the method’s failure modes, illustrate the limits of disentanglement, and help readers assess robustness and interpretive reliability under less ideal conditions.", "broader_impact_concerns": "No.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KxBe3HOYmc", "reviewer": "Reviewer_CTGW", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses concept hallucination in CLIP-based Concept Bottleneck Models (CBMs), where CLIP incorrectly predicts concept presence based on contextual cues rather than actual visual presence. The authors propose CHILI (Concept Hallucination Inhibition via Localized Interpretability), a post-hoc method that disentangles CLIP activation maps into object-related, context-related, and pseudo-register components without requiring model fine-tuning.\n\n**Key Strengths:**\n- HIghlights an important problem in CLIP-based CBMs through statistical analysis across three datasets (ImageNet, MonumAI, CUB)\n- Proposes a principled decomposition method.\n- Shows improvements in localization tasks (mIoU, mAP)\n- Works as a post-hoc method without retraining, making it practical for existing systems\n\n**Key Weaknesses:**\n- The performance gains over different baselines are very marginal and may not be statistically significant.\n- Table 1 presents a statistical analysis comparing results across three datasets to demonstrate the hallucination problem. However, this same style of analysis is not repeated after introducing CHILI to verify whether the proposed method actually resolves the identified issues.\n- The paper provides no justification or evidence for why the median filter is the appropriate choice for removing pseudo-register artifacts, or why this operation specifically removes non-spatial global information.\n- The paper suffers from clarity and organization issues. Baselines are not denoted in a standard way, sections could be better structured, and multiple readability problems exist throughout.", "claims_and_evidence_yn": "No", "claims_explanation": "Several claims need stronger support:\n\n- While Table 1 documents the hallucination problem with 40-50% failure rates, Table 2 shows CHILI only achieves AUROC scores of 0.53-0.60, which suggest concept detection with barely above chance probability. This suggests the method only partially resolves the issue, yet the paper doesn't adequately explain this limitation.\n\n- The segmentation results in Table 3 show modest improvements (2-3 points in mIoU), but it's unclear how these metrics directly measure concept hallucination reduction versus general localization ability.\n\n- The claim that CHILI produces \"more interpretable CBMs\" lacks quantitative support. Section 6.2 provides only qualitative visual examples without user studies or established faithfulness metrics.\n\n- The individual contributions of pseudo-register removal, the weighting scheme, and the hyperparameter alpha are not isolated. Without this, it's difficult to assess which design choices matter.\n\n- No statistical significance tests are provided for any results. Given the small effect sizes in some experiments, it's unclear whether improvements are meaningful or within noise margins.\n\n- The performance improvement claims seem exaggerated and must be toned down.", "audience_yn": "Yes", "audience_explanation": "The paper addresses relevant topics for the machine learning community. The problem is timely and important. Concept Bottleneck Models and vision-language foundation models are active research areas, and understanding their failure modes matters for building trustworthy systems. The finding that CLIP hallucinations undermine CBM faithfulness has practical implications for researchers and practitioners.\n\nThe methodological contribution, builds on existing attention decomposition techniques, hence offering a useful adaptation to the CBM context with a specific focus on localization versus contextual suggestion.\n\nThe experimental design is clear and uses standard datasets, which should facilitate future work in this area.", "requested_changes": "### Critical Changes\n\n- **Add ablation studies** (Section 4.2): Separate experiments are needed to isolate contributions from (a) pseudo-register filtering, (b) the weighting scheme, and (c) the hyperparameter alpha. Without this, readers cannot understand which components drive the results.\n- **Provide statistical significance tests**: Results of table 2 need confidence intervals or significance tests to demonstrate that improvements are not due to chance variation.\n- **Repeat Table 1 style analysis after introducing CHILI**: The statistical analysis format used in Table 1 should be performed on CHILI to verify whether CHILI helps with the hallucination issues identified initially, against the S_context representation and complete representation.\n- **Justify median filter choice**: Provide theoretical or empirical justification for why the median filter is appropriate for removing pseudo-register artifacts and why this operation removes non-spatial global information.\n- **Improve writing and organization**: Use standard notation for baselines throughout the paper, restructure sections for better flow, and address readability issues.\n\n \n### Changes that Would Strengthen the Work\n\n- **User study for interpretability**: Human evaluation would validate whether CHILI-based explanations are actually more interpretable.\n- **Computational cost analysis**: Report time and memory overhead compared to standard CLIP-based CBMs.", "broader_impact_concerns": "The paper does not include a Broader Impact Statement. While the work on improving interpretability is valuable, the authors should briefly address its limitations in high-stakes applications. Since CHILI only partially mitigates concept hallucination, practitioners should understand the remaining limitations when deploying these methods in sensitive domains like healthcare or legal systems.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "EnyyRMmXng", "reviewer": "Reviewer_LVKt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents CHILI (Concept Hallucination Inhibition via Localized Interpretability), a novel, post-hoc method designed to improve the faithfulness and localization of concepts within Concept Bottleneck Models (CBMs) that utilize CLIP for zero-shot concept extraction .\nThe core contribution is demonstrating and mitigating concept hallucination in CLIP , a phenomenon where high similarity scores reflect contextual or semantic cues instead of the concept's actual physical presence. CHILI addresses this by decomposing the CLIP embedding and score into three distinct, interpretable components: the Object component (linked to physical location), the Context component (linked to features suggesting presence but not directly representing the concept), and a Pseudo-Register component (high-norm artifacts) . This decomposition is achieved by calibrating the contribution of each attention head and layer using an IoU-based metric derived from a dedicated probing dataset containing ground truth segmentations . The resulting Object score then serves as the faithful, localized conceptual representation in the CBM.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims are supported by strong, quantitative evidence. First, the statistical test in Section 3.2 clearly justifies the problem by showing that the raw CLIP score fails to separate concept presence from semantic association, with up to 50% failure rates in tested scenarios. Second, the efficacy of the proposed disentanglement is proven by the superior performance of the Object component in binary concept detection (Table 2), where it significantly outperforms the baseline CLIP score and concurrent methods. Third, the localization capability is confirmed by the segmentation results (Table 3), with the Object activation map improving mIoU by +3.1 points over the raw attention baseline. Finally, the trade-off is validated by showing that applying CHILI to CBMs results in only a minor, acceptable loss in predictive accuracy.", "audience_yn": "Yes", "audience_explanation": "The paper addresses critical issues highly relevant to the TMLR audience, spanning XAI, interpretability, and Vision-Language Models (VLMs). The mitigation of concept hallucination enhances the trustworthiness of CBMs, a key model family for interpretable AI. The analysis of CLIP's internal attention mechanisms and the development of a post-hoc, IoU-calibrated decomposition method provides a valuable analytical tool for VLM researchers. The findings are directly applicable to building more faithful AI systems in sensitive domains.", "requested_changes": "The submission would be significantly strengthened by addressing a key area regarding the calibration mechanism and evidence. Discuss or investigate the generalizability of the learned head weights. Reporting results for weights calibrated on one dataset (e.g., CUB) but applied to another (e.g., ImageNet) would demonstrate if the object/context separation is a general property of specific attention heads. Furthermore, to provide essential context for this analysis, the Abstract and Introduction must clearly state that the CHILI calibration step requires a probing dataset with ground truth segmentation masks for concept localization, as this data dependency affects the zero-shot claim for the overall process.", "broader_impact_concerns": "No Impact Concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)