Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Authors: Mateo Espinosa Zarlenga
Gabriele Dominici
Pietro Barbiero
Zohreh Shams
Mateja Jamnik
Venue: ICML 2025 poster (2025)
Keywords: XAI
Concept-based Models
Concept Bottleneck Models
Concept Interventions
Out-of-distristribution
OOD
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: We show that current concept-based architectures are unable to properly intake concept interventions when inputs are OOD, and we propose a novel architecture to address this.

Abstract: In this paper, we investigate how concept-based models (CMs) respond to out-of-distribution (OOD) inputs. CMs are interpretable neural architectures that first predict a set of high-level *concepts* (e.g., "stripes", "black") and then predict a task label from those concepts. In particular, we study the impact of *concept interventions* (i.e., operations where a human expert corrects a CM’s mispredicted concepts at test time) on CMs' task predictions when inputs are OOD. Our analysis reveals a weakness in current state-of-the-art CMs, which we term *leakage poisoning*, that prevents them from properly improving their accuracy when intervened on for OOD inputs. To address this, we introduce *MixCEM*, a new CM that learns to dynamically exploit leaked information missing from its concepts only when this information is in-distribution. Our results across tasks with and without complete sets of concept annotations demonstrate that MixCEMs outperform strong baselines by significantly improving their accuracy for both in-distribution and OOD samples in the presence and absence of concept interventions.

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Mateo Espinosa Zarlenga
Gabriele Dominici?
Pietro Barbiero Zohreh Shams Mateja Jamnik'
Abstract
high-level units of information known as "concepis" (e.g.. "stripes", "white". and "black" when predicting "zebra"). They achieve this through two-step process: first, they predict concepts from the inputs, forming a bottleneck, and then they use these concept predictions to de- termine the task label. This design enables CBMs to provide human-like explanations for their predictions by grounding them in interpretable concept representations. More im- portantly, these models enable concept interventions (Koh et al., 2020; Chauhan et al., 2022; Sheth et al., 2022; Shin et al., 2023), where an expert interacting with the model at test time can correct mispredicted concepts, leading to sig- nificant improvements in accuracy once the CBM updates its prediction considering such feedback (Figure 1, left).
In this paper, we investigate how concept-based models (CMs) respond to out-of-distribttion (OOD) inputs. CMs are interpretable neural architectures that first predict a set of high-level concepts (e.g., stripes, black) and then predict a task label from those concepts. In partic- ular, we study the impact of concept interventions (i.e., operations where human expert corrects CM's mispredicted concepts at test time) on CMs' task predictions when inputs are OOD, Our analysis reveals a weakness in current state-of- the-art CMs, which we term leakage poisoning. that prevents them from properly improving their accuracy when intervened on for OOD inputs. To address this, we introduce MixCEM, new CM that learns to dynamically exploit leaked information missing from its concepts only when this information is in-distribution. Our results across tasks with and without complete sets of concept annotations demonstrate that MixCEMs outperform strong baselines by significantly improving their accuracy for both in-distribution and OOD samples in the presence and absence of concept interventions.
Recent works have significantly advanced the performance and usability of CMs within challenging in-distribution (ID) test sets by overcoming the incompleteness gap the fact that training concept annotations may be insufficient for ac- curately predicting the downstream task. By exploiting by- pass mechanisms, such as dynamic concept embeddings (Es- pinosa Zarlenga et al., 2022; Kim et al., 2023; Xu et al., 2024) or residual connections (Mahinpei al., 2021; Havasi et al., 2022; Yuksekgonul et al., 2023), state-of-the-arr CMs enable information to "leak" directly from the features to the task predictions, bypassing the concept bottleneck and sig- nificantly increasing the model's task accuracy even when the set of training concept annotations is incomplete.
1. Introduction
In this work, we argue that, although useful, blindly incorpo- rating these bypasses can severely affect how CMs behave for aat-of-distributii suggest that such bypasses can themselves become out-of- distribution for OOD samples, resulting in the "poisoning" of the model's predictions and in concept interventions fail- ing to achieve the intended accuracy improvements (Fig- ure 1, right). Given how interventions can aid a CM in adjusting real-world OOD shifts (e.g., an expert can help CM process noisy yet still-interpretable X-ray scan by in- tervening some concepts), such leakage poisoning casts serious doubts on the real usability of existing CMs.
Recent years have seen surge of interpretable models whose performance is comparable to that of power- ful black-box models such as Deep Neural Networks (DNNs) (Alvarez-Melis & Jaakkola, 2018; Chen et al., 2019; Yuksekgonul et al., 2023). Amongst these, concept- based models (CMs) (Chen et al., 2020; Espinosa Zarlenga et al., 2022), and in particular Concept Bottleneck Models (CBMs) (Koh al., 2020), have paved the way for designing expressive yet interpretable models. CBMs and their variants predict downstream task labels by exploiting
(OOD) samples. Specifically, we
Universiity of Cambridge Universiià della Svizzera Italiana 'IBM Research 'Leap Laboratories Inc.. Correspondence to: Ma- teo Espinosa Zarlenga
To address these limitations, we propose the Mixture of Concept Embeddings Model (MixCEM), a concept-based interpretable model with high generalisation and receptive- ness to interventions across data distributions. MixCEMs
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

achieve this by learning, for each concept, an embedding formed by mixing a global, sample-agnostic embedding and a contextual, residual embedding. By introducing a confidence-based gating mechanism to control the residual embedding's 's contribution, MixCEMs learn to decide when the residual information may be detrimental and, therefore, should be dropped. This allows MixCEMs to exploit the residual component when a bypass is needed (e.g., concept- incomplete setups) while dropping for OOD samples, leading to impactful interventions across data distributions. Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Input
Concept Predictor
Concept Bottleneck
Label Predictor
ID Sample
Conceep--nsoomppette CUB Task
ourr
Eatripos Ojarpe
Took Alcuraiy (56)
lopksupe polsoning
g(x²
f(e)
+
& Okapt Zebra (poeth beaneetter
@vilte
Cbrown
Nonay OOD Sample
Intervention
Concest
80 100
Latervened Concepts (%)
Human Expert
Vanill CE04
"di" CEM
- MitCEMM (pum))
Figure 1. (Left) A concept intervention on CBM triggering prediction update. (Right) Task accuracy as concepts are intervened on in a concept-incomplet task. When intervening on ID samples (solid), enabling models (e.g., CEMs) overcome the "incompleteness gap." However, for OOD samples (dashed), the same models underperform due to "leakage poisoning." MixCEM overcomes the incompleteness gap and leakage poisoning, maintaining high accuracy in both setups.
Crabbé & van der Schaar, 2022; Sheth & Ebrahimi Ka- hou, 2024) or can be discovered (Alvarez-Melis & Jaakkola, 2018: Ghorbani et al., 2019; Yeh et al., 2020; Magister et al., 2022; Espinosa Zarlenga et al., 2023b; Yang et al., 2023; Oikarinen al., 2023), enable these methods to circumvent the unreliability (Kindermans et al., 2017; Adebayo et al., 2018) 8) and lack of semantic alignment (Kim et al., 2018) of traditional XAI importance approaches (Ribeiro et al., 2016; Erhan et al., 2009; Lundberg & Lee, 2017).
Within C-XAI, Concept Bottleneck Models (CBMs) (Koh et al., 2020) provide powerful framework for designing concept- based interpretable DNNs. A CBM (g. f. {=)) is composition of two functions (g. f) supported by scor- ing functions {8: H= all usually parameterised as DNNs. The concept encoder g R2" -> C maps an input x E R" to a "bottleneck" = g(x) E R of k concepts in con- cept space C S R Here, the i-th output of g. é, c= gxx); is designed such that the score pt s;(O;) is maximised when the i-th concept is "active", and minimised other- wise. The label predictor f C - R² maps the bot- tleneck è to distribution over L task labels ý E R... Together, these functions predict a label û = f(((x) for a sample x that can be explained via the concept scores s()) := [s (č,).. s,(C,,)" When m 1 and (ē.) = C, we call this a Vanilla CBM (i.e., Koh et al.'s for- mulation). Vanilla CBMs can be trained jointly (optimising ʃ and y together), sequentially (training y first and then ʃ using g's outputs), or independently (training g and f using ground-truth features and concepts as inputs).
Summary of Contributions Our main contributions are: (1) we provide the first study, to the best of our knowledge, of how concept interventions fare when there are distribu- tion shifts. Our experiments suggest that all bypass-based approaches do not necessarily or significantly improve their OOD task accuracy when intervened on; (2) we introduce the notion of leakage poisoning, a previously unknown con- sequence of information leakage (Mahinpei et al., 2021). Then, we argue that this poisoning is important design consideration given the existence of a trade-off between avoiding leakage poisoming and achieving high accuracies; and (3) we propose MixCEM, a CM that avoids leakage poisoning'. We show that MixCEMs maintain high task and concept accuracies while significantly improving their performance when intervened on, both for OOD and ID samples and even in concept-incomplete training sets.
2. Background, Notation, and Related Work
Concept-based Explainable AI (C-XAI) Concept-based XAI methods explain a black-box model's predictions via high-level units of information, or concepts, that experts would use explain the same task (Bau et al., 2017). Such concepts. which can be provided as training labels (Chen et al., 2020; Kazhdan et al., 2020; Rigotti et al., 2021;
CBM Extensions Recent works have addressed sev- eral limitations of Vanilla CBMs: (1) Concept Embed- ding Models (CEMs) (Espinosa Zarlenga al., 2022) and Intervention CEMs (IntCEMs) (Espinosa Zarlenga et al., 2023a) overcome the aforementioned incomplete- ness gap (Yeh et al., 2020), (2) Probabilistic CBMs (ProbCBMs) (Kim et al., 2023) and tnergy-baseet CBMs (ECBMs) (Xu et al., 2024) enable better uncertainty and
'Our code and experiment configs be found at https//gtthhh.ommmmaccccppposss
2

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
conditional probability estimations, (3) Post-hoc CBMs (P-CBMs) (Yuksekgonul et al., 2023) allow for effec- tive fine-tuning of models into CBMs, and (4) Label-free CBMs (Oikarinen et al., 2023; Yang et al., 2023) exploit vision-language models to extract concept annotations.
That is, if (i) (f(P),g())) {s ( is a CBM learnt from any training distribution Pu. and (ii) C., Y) is a concept-complete distribution (i.e., I(X:Y) S T(C,;Y,, where I(-) is the mutual in- formation), then
z (g(P)) This condition implies that, regardless of the set of concepts used to train the CBM, its downstream task accuracy should remain relatively unchanged. Therefore, a model achieves CA if it can perform equally well (in terms of task fidelity) regardless of the span of its training concept set.
Concept Interventions CBM-based models enable con- cept interventions, where a numan-in-the-lor can correct mispredicted concepts at test time, potentially triggering change in task prediction. Formally, intervention concept C₁ fixes 8; (C:) to its maximum if the expert deter- mines C is active or to its minimum otherwise. Previous works have shown that CBMee models can significantly increase their task accuracy when the corrected concepts are carefully selected via an intervention policy (Shin et al., 2023; Chauhan et al., 2022), or even when they are ran- domly selected (Koh et al., 2020; Espinosa Zarlenga al., 2022; Xu et al., 2024). Recent works have improved the effect of interventions by incorporating intervention-awan losses (Espinosa Zarlenga et al., 2023a), intervention mem- ories (Steinmann et al., 2023), or cross-concept relation- ships (Havasi et al., 2022; Vandenhirtz et al., 2024).
2. Bounded Intervenability (BI): For any test distri- bution Pe and any concept subset S S {1,-.. ,k), when concepts in S are intervened on with values cg. a CBM's task accuracy should be at least as high as the accuracy of a Bayes Classifier in the real data distribution Pa given ey:
E(x.o.u)>PP [(x, S, cs)s 2 ((x.o.))P (P(y [Ply
cs)]
where I(x, cs) is the CBM's task prediction after intervening on concepts S S (1..... .}] using values Cs. Notice that the magnitude of (fx.ew)-p__((((()),,, is equivalent to the expected accuracy of the CBM (g. J, (s))/=)) on a test set sampled from Pie. Intuitively, BI says that, when we are given the ground-truth labels for concepts S, we should strive to perform at least as well as model that only has access to the labels for concepts in S.
OOD Detection This paper studies concept interventions when OOD shifts occur. Hence, our work is related to re- search in OOD generalisation (Sagawa et al., 2019), open set recognition (Scheirer et al., 2012), and anomaly (Zhou & Paffenroth, 2017; Schlegl et al., 2017) and distribution shift (Rabanser et al., 2019) detection. Within concept- based XAI, concepts have been used to explain distribu- tion shifts (Wijaya et al., 2021; Sevyeri et al., 2023; Choi al., 2023; Dreyer et al., 2024), while OOD detectors have been used to detect unwanted leakage in concept representa- tions (Marconato et al., 2022). Rather than explaining shifts or detecting leakage, our work focuses on understanding concept interventions when OOD shifts occur.
Completeness-agnostioen yields task-aceurate CBMs even when their training sets lack all task-relevant concepts, a common scenario considering the difficulty of labelling concepts in both supervised (Collins et al., 2023) and un- supervised (Oikarinen et al., 2023) settings. In contrast, bounded intervenability ensures that a CBM will properly incorporate interventions, even for out-of-distribution in- puts, providing a sensible and computable lower bound for the post-intervention accuracy.
3. Conflicting Objectives in CBMs
CBMs have been traditionally designed with three core objectives in mind: (1) task fidelity (the model should accu- rately predict its task), (2) concept fidelity (the model's explanations should be accurate), and (3) intervenabil- ity (Marcinkevičs et al., 2024) (task fidelity should improve when a model is intervened on). These three properties cap- ture the fact that model trustworthiness, under reasonable definitions of the term (Shen, 2022), cannot rely solely concept and task fidelity without incorporating intervenabil- ity. It is important, however, to place CBMs within the context of real-world datasets, where labelling limitations and distribution shifts are commonplace. Considering this, we argue that CBMs ought to have two additional properties:
Leakage Poisoning State-of-the-arr CBMs have embed- ded incentives within their loss functions and architectures for each core objective. More recently, new architectures have incorporated notions similar to that of CA in their design (e.g.. leakage bypasses). The notion of bounded intervenability, however, remains overlooked in the design and evaluation of CBMs. In this section, we argue that this disregard for BI has led to serious limitation of current CA- CA-supporting models to remain unnoticed.
In Figure (right), we observe that completeness-agnost methods like CEMs indeed overcome the "completeness gap", attaining high task accuracies compared to non- completeness-aware approaches (e.g., Vanilla CBMs) on
1. Completeness-Agnostiic Agnosticism (CA): Task fidelity should be independent of the set of training concepts.
3

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Unintervened
All Intervened
Concept Predictions
points for both ID and OOD samples. This is because inter- ventions on Vanilla CBMs result in global constant changes to their bottlenecks (e.g., setting è; := 1 if c; is "active"), Hence, the more one intervenes, the more the bottleneck will become in-distribution. Second, concept predictions are significantly more uncertain (i.c., have higher entropy) for OOD samples (Figure 2, right). Thus, the uncertainty in P is helpful indicator of sample going OOD, a property also exploited in OOD detection (Hendrycks & Gimpel, 2016).
Counts (%8
0OD
Test OOD
((p.)
Figure 2. CEM concept bottlenecks and predicted concept en- tropies for ID and noisy (OOD) test CUB samples. (Left and centre) t-SNE projections of CEM's bottlenecks before and after all concepts are intervened on. (Right) Distribution of predicted concept entropies for all concepts.
4. Mixture of Concept Embeddings Model
In this section, we build upon our observations above to propose the Mixture of Concept Embeddings Model (Mix- CEM). MixCEM novel CM where interventions can dynamically leak information when the sample is ID, hence enabling completenes--gnoosiiiss, while they are reduced to global constant changes to the bottleneck when the input is OOD, hence avoiding leakage poisoning (Figure 3).
concept-incomplete tasks. However, we also observe that these approaches struggle to properly incorporate interven- tions for OOD inputs, even when the shift is subtle random noise. Surprisingly, we see that when all concepts are inter- vened on, CEM's accuracy is significantly worse for OOD samples than for ID samples, something not observed for Vanilla CBMs. This suggests OOD shifts somehow affect a CEM's s bottleneck even after intervening on all concepts.
Overview Given a training set D = {(x(), cb)), (())y))))) ) with & -eenaangeeeetted or label-free concept annotations, MixCEM learns k pairs of m-dimensional global embed- dings C = ((c(t)
(+) of " such that concept e2 is rep- resented by c when is "active" and cf-) otherwise. These embeddings will be used for concept prediction and for constructing intervenable bottleneck è from which we predict task labels. However, to achieve completeness- agnosticism, MixCEM will Iearn to adjust these embeddings to allow tesk-relevant information missing in the concept annotations с to leak when this information is beneficial.
To understand this, we emphasise that state-of-the-ar CBMs achieve CA by enabling information about y missing in the concepts leak the downstream task predictor f. In practice, this is done using dynamic high-dimensional con- cept embedding representations (e.g., CEMs, ProBCBMs, and ECBMs) or residual side-channels that extend or update the bottleneck after each concept's 's representation has been constructed (c.g., Hybrid CBMs (Mahinpei et al., 2021), autoregressive CBMs (Havasi et al., 2022), and residual P-CBMs). Although useful, the concept bottlenecks yielded by such models form very distinct distributions for ID and OOD samples (Figure 2, left). More importantly, these distributions remain distinct even after intervening on all concepts (Figure 2, centre). This is because, when we en- able information not in C to leak into the bottleneck using high- nigh-dimensional embeddings or residual pathways, this information persists even after an intervention is performed. Hence, when input goes OOD, the leakage path may go OOD itself, becoming detrimental, or "poisonous", for the model's 's ability to intake interventions in general.
Residual Embeddings Given x E R", we introduce a leakage mechanism in our concept embeddings by using a latent code h E R generated from backbone model (x() (e.g., a pre-trained ResNet (He et al., 2016)), to construct pair of residual concept embeddings (x) (x)) for each concept C. We learn these residuals using two linear functions r / (x) == R(+/-) +(x) + b t/-) with learnable weights R (+/-) and biases by ((/-), These residuals will be used to update our global embeddings.
Concept Likelihood From the global and residual embed- dings, we estimate the likelihood p. = =P(c; = 1 x, C) using a linear scoring function sx(x) = (v., (c(t) +
The petrreaabbiit-IInooeeeeeee
Trade-off The ob- served existence of leakage poisoning suggests there is a trade-off between satisfying completeness--agnssiiiss (which requires one to bypass information directly from x to y even after an intervention is performed) and satisfy- ing bounded intervenability (which requires interventions to lead to ID bottlenecks even for OOD samples). Yet, our study reveals two other critical observations that will form the basis of our solution to leakage poisoning: first, the intervention curves for Vanilla CBM converge to the same
ri (+) (x), cf- +r. (x0)7) with weights v, E R 2m shared across concepts. This enables task and concept feedback to influence how we learn our global and residual embeddings.
Contextual Concept Embeddings By mixing each con- cept's global and residual embeddings as we did for fi, we can construct contextual embeddings ,(tt,cc'* E R. that encode both a concept's 's global and sample-specific information. However, we want these contextual embed-
4

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Embedding Construction
Mixing
Bottleneck
Backbone
Label Predictor
Fiel + (1
+x()
(hi
H(R))
(1)
f(i)
14%
H(s)))r #
C
(af
bie + -1 AJee
(h)
H(s))) (b)
Global Fimbedddings (earmable)
Concept Likelihood
Learnable Model
A - ([e]
4 (b)) +of ((0)))
Legend
Entropy- Gating
Embedding Mixing
Figure 3. Given x, a MixCEM predicts concepts P and task labels y. It achieves this by (1) learning global concept embeddings a(+) d E R km and residual embeddings (x)) (x), E R for each training concept c; (2) using these embeddings to estimate P. = P(c, = 1 | x, C') and to construct contextual concept embeddings (c(+), c(-"), (3) mixing contextual embeddings to produce single embedding c. and (4) predicting y from the bottleneck è = [e Cc.""
dings to avoid being poisoned when x is OOD. We achieve this in two ways. First, before mixing these embeddings, we adjust the magnitude of the residual component so that it loses its influence when the sample is likely OOD, We do this by scaling the residuals inversely proportionally to their concept prediction's 's uncertainty, or entropy H (p.): (+/-)- c!-/-1 + (1 H (f:)) rr; ((::))r(+/>((( (+/--(xx).
ing P. to its ground-truth value when computing é; (e.g.. if
concept os is active, then the expert sets p; := 1). This re- sults in è becoming cf if c; is active and c, (-) otherwise.
Training Objective Given task-specific loss Cugk(y.V) (e.g., cross-entropy), we train MixCEM by minimising:
(x).. As the en- tropy H of a Bernoulii r.v. is in (0, 1). and it increases if uncertainty is higher, the scaling factor (1 (fx) controls leakage function of concept uncertainty. Second, after the MixCEM is trained, we use Platt scaling (Platt et al., 1999) to calibrate its concept predictions P to capture better the model's s true uncertainty (see Appendix A for details).
F(x.e.))< [Luxx (Luus (v. f(o(x))) +A,BCE(c, p) +Ap (v. f(c))
c;
As in jointly trained Vanilla CBMs, the first term here is the task accuracy, while the second term is the mean binary cross-entropy between concept labels and predicted scores. The hyperparameter À E B controls how much weight we give correctly predicting concept labels vs task labels.
Intuitively, one can think of c and cf-) as priors repre- senting what we know about the implications of concept e, being "active" or "inactive", respectively. Under this interpretation, the residuals (x) and (x) can be thought of as evidence that will enable us to perform poste- rior updates to these priors after we observe task-relevant information missing from the concept annotations.
The third term, which we call the prior error and scale it by X E IR+ : maximises the task accuracy when only the global embeddings are used. Here, è represents the bottleneck formed by mixing the global concept embeddings using the ground-trut concept labels as coefficients:
c (arcf +(1 + (acc) +(1 (a.c)" +(1
This term maximises the information about the downstream task y encoded in the global concept embeddings. In Ap- pendix B, we prove that MixCEM's 's objective sineectieeeunccce function natu- rally arises as the MLE of a probabilistic graphical model.
Given contextual embeddings (c(+), .) we build a concept bottleneck è from where we estimate the task likelihood Pyy x) using a (linear) label predictor model y f(c). We construct this bottleneck by first building a single concept representation é, for each concept, which then concatenate into single bottleneck vector é : .... èx)"" As in CEMs, we do this by mixing contextual embeddings c, of c using the predicted concept probability f. as mixing coefficient: é == f (+) + (1 p.)c.²² That way, è₁ is closer to cf (+) if e is predicted be active and closer to c otherwise.
Task Likelihood
Non-deterministie Fallbacks We incentivise MixCEMs to be more receptive to interventions in two ways. First, we randomly set (i.e., intervene on) p. to its ground truth value e; with probability Pint during training (we use Pint = 0.25). We follow this procedure as was shown to improve intervenability in CEMs (Espinosa Zarlenga et al., 2022), and discuss its importance in Section 6. Second, to enable MixCEMs to handle dropping arbitrary residual concept embeddings, we zero the residual r (++-) with probability Pdrop E [0, 1] during training. As Dropout (Srivastava
Intervening At test time, can intervene G by forc-
5

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Table 1. Task accuracy and mean concept ROC-AUC reported as mean stds (%) across three sceds. Each task's best result, and those not significantly different from it (paired E-test, p = 0.05), are underlined. Note that, as outlned in Section 5,1, we do not expect MixCEM to attain the best result for these metries, However, we want to remain competitive against other powerful baselines.
Method
CUB
2ww52 Iacomplete
CIFARIC
Celeba
DNN
11.18
/ NIA
71:42.000/ / NIA
29.20
/ NNA
00.33 / NNA
80.79...22 / NNA
25.30
NNA
Vanilla CBM
70.97=076/ 39.
56. 40.46-0.4s
/ 68, 88.15
87.52e0.41
01.42-m0.1
76, 70.28
88.25
76.97
73.99
24.15:0045
80.12
Hybrid CBM
73.65_003
94.53
72.13_002
88.97
94.42.301 88.18
89.39
79.12.00 95.81
79.
73.78
35.40.00.2
87.66
ProbCBM
08.16_1.41
/ 19.15
00.56_1.11
19.26
85.34
/
91.09 .0:25
67.12.011
/ 19107 10.26
61.80.5.1 12
j2.0
31.7410.00
38. 38.14
P-CBM
09:00-0.o1 63.22220..5
48.8% 48.88-10.48
-0.33
85 85.30la0.2
76.77=0.44
85.7840.55
72.16.0.00
17.18+2.47
76 =6.40
Residual P.CBM
71.84_000
63.36.3.u7
20.00
61.70.0..4
90.60
85.39.20as
89.56 89.56.00
85.7740.0
70:90.00.
72.1640.0
15.4022..1
76.492100
CEM
78.67 011 / 89.00
74.42 0.90
69.35
51.07 10.01
/ 94.81.9.4
90.12:omm
80.05 ..ar
73.67 19.67.9.0
34.89 1o.99...4
87,79 Lo.n
InCEM
13.33_0 00
84.344
58
02.30.2.. 45
89.6246.02
84.28821
88 65.65 ae
96.1020.0
78-48.0.0
68.7920 90
36 16.13321 07
MixCEM (ours)
76.54_
18.00
74.54
0.39
17.24.0.15
80.94 L9.34
10.3
88.68
95.19
78.64.0044
72.52
35.58.0.72
/87661.000
et al., 2014), this can be seen as learning an ensemble of models where each model includes only a subset of the residuals. At inference, we adjust for this effect by sampling M bottlenecks (we fix M = 50 in practice) and averaging the prediction made from all M bottlenecks similarly to how Monte Carlo Dropout operates (Gal & Ghahramani, 2016). This mechanism has the added benefit of mitigating overfitting and further preventing poisonous leakage. For a thorough ablation of MixCEM's hyperparameters showing its robustness across values, see Appendix J.
baselines are given the same capacity and budget for fine- tuning and training. We select hyperparameters based on the area under the validation task-accuracy vs intervention curve and describe all hyperparameters and architectures in Appendix D. Finally, for Vanilla CBMs, here we focus their sigmoidal and jointly trained versions. However, we discuss results for different variants (e.g., sequential, independent, and logit CBMs) in Appendix E.
We note that we do not explicitly include Label-free CBMs (Oikarinen et al., 2023) in our evaluation, although we do use their labelling procedure to obtain concept labels in CIFARIC, C, in datasets where we have concept annota- tions (e.g., CUB), it is difficult to fairly compare Label-free CBMs and concept-supervised methods without ground- truth labels for label-free concepts. Moreover, we do not include GlanceNets (Marconato et al., 2022) in our evalua- tion, even though these models use leakage/OOD detector, because once leakage is detected by GlanceNet's OOD de- tector, the model does not provide solution that allows operations like interventions work in that instance.
5. Experiments
Research Questions We explore the following questions:
(Q1) Do MixCEMs have a high concept and task fidelity?
(Q2) Do MixCEMs remain intervenable and correctly bounded for ID and OOD samples?
(Q3) Are MixCEMs robust to OOD shifts?
(Q4) Do MixCEM's bottlenecks go OOD for OOD inputs?
Datasets We study these questions on the following tasks: (1) CUB (Wah et al., 201 1), a bird classification task with
5.1. Task and Concept Fidelity (Q1)
200 classes and 112 concepts selected by Koh et al. (2020), (2) AwA2 (Xian et al., 2018), an animal classification task with 50 classes and 85 concepts, (3) CelebA (Liu et al., 2018), face recognition task with 256 classes and 6 con- cepts selected by Espinosa Zarlenga et al. (2022), and (4) CIFAR-10 0 (Krizhevsky et al., 2009), classification task with 10 classes and with 143 concepts obtained in an un- supervised manner by Oikarinen et al. (2023). Finally, we construct concept-incomplete versions of CUB and AwA2 by randomly selecting 25% and 10% of their concepts, re- spectively. All datasets are described in Appendix C.
We first study MixCEM's task and concept fidelity through its task accuracy and mean concept ROC-AUC. As we are interested in designing models that satisfy both CA and BI, we emphasise that we do not expect MixCEM to be the best best-performing baseline in terms of its ID task and concept fidelity. Nonetheless, we use this study to verify that MixCEM's task and concept performances are competitive against existing methods. Our results, summarised in Table I and discussed below, suggest that this is indeed the case.
CiX is completeness-ggnostic and attains compet- itive task accuracies (Table 1, red). In three out of six tasks, MixCEM is in the set of best-performing base- lines with respect to task accuracy. We note that, in tasks where underperforms, MixCEM's drop against the best- performing CM (i.e., CEM) is relatively small (at worst close to 2% difference). More importantly, noticing that MixCEM's 's accuracy outperforms or closely matches that of black-box DNNs on concept-complete and incomplete
Baselines We compare MixCEMs against Vanilla CBMs (Koh et al., 2020), Hybrid CBMs (Mahinpei et al., 2021), CEMs (Espinosa Zarlenga et al., 2022), IntCEMs (Fi-- pinosa Zarlenga et al., 2023a), ProbCBMs (Kim et al., 2023), and P-CBMs (including their residual version) (Yuksek- gonul et al., 2023). Moreover, we include a vanilla DNN as a representative black-box baseline. When possible, all
6

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
CUB
Iacouplote
Axd2
Av12 Incoxplete
CIPAR1O
Colebd
11 Aeeurac (36)
130
25
33
180
103
003 Accann
10
23
100
103
100
Interoene Chooogie (95)
Intermmee Choorggt (90)
brorosee Conoggea (50)
Intersnn Comppoa (36)
Intersened Ceompph (75)
Internoual Commutte (SV)
Vunia CRM
Hybrd CRM
CIFM
InCRM
FrelCRM
P-CRM
Rosiual P-CRM
MisCIM (sem))
Bayes Choooifee
Figure+ 4. Task accuracy we intervene on concepts, selected at random, for ID (top) and OOD (hottom) test samples. OOD samples have a form of salt & pepper noise injected into at most 10% of their channels (similar results on other forms of distribution shifts can be seen in Appendix H). For the sake of efficiency, in the OOD plots approximate Bayes classifier that takes concepts using a masked MLP (sce Appendix D.4 for further details). Notice that across all tasks, both in ID and OOD instances, the area under the intervention curve for MixCEM higher than that of competing approaches. See Appendix M for tabulation of these results, including the estimated area under each intervention curve.
input only the intervened
tasks, our results indicate that MixCEMs are completeness- agnostic. The same does not hold for Vanilla CBMs, P- CBMs, and ProbCBMs.
cantly improve its performance in concept-incomplete tasks, suggesting that its embeddings properly leak information even after interventions. This is in contrast with non-leaky approaches (e.g., Vanilla CBMs and P-CBMs) and global- embedding- approaches (e.g., ProbCBMs), which sig- nificantly underperform in concept-incomplete tasks.
MixCEM has slight drop in concept AUC, yet re- mains competitive with respect state-of-the-art CMs (Table I, blue). MixCEM's mean concept ROC-AUC is slightly below the best-performing model in four of our tasks. Nevertheless, MixCEM's drop in mean concept ROC- AUC is relatively small for most tasks (< 2%) and con- sistently within the performance of similar SotA baselines, such as CEMs or InCEEMs The only task exception in CUB, where Hybrid CBMs attain significantly higher scores. However, discussed next, Hybrid CBMs are not generally intervenable. Therefore, they are not suitable candidates for -uman-in-the-log, scenarios, our setup of interest.
MixCEM achieves bounded intervenability and better OOD intervention accuracy (Figure 4, bottom). Fig- ure 4 shows the results of intervening on test samples cor- rupted with a form of "Salt & Pepper" noise. We study this form of noise as it is common within real-world de- ployment (Hendrycks & Dietterich, 2019; Mousavi et al., 2017) (see examples of corrupted images in Appendix G). However, we emphasise that, as shown in Appendix H, MixCEM's intervention improvements discussed below are also seen for other forms of real-world distribution shifts. This includes distribution shifts caused by downsampling, blurring, random affine transformations, and domain shifts (e.g., a model trained with MNIST (Deng, 2012) digits is intervened on samples containing real-world colour digits).
5.2. Intervenability (Q2)
We evaluate MixCEM's intervenability and show it is bounded for OOD samples. For this, we look at model's 's task accuracy as we perform concept interventions. As in previous works, we select intervened concepts uniformly at random and intervene on groups of related concepts simul- taneously when these groups are known (e.g., in CUB).
When looking at interventions on OOD samples, we that MixCEM is the only completeness-agnoss whose interventions are usually bounded: all of MixCEM's OOD intervention curves, except in CIFAR1 O and a short instance in Ce are always near or above the accuracy of the Bayes Classifier (BC, black dashed line). We be- lieve MixCEM's underperformance with respect to the BC in CIFAR10 and CelebA results from concepts in both datasets being difficult to properly learn for all methods due to concept label noise (their concept annotations come from CLsP- classification or subjective human annota- tions, both prone to mistakes). Nevertheless, across all tasks, MixCEMs have significantly higher OOD intervention ac-
baseline
When intervened on for ID samples, MixCEM outper- forms competing baselines (Figure 4, top). Our results show that MixCEMs not only significantly improve their task accuracy the more one intervenes (i.e., they are inter- venable), but they perform better or on par with IntCEMs (the best-performing baseline here). More crucially, they achieve this without expensive training-time time sampling, lead- ing faster training times than IntCEMs (see Appendix F). Finally, we observe that MixCEM's interventions signifi-
7

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
626
Noise 2556
Travwl Eng
Trane Lnngtrrde Thooeplate wtw
Uniateeeaed
de lold CEAL
beftia
MarTM
Tak Jornaey
190
Intervaned (85)
Interweued (R)
Intervesed (59) SEatEM (A22))
Internedd
Inservcard
0E3 X
IICEE
ProLCEM
OEN
MoCEM load)
In-Tietrilatioon (Ih)
Che (OOD)
Figure 5. (Leftt) CUB-InnCne Incomplete intervention curves. Test samples are perturbed by adding "Salt & Pepper" noise with increasing levels (% pixels corrupted), (Centre) Trave wiwieeiinnirrss intervention curves. We show results on a spuriously correlated validation set (dashed) and a test set without the spurious correlation (solid). (Right) t-SNE projections of bottlenecks before (top) and after (bottom) all concepts are intervened on for ID and OOD samples in CUB-Incomplete.
curacies than competing completeness-agnoeee baselines, especially CEMs and IntCEMs (up to approx. 48% and 41% improvement in AwA2 AwA2-Incomte Incomplete ete over CEM and IntCEM, respectively). Finally, the fact that MixCEM even surpasses the BC when all concepts are intervened on in concept-incomplete tasks (e.g., AwA2-Incomp lete and Ce CelebA) suggests that MixCEMs can exploit useful leak- age for OOD samples while avoiding leakage poisoning.
for test sets with the spurious correlation (ID samples) and without it (OOD samples). In particular, MixCEM's intervened performance is drastically better than that of IntCEMs when the spurious correlations disappear (adding more than 10% points in task accuracy when all concepts are intervened on). Nevertheless, we also notice that in concept-complete setups, although MixCEMs outperform InCEMMs once more, they are outperformed by ProbCBMs when no interventions are made on the test set without the spurious correlation (OOD samples). In Section 6, we provide some intuition to why this may be the case. Regardless, when compared against other completeness-agnosiie approaches (e.g., Residual P-CBMs, CEMs, InCCMMs. Hybrid CBMs), our results suggest that MixCEMs are more robust to different forms of OOD shifts.
5.3. Unintervened OOD Robustness (Q3)
Next, we study MixCEM's performance across different dis- tribution shifts. In particular, we evaluate MixCEMs as we (I) vary the amount of test noise on Inompll etee, and (2) train on the Travel ingniiiniiii rds dataset (Koh al., 2020), a variation of CUB where training-time spuri- ous correlation is introduced between the background and the downstream task labels (see Appendix C for examples). Below, we discuss how our results in Figure 5 suggest that MixCEM exhibits better OOD robustness. For simplicity, we focus on the best-performing baselines. However, we show our observations extend to all baselines Appendix I.
5.4. Concept Bottleneck Analysis (Q4)
Finally, we qualitatively study how concept bottlenecks are affected by concept interventions when samples go OOD, For this, we look at the t-SNE (Van der Maaten & Hinton, 2008) projections of concept bottlenecks for different CMs. Our experiments' conclusions are described below.
With and without interventions, MixCEM is more robust to test-time noise (Figure 5, left). Our noise ablation for CUB-Incomplete (Figure 5, left) shows that across noise levels, MixCEM achieves better task accuracy as it is intervened on than our baselines. More importantly, we see that for low-to-medium noise regimes (e.g., 5% and 10%), MixCEM achieves the best unintervened performance (left- most points). For example, when noise 5%, MixCEM's unintervened task accuracy is 29.69% vs CEM's 24.08%. This trend is observed for incomplete and complete tasks (see CUB results in Appendix I.... These results suggest that MixCEMs are robust models even without interventions.
MiX'S bottlenecks remain within distribution for both ID and OOD samples (Figure 5, right). In Figure 5 (right), we see that in contrast to bottlenecks in CEMs and Hybrid-CBMs, MixCEM's unintervened bottlenecks (top of figure) appear to remain within their ID bottleneck distri- bution for OOD samples. More importantly, they appear to closely match their original distribution when all concepts are intervened. From our baselines, only IntCEM's bottle- necks seem to remain closer to their original distribution after all interventions. This may explain why InCCEM's OOD interventions outperform CEM's. However, we ob- serve that MixCEM's bottlenecks capture a significantly greater proportion of the variance their ID bottleneck dis- tribution than InCCEM's, ss potentially explaining why OOD interventions more effective in MixCEM. These results suggest MixCEM learns concept representations that remain within distribution and avoid leakage poisoning for both ID and OOD samples.
MixCEM is more robust to spurious correlations in concept-incomplete tasks (Figure 5, centre). In Figure 5 (centre), we look at how training-time spurious correlations affect MixCEM's performance in concept-complete and incomplete versions of Travel ingBi rds. In the incomplete task, we see that MixCEM's unintervened and intervened accuracies are better than our baselines', both
8

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
6. Discussion and Conclusion
samples will not go OOD. Considered together, all of these results strongly suggest that IntCEM's intervention learning and MixCEM's embedding decomposition serve general design principles that can be applied to build better, more intervenable CMs depending on the circumstances under which one expects the CM to be deployed.
Prior Optimisation and Intervention Awareness Our experiments hint at a relationship between OOD inter- vention robustness and intervention awareness: InCEMM, which has intervention-aware loss, and MixCEM, which has a robustness-awan aware loss minimising prior's error Cask (v. fe)), achieve the best intervention accuracies for ID samples across all baselines. This suggests that robust- ness awareness can have a positive effect on ID intervenabil- ity. In the case of MixCEM, we believe its prior error min- imisation leads to both better ID and OOD intervenability because this term has an implicit incentive to maximise the model's 's performance both when all concepts are intervened on (as č is constructed by mixing embeddings based on the ground-truth concept labels) and when no leakage is allowed. Hence, this term has the extra effect of penalising MixCEM for mispredicting y when all concepts are intervened on. This has three surprising results: (1) MixCEM's ID inter- venability matches IntCEM's, s, without needing complex InCCE--like sampling- based loss; (2) even when set the concept weight loss to zero (i.e., Xe= = 0), MixCEMs remain highly intervenable as long as A > 0 (see Appendix J.1); and (3) improvements from training-time interventions (i.e., when Pint > 0) are much smaller for MixCEM than what has been observed for CEMs (see Appendix J.5). All of these suggest that prior error minimisation serves as ro- bust intervention-awaree regulariser.
Bias Mitigation Our Travel LngBirds results suggest that models incorporating constant global embeddings (e.g.. MixCEMs and ProbCBMs) better deal with spurious corre- lations. We believe this is because global embeddings, by defimition, block the flow of concept-indeepenenn informa- tion during inference. Thus, the model must learn to operate with the same shared representations for samples with and without the spurious correlation. This leads to models learn- ing representations that better capture under-represented groups (e.g., samples without spurious correlation) and may explain why ProbCBM, built on top of global em- beddings, achieves a high OOD unintervened accuracy in Travel iiniiinniiiii rds. Future work could then explore how global embeddings can be exploited for generalisation.
Limitations MixCEMs require more parameters (i.e., (km) weights for C) and hyperparameters' (e.g.. Ap and Pdrop) than CEMs. Although MixCEM is generally robust to its hyperparameters (see Appendix J), its memory and fine- tuning footprint open the door for future work to alleviate these constraints. Moreover, we foresee at least two poten- tial failure modes in MixCEMs: First, when a concept goes OOD and the shift renders the concept incomprehensible for an expert, MixCEMs may fail completely block leakage poisoning as one cannot intervene on such concept. Hence, future work can explore mechanisms for blocking all un- wanted leakage without knowing concept's 's label. Second, in incomplete tasks, intervened MixCEMs do not always recover the full ID performance in OOD inputs. Therefore, future work can explore how information about unprovided concepts can be better preserved after an intervention. Fi- nally, future work could explore (1) extending MixCEM's meth-
Learning Intervention Policies In MixCEM As Mix- CEMS are embedding- based methods, we can utilise the intervention-aware training pipeline used in InCEMMs to learn an intervention policy as by-product of training. In Appendix K we evaluate this variant of MixCEM, which we call "MixI"CEM", for the CUB-based datasets. There, we observe four interesting results. First, MixInCEMS re- main as accurate, without any interventions, as comparable InCCEMs and MixCEMs. Second, MixInCCEM learns a highly effective intervention policy that significantly out- performs a random intervention policy on the original Mix- CEM, both for complete and incomplete tasks. Next, see that MixInCEMMss OOD intervenability remains properly bounded and high (almost identical to the original Mix- CEM when interventions are selected randomly). More importantly, however, we see that MixIntCEM's interven- tion policy yields better intervention curves than all other baselines for OOD test sets. Finally, we nonetheless observe that, for some instances, intervening on In(CEM using its learnt policy yields better intervention curves than those seen when we intervene on MixInICEM using its own learnt policy. Nevertheless, we observe this only when intervening on ID test sets for complete datasets, suggesting there is still some value in maintaining the original IntCEM pipeline if we know we are operating in a complete task where test
embedding decomposition to other embedding- ods, such as ECBMs, and (2) devising better ways to inject priors into its global embeddings.
Conclusion In this paper, we show that previous state- of-the-art concept-based models are ill-equipped to concur- rently handle both concept-incompleeness and test-time interventions when inputs are OOD. To address this, we introduce MixCEM, a new concept-based architecture that uses an entropy-based gating mechanism to control when and how concept-independent feature information is leaked. Through an extensive evaluation across concept-complete and concept-incompie
tasks, we show that MixCEMs out- perform strong baselines by significantly improving accu- racy for both ID and OOD samples in the presence and absence of concept interventions.

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Impact Statement
Chen, Z., Bei, Y., and Rudin, C. Concept whitening for inter- pretable image recognition. Nature Machine Intelligence, 2(12):722-782, 2020.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Choi, J.. Raghuram, J., Feng, R., Chen, J.. Jha, S., and Prakash, A. Concept-based explanations for out-of- distribution detectors. In International Conference on Machine Learning, pp- 5817-5837. PMIR, 2023.
Acknowledgements
Collins, K. M., Barker, M., Espinosa Zarlenga, M., Raman, N., Bhatt, U., Jamnik, M., Sucholutsky, I., Weller, A., and Dvijotham, K. Human Uncertainty in Concept-Based Al Systems. In Proceedings of the 2023 AAAI/ACM Confer- ence on AI, Ethics, and Society, pp. 869-889, 2023.
MEZ acknowledges support from the Gates Cambridge Trust via a Gates Cambridge Scholarship. GD acknowl- edges support from the European Union's Horizon Europe project SmarCHANGE (No. 101080965), TRUST-ME (No. 205121L and from the Swiss National Sci- ence Foundation projects XAI-PAC (No. PZ00P2.216405). PB acknowledges support from the Swiss National Science Foundation project IMAGINE (No. 224226). MJ acknowl- edges the support of the U.S. Army Medical Research and Development Command of the Department of Defense; through the FY22 Breast Cancer Research Program of the Congressionally Directed Medical Research Programs, Clin- ical Research Extension Award GRANTI3769713, Opin- ions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the Department of Defense.
Crabbé, J. and van der Schaar, M. Concept activation re- gions: A generalized framework for concept-based ex- planations. Advances in Neural Information Processing Systems, 35:2590-2666 2022.
Deng, L. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6): 141-142, 2012.
Dreyer, M., Achtibat, R., Samek, W., and Lapuschkin, S. Understanding the (extra-) ordinary: Validating deep model decisions with prototypical concept-based expla- nations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3491- 3501, 2024.
References
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B. Sanity checks for saliency maps. Ad- vances in neural information processing systems, 31: 9505-9515, 2018.
Erhan, D., Bengio, Y., Courville, A., and Vincent, P. Visual- izing higher-layer features of deep network. University of Montreal, 1341 2009.
Espinosa Zarlenga, M., Pietro, B., Gabriele, C., Giuseppe, M., Giannini, F., Diligenti, M., Zohreh, S., Frederic, P., Melacci, S., Adrian, W., et al. Concept embedding mod- els: Beyond the accuracy-explainabiiit trade-off. In Advances in Neural Information Processing Systems, vol- ume 35, pp. 21400 2-00111413. Curran Associates, Inc., 2022.
Alvarez-Melis, D. and Jaakkola, T. S. Towards robust inter- pretability with self-explaining neural networks. arXiv preprint arXiv:1806.0753 07538, 2018
Azzeh, J., Zahran, B., and Alqadi, Z. Salt and pepper noise: Effects and removal. JOIV: International Journal on Informatics Visualization, 2(4):252 2018
Espinosa Zarlenga, M., Collins, K., Dvijotham, K., Weller, A., Shams, Z., and Jamnik, M. Learning to Receive Help: Intervention-Awant Concept Embedding Models. Advances in Neural Information Processing Systems, 36, 2023a.
Bau, D., Zhou, B., Khosla, A., Oliva, A., and Torralba, A. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 5541-6549, 2017.
Espinosa Zarlenga, M., Shams, Z., Nelson, M. E.,, Kim, B., and Jamnik, M. TabCBM: Concept- based Interpretable Neural Networks for Tabular Data. Transactions on Ma- chine Learning Research, 2023b.
Chauhan, K., Tiwari, R., Freyberg, J., Shenoy, P., and Dvi- jotham, K. Interactive concept bottleneck models. arXiv preprint arXiv:2212.074300 2022.
Falcon, W. A. Pytorch lightning. Gitflub, 2019.
Chen, C., Li, O., Tao, D., Barnett, A., Rudin, C., and Su, J. K. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019.
Gal, Y. and Ghahramani, Z. Dropout as bayesian approx- imation: Representing model uncertainty in deep learn- ing. In international conference on machine learning, pp. 1050-1059. PMLR, 2016.
10

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Ghorbani, A., Wexler, J., Zou, J., and Kim, B. Towards automatic concept-based explanations. arXiv preprint arXiv:1902.0331 2019.
Krizhevsky, A., Hinton, G., al. Learning multiple layers of features from tiny images. Technical Report TR-2009, University of Toronto, 2009.
Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In International conference on machine learning, pp. 1321-1330. PMLR, 2017.
Liu, Z., Luo, P., Wang, X., and Tang, X. Large-scale celeb- faces attributes (celeba) dataset. Retrieved August, 15 (2018):11, 2018.
Lundberg, S. M. and Lee. S.-L. unitied approach to inter- preting model predictions. Advances in neural informa- tion processing systems, 30, 2017.
Havasi, M., Parbhoo, S., and Doshi- Velez, Addressing leakage in concept bottleneck models. In Advances in Neural Information Processing Systems, 2022.
Maas. A. L., Hannun, A. Y., Ng. A. Y., et al. Rectifier nonlinearities improve neural network acoustic models. Proc. ieml, volume 30, pp. Atlanta, Georgia, USA, 2013.
He, K. Zhang, X., Ren, S., and Sun, J. Deep residual learn ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Magister, L.. C.,, Barbiero, P., Kazhdan, D., Siciliano, F, Ciravegna, G., Silvestri, F,, Jamnik, M., and Lio, P. En- coding concepts in graph neural networks. arXiv preprint 1.Xiv:2207.13880 2022.
Hendrycks, D. and Dietterich, Benchmarking neural network robustness common corruptions and perturba- tions. arXiv preprint arXiv: 00t::/9903. /903. /2261, 2019.
Hendrycks, D. and Gimpel, K. A baseline for detecting misclassified and out-of distribbittionn examples in neural networks. arXiv preprini urXiv: 2016.
Mahinpei, A., Clark, J., Lage, I., Doshi Velez, F., and Pan, W. Promises and pitfalls black- box concept learning models. arXiv preprint
2021.
Hunter, J. D. Matplotlib: A 2d graphics environment. Com- puting in Science & Engineering, 9(3):90 2007. doi: 10.1 10.1109/MCSE2000
Marcinkevičs, R., Laguna, S., Vandenhirtz, M., and Vogt, J. E. Beyond concept bottleneck models: How to make black boxes intervenable? arXiv preprint arXiv:2401. 135.44 2024.
Kazhdan, D., Dimanov, B., Jamnik, M., Liò, P., and Weller, A. Now you me (eme): concept- based model extrac- tion. arXiv preprint arXiv:2010. /3233, 2020.
Marconato, E., Passerini, A., and Teso, S. Glancenets: In- terpretable, leak-proof concept-based models. Advances in Neural Information Processing Systems, 35:21212- 21227, 2022.
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., et al. Interpretability Beyond Feature Attri- bution: Quantitative Testing With Concept Activation Vectors (TCAV). In International conference on machine learning, pp. 2668-2677. PMLR, 2018.
Mousavi, S. M., Naghsh, A., Manaf, A. A., and Abu- Bakar, S. A robust medical image watermarking against salt and pepper noise for brain mri images. Multimedia Tools and Applications, 76: 10313-10333, 2017.
Kim, E., Jung, D., Park, S., Kim, S., and Yoon, S. Proba- bilistic concept bottleneck models. In International Con- ference on Machine Learning, pp. 16521-16540. PMLR, 2023.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A. Y., et al. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop Deep Learning and Unsupervised Feature Learning, pp.
Kindermans, P.-J., Hooker, S., Adebayo. J., Alber, M., Schütt, T., Dähne, S., Erhan, D., and Kim, The (un) reliability of saliency methods. arXiv preprint 1rXiv:1711.00880 2017.
4. Granada, 2011.
Oikarinen, T., Das, S., Nguyen, L. M., and Weng. T.-W. Label-free concept bottleneck models. In The Eleventh International Conference on Learning Representations, 2023.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01700 2019.
Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., and Liang, P. Concept bottleneck models. In International Conference on Machine Learning, pp. 1338-5344 PMLR, 2020.

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Platt, J. al. Probabilistic outputs for support vector ma- chines and comparisons regularized likelihood meth- ods. Advances in large margin classifiers, 10(3):11-74, 1999.
time interventions. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurlPS 2022, 2022.
Shin, S., Jo, Y., Ahn, S., and Lee, N. A closer look the intervention procedure of concept bottleneck models. arXiv preprint arXiv:2302. 14260, 2023.
Rabanser, S., Günnemann, S., and Lipton, Z. Failing loudly: An empirical study of methods for detecting dataset shift. Advances in Neural Information Processing Systems, 32, 2019.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: simple way to prevent neural networks from overfittigg The journal of machine learning research, 15(1):1929-1958 2014.
Radford, A., Kim, W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learming, pp. 8748-8763. PMLR, 2021.
Steinmann, D., Stammer, W., Friedrich, F., and Kersting, K. Learning to intervene on concept bottlenecks. arXiv preprint arXiv:2308. 13453, 2023.
Ribeiro, M. T., Singh, S., and Guestrin, C. "Why should trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. pp. 1135-1144, 2016.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008,
Vandenhirtz, M., Laguna, S., Marcinkeviãs, R., and Vogt, J. E. Stochastic concept bottleneck models. arXiv preprint arXiv:2406. 2e0222406.9972 19272, 2024.
Rigotti, M., Miksovic, C., Giurgiu, I., Gschwind, T., and Scotton, P. Attention-based interpretabilityy with concept transformers. In International conference on learning representations, 2021.
Wah, C., Branson, S., Welinder, P., Perona, P., and Belongic, S. The caltech-ucsd birds-200-2011 dataset. Technical Report CNS-TR-201 1-001, California Institute Tech- nology, 2011.
Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case gener- alization. arXiv preprint arXiv:1911.08733 2019.
Wijaya, M. A., Kazhdan, D., Dimanov, B., and Jamnik, M. Failing conceptually: Concept-based explanations of dataset shift. arXiv preprint arXiv:2104.089) 08952, 2021.
Scheirer, W. J., de Rezende Rocha, A., Sapkota, A., and Boult, T. E. Toward open set recognition. IEEE transac- fions on pattern analysis and machine intelligence, 35(7): 1757-1772, 2012.
Xian, Y., Lampert, C. H., Schiele, B., and Akata, Z. Zero- shot learning- comprehensive evaluation of the good, the bad and the ugly. IEEE transactions on pattern anal- ysis and machine intelligence, 41(9):2251-222 2018.
Schlegl, T., Seeböck, P., Waldstein, S. M., SeSSSShtm-trfurtt U. and Langs, G. Unsupervised anomaly detection with generative adversarial networks to guide marker discov- cry. In International conference on information process- ing in medical imaging, pp. 146-157. Springer, 2017.
Xu, X., Qin, Y., Mi, L., Wang, H., and Li, X. Energy- concept bottleneck models: unifying prediction, con- cept intervention, and conditional interpretations. arXiv preprint 14X1v:2401.14142, 2024.
Yang, Y., Panagopoulou, A., Zhou, S., Jin, D., Callison- Burch, C., and Yatskar, M. Language in bottle: Lan- guage model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19187-19197, 2023.
Sevyeri, L. R., Sheth, I., Farahnak, F., Kahou, S. E., and Enger, S. A. Transparent anomaly detection via concept- based explanations. arXiv preprint arXiv:2310. 10702, 2023.
Shen, M. W. Trust in Al: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient. arXiv preprint arXiv:2202.05332. 2022.
Yeh, C.-K., Kim, B., Arik. S., Li, C.-L., Pfister, T., and Ravikumar, P. On completeness-awee aware concept- explanations in deep neural networks. Advances in Neural Information Processing Systems, 33:20554 205665, 2020.
Sheth, I. and Ebrahimi Kahou, S. Auxiliary losses for learn- ing generalizable concept-based models. Advances in Neural Information Processing Systems, 36, 2024.
Yuksekgonul, M., Wang, M., and Zou, J. Post-hoc con- cept bottleneck models. In ICLR 2022 Workshop on PAIR2Struct: Privacy. Accountabilityy Interpretability, Robustnsss Reasoning on Structured Data, 2023.
Sheth, I., Rahman, A. A., Sevyeri, L. R., Havaei, M., and Kahou, S. F. Learning from uncertain concepts via test
12

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Zhou, C. and Paffenroth, R. C. Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD international conference knowledge discovery and data mining, pp.
2017.
13

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
A. MixCEM Concept Probability Calibration
Platt scaling (Platt et al., 1999) is a post-hoe calibration method used to transform the outputs a probabilistic classifier into well-calibrated probabilities (i.e., probabilities that better represent the model's true uncertainty). In this work, we apply common adaptation of Platt Scaling by Guo al. (2017) calibrate a MixCEM's concept probabilities so that they better represent their uncertainty and, therefore, may serve as better indicators for when concept might have gone OOD.
In practice, Platt scaling involves fitting a logistic regression model to the logits (pre-sigmoid activations) of MixCEM's concept scores P on a validation dataset. Specifically, for each concept in our training set, given the logit 2 := 0 (p:) of concept prediction pt. we calibrate its output by learning a linear transformation of 2;:
(c = 1 2.) (a:z + by);
where 0g by R parameters specifie to each concept. These parameters are learnt using maximum likelihood estimation for Ecal epochs the validation data after the MixCEM has been trained.
When we learn parameters a and b, we freeze all other parameters in MixCEM and minimise the Binary Cross Entropy loss between scaled concept predictions and their ground-truth labels in the validation set. This means that the model's concept validation accuracy remains the same throughout this process, although the task accuracy may change slightly (as the task predictions are function of the concept predictions). By the end of this optimisation, MixCEM's concept predictions are a better fit to represent the true model uncertainty and can therefore be better at identifying when a sample's concepts have gone OOD, The results of including Plat Scaling as a post-processing step of MixCEMs are further discussed in Appendix J.4.
B. Maximum likeeihood of MixCEM
Once we have constructed MixCEM's global, residual, and contextual embeddings, we need to devise a proper training objective that captures all our intended desiderata. In this section, we derive such an objective by framing MixCEM as a graphical model and deriving a likelihood function, the maximisation of which will enable to learn our model's parameters given the concept and task likelihoods we derived above.
To construct our training objective, we note that most CMs implicitly assume that there exists a complete set of concepts C* that are the generating factors of variation of the samples X and the downstream tasks Y (see Figure 6a). When we train a CM such as CBM, however, we operate under the assumption that we observe features X and, from these features, we need infer a potentially incomplete subset of concepts C' (i.e., the training concepts) from which we then infer a downstream label Y (see Figure 6b). Notice therefore that because C' may not contain all the information found initially in C*, traditional CBMs may struggle to achieve CA unless additional information bypasses channels introduced in their inference process. Therefore, we will frame MixCEM as graphical model that explicitly represents the fact that the set of training concepts C will require some additional residual information R for us to properly reconstruct a complete set of concepts C* from which we can accurately predict Y.
X
C
X
C
Y
(a) Data generative process
(b) Graphical model of traditional CBM
Figure 6. Assumptions underlying CMs. (a) Generative process assumed by CMs, and (b) graphical model underlying CBMs, where darkened nodes represent observed variables (i.e., inputs). The implied generative process for CMs assumes the existence of of complete concepts C* that can perfectly describe features X and task labels Y. In contrast, traditional CMs such as CBMs operate under the assumption that, given a set of observable input features X, they must first infer a potentially incomplete concept set C (i.c., the training concept set) and then they must infer downstream task label Y from those concepts.
14

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
C
X
C
Y
R
Figure 7. MixCEM's graphical model. MixCEM assumes that a complete set concepts C* can constructed from (1) set concepts aligned with the training concept annotations C, that follow some prior C, and (2) set of residual variables H. that describe all information missing from C' that is relevant for predicting the downstream task Y.
In our framing of MixCEM in Section 4, we argued that we can decompose our concept activations C' into two components:
one that is samplc-specific (i.e., the residual embeddings) and one that is concept-specific (i.e., the global embeddings). Without loss of generality, we can formally express this consideration in a single graphical model (see Figure 7) by letting the complete concept set C* be a function of three factors: two independent factors I and C and third dependent factor C. Here. (1) C' represents a potentially incomplete of training concepts given to MixCEM, (2) C represents a prior set of beliefs about concepts in C' (it is independent of specific input observation), and (3) represents all residual information in X that is captured by the complete concept set C* but is missing from the training concepts C. Hence, R provides the appro- priate context and higher level of detail about the input observation when predicting Y from set of incomplete concepts C.
Assuming we observe X. MixCEM's graphical model can be factorised as follows:
C(Y,CJĆ,X) ΣEP( CC CC, 1X,
(I)
R C
where:
PY C*) is categorical distribution over task labels given the complete concepts C*, In MixCEM, we parameterise this model via the label predictor classifier :C Y.
(CC| X, C') is Bernoulli distribution that models the presence or absence of each concept in concept set c c C given X and C. In MixCEM, this distribution is parameterised by the concept scoring functions {s;] where we interpret C as learnable parameters of the classifier corresponding to MixCEM's 's global embeddings.
PCC" |C Rt) is a deterministic bottleneck encoder 7 C x R x C + C generating MixCEM's bottleneck è from the concept values, the priors, and the residuals. As both the concepts and the residuals are functions of x, we express this bottleneck as 7xx C).
(R X) is deterministic function capturing MixCEM's residuals (x)(*)()),)) (x) Abusing notation, encapsulate this learnable function as X + R.
We can use this factorisation learn MixCEM's parameters. Specifically, given a annotttee dataset of i.i.d. triples D ((x(1) ((0))))),,,,,)))))))))))) ( MixCEM's parameters can be learnt via gradient descent by maximising the empirical log-likelinee log-likelihood the training data. For this, we consider the prior variables C ~ Ö as parameters for the optimisation, similar the weights 0y and fy, which represent the weights of the concept encoder and label predictor, respectively. This yields the following objective function:
(P( P((Y v, = x = =x;; 0g 0, Ci
8g Bg C
arg max ( ((x.o.))>D Gg.0,.C
(2)
Considering that both the residual and bottleneck encoder distributions are deterministic functions of their inputs, for any sample (x, c, y) ~ we can use Equation 1 to rewrite the objective function the expectation above as:
P y. C c X x: 0, 0y, C)- ( y C 7x, C);0,, P,
x5 C)
15

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Here, we make a practical modelling assumption. When learning our parameters 0z. 0g. and C, want disentangle the gradients from the downstream task Y that update parameters 0g and 0g from those that update C. This is because we want our prior embeddings C to be, on their own, informative possible about the downstream task so that the residuals r r(X) only contribute to C" with information about Y that is not already encoded in C. As such, we factorise the likelihood of the downstream task Y with respect to C* as:
P(Y (TT = P(Y - (()) (e)
(3)
where
PY C,,((), C) represents the task distribution when observing the residual embeddings, the predicted training concepts, and the prior embeddings. In other words, this is the task likelihood we wrote down for MixCEM in Section 4.
(Y é) represents the task distribution given the training concepts alone. This likelihood can be interpreted as the label f(c) predicted when we only provide the label predictor f with the global prior embeddings of the training concepts. Therefore, in practice, č can be thought of as the bottleneck formed by mixing the global (prior) concept embeddings using the ground-truti concept labels as coefficients:
= [(rccft] + - ca)e(-))), (cack (+) cc)c
(4)
This simplification allows us to express MixCEM's log- likeliielihood LL y) = Pog = J.C =c =x; 0g. 0) of a data sample (x, C, y) as:
LL(x, v) = log (P(y C* = =x(x); fz. 0g. C)P(y C* = E; c; 0 0,) x= 09) = -log C* -===9(x =9(x)) + log P(y C* === + log ( x xx)
Plugging in MixCEM's likelihoods for the task and concept predictions derived in Section 4 and minimising the negative log-like oogllikelihood, we get final training objective:
arg min L (xx 0g,0,
log logP(y ((x)) - log a) - -log (cc xx]
arg
arg min E =( Ci (v. + Xpla f(e) + AcB
0,00,.C
where 8)) is task-specific loss that minimises the negative log likelihood of the downstream task labels (e.g.. cross-entropy) and BCE(c, () is the average binary cross-entropy loss between the ground-truth concepts e and MixCEM's predicted concept probabilities p.
Notice that in the final objective loss function, we include two regularisation hyperparameters, de and Ay- that allow control how much each term affects the parameters we learn (we discuss the effect each hyperparameter has in Section 6). As in traditional jointly trained CBMs and CEMs, A. b controls how much we weigh concept fidelity vs task fidelity; in other words: how much we weigh the third term of the objective function above. In contrast, our second hyperparameter, Ap E B maximises the task accuracy when only the global embeddings are used (i.e., the second term in the objective function). The loss term corresponding to this byperparameter, which we call the prior error, maximises the information about the downstream task y encoded in the global concept embeddings, ensuring that the learnt global embeddings are informative enough to predict the downstream task even when no additional information leaked from x into our label predictor. From an information-theorettee perspective, this leads to MixCEM creating information bottleneck where most of the information about Y is encoded in C' and only minimal residual information flows from X in the form of the residuals.
Information theoretic discussion In IID cases, if the concept prior Ö is not informative enough to make predictions, the model could exploit the contextual information in X to refine concept beliefs and generate more accurate concept predictions. aanietteessssgnoottsiimmm However, in non-i.i.d settings,
In this setting, X is useful to attain high task fidelity (ensuring X encodes features from an unknown distribution, which may result in concept posteriors that are worse than the prior. For
16

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
this reason, we may need exclude X from the computation of Y in such cases. If we exclude X from the computation, we need C' to incorporate as much information possible about Y; otherwise, our predictions will not be better than random chance, and interventions will be ineffective.
From an information theoretic perspective, MixCEM creates an interpretable information bottleneck where most the information about Y is encoded in C. and only minimal residual information flows from X (i.e., I(Y;C) > I(Y;)) as the loss term Lask (y. f()) encourages the model to make predictions from global concepts alone.
C. Datasets
Below, we discuss the tasks and datasets used for our experiments in Section 5. A summary of the main characteristics of each task can be found in Table 2.
Table 2. High-lovel properties of all tasks used in our experiments.
Dataset
Training Samples (N)
Validation Samples
Testing Samples
Inpur Shape (i)
# Labels (I)
# Concepts (k)
Concept Groups
CDR
4,796
1,198
5,794
(3, 2999. 299)
200
112
28
CUB Incemmeeee
4,796
1,196
5,794
(3 299,9
200
22
AwA2
22,393
7.464
7.465
(3,224, 224)
50
85
28
SwA2-Innomglite
22,393
7,464
7,465
(3,224, 224)
9
CelebA
11,818
1,689
3,376
(3,64, 64)
256
6
N/A
CIFAR1O
40,000
10,000
10,000
(3,32, 32)
10
143
N/A
Travelings:rds
4,796
1,198
5,794
(3,299, 299)
200
112
28
Travel 1
4,796
1,198
5,794
(3, 299, 299)
200
22
7
CUB The CUB bird classification image task is constructed from the Caltech-UCSD Birds-200-2011 dataset (Wah et al., 2011). Each sample in this task corresponds to a (3 x 299 x 299) RGB image of bird (normalised in [o, 1)). annotated with one of 200 bird species. Here, each image has 312 binary attribute annotations (e.g.. "black nape", "ye 1 1 oo wi wing ng colour", etc.). We construct a set of 112 binary concepts following the selection of attributes used by Koh al. (2020), Moreover, we follow the same majority-voting standardisation of concepts across classes as in (Koh et al., 2020). This leads to all samples from the same class having the same concept profiles. We do this that this task's concept annotations truly complete with respect to the downstream task (i.e., they can fully describe each downstream label) and to maintain this dataset aligned with how it is usually used in the concept-based XAI literature. Finally, all 1 112 concepts can be grouped into 28 groups that encapsulate semantically related concepts (e.g., "black wing colour" " and "zed wing colour"). When performing interventions, all concepts within the same group are intervened on at once, meaning we do interventions on a group-level basis. As it traditionally done for this dataset Koh et al. (2020); Espinosa Zarlenga et al. (2022); Kim et al. (2023), we normalise all images, and during training, we randomly flip and crop images. For this task, and its incomplete version, use the same main-validation-tee. splits in (Koh et al., 2020).
CUB-Incomplete The CUB- Incomplete task is generated from our CUB task by randomly selecting 25% of CUB's concept groups before training and using the labels of concepts within those groups as each sample's concept labels. We do the concept subsampling only once and use the same subselection across all sceds/rounds of experiments. This resulted in us randomly selecting the following 7 groups of concepts ("bill shape", "head pattern", "breast colour", "bill length", "wing shape", "tail pattem".. "bill colour"). Together, all of these concept groups yield total of 22 binary concept annotations. We use this dataset as an example of a conoept-incomplete task. We use the same splits and training augmentations as in CUB.
AwA2 The AwA2 task is constructed from the Animals with Attributes 2 (Xian et al., 2018) dataset. Each sample in this task consists of normalised (3, 224, 224) RGB image of an animal annotated with one out of 50 species classes (e.g., "zebra", "polar bear", etc.) In addition to species label, each sample is annotated with 85 binary attributes (e.g., "black", "white", "stripes", "waterr", etc.). We use these binary attributes as concept labels and split them across 28 groups of semantically related concepts. Specifically, we group concepts across the following categories: { ('colour", "fur pattern", "size", "limb shape", "tail", "teeth type". "horns", "claws", "tusks", "smelly", "transport mechanism". "speed", "strength". "musccl", "movement move". "active", "nocturnal", "hibernate", "agility", "diet", "feeding type", "general location", "biome", "fierceness", "smart", "social mode", "nest spot", , "domestic"). Similar to CUB, all samples with the same class in this dataset share the same concept profiles. The train-validation-tee data splits are produced via a random 60%-20%%209 split, and samples are randomly cropped and flipped during training as in CUB.
AwA2 -ncmmInnoeeeeee The AwA2-T Inco-IInompeeeeee is constructed from the AwA2 task by selecting, at random, 10% of AwA2's concepts to use as training annotations. This resulted in us selecting the following 9 concepts ("black", "gray",
17

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
"stripes", "hairless", "flippers", "paws", "plains", "fierce", "solitary"). This dataset provides another example of more realistic concept-annotated dataset where the concept labels are not complete descriptions of the downstream task.
CIFAR10 To explore our method in tasks without concept labels, we incorporate the CIFARIC (Krizhevsky et al., 2009) dataset as part of our evaluation. In this image detection task, each sample is (3, 32, 32) normalised RGB image that can be one out of 10 object types (e.g., "aeroplanes", "cars", "birds", "cats", etc.). As done by Marcinkevićs et al. (2024) and Vandenhirtz et al. (2024), we annotate all samples in this dataset with 143 textual concepts whose semantics were obtained in an unsupervised manner by Oikarinen et al. (2023). This allows us to construct numerical, unnormalised concept scores using the CLIP (Radford al., 2021) similarity score between each image and each concept's textual description.
In contrast to how Marcinkevičs et al. (2024) and Vandenhirtz et al. (2024) binarise these concept scores, however, we do not use zero-shot CLIP classifier selecting between a concept description and its textual negation. This is because generating binary concept labels in this manner leads to significantly imbalanced labels. Hence, such labels are extremely difficult for models to accurately learn as measured by their mean concept AUCs (although one can easily train models that achieve high concept accuracies, similar to those reported (Marcinkevič' et al., 2024), given the high label imbalance). Therefore, here, we instead binarise concepts by thresholding their scores based on their 50th percentiles, as estimated from the entire dataset. This leads concepts that are both balanced and still informative (e.g., we can see that our Bayes Classifier achieves high accuracies when provided with all concept labels in Figure 4 (bottom)). Nevertheless, note that by relying unsupervised model such as CLIP to annotate these labels, this dataset will undoubtedly contain noisy concept labels that do not always accurately represent their intended semantics.
external,
CelebA Our CelebA task is the same as used by Espinosa Zarlenga et al. (2022) based on the Large-scale CelebFaces Attributes dataset (Liu et al., 2018). Here, every sample an annotated normalised RGB facial image that has been downsampled to have shape (3, 64, 64). As in (Espinosa Zarlenga et al., 2022), we select the top-8 most balanced attributes from all of CelebA's 40 attributes to construct downstream label y as the decimal representation of the vector containing all 8 attributes. We point out that although this process can yield up to 256 distinct task labels, in practice, we could only find 230 of them within the training samples, and we noticed that these labels were highly imbalanced (which results in a very difficult task to solve). To make this task concept-incomplete, we provide as concept annotations only the top-6 most balanced attributes, leaving two of the concepts needed to predict y out of our training annotations. Finally, as in (Espinosa Zarlenga et al., 2022) and for consistency with previous works, our training set here is formed by randomly subsampling the original CelebA's 's training set to a 12th of its size.
TravelingBirds and TrawelLLnghiirddd-TTroop.Leeeet
The TravelingBirds task and its incomplete version are variations of their respective CUB tasks. These tasks, based on the Srrvelinggize dataset proposed by Koh et al. (2020), introduce a new background to all images in CUB. This background is sampled from a category (e.g.. "seashores", "forests", "coffee shops", etc.) which is, by design, correlated with the sample's s task label (see Figure 8). As in the original Travel TravelingBi rds dataset, the test sets we use for these tasks exhibit a distribution shift where the background of each bird is assigned to a different category, making these tasks suitable test beds for generalisation and OOD evaluation.
Train
Test
Figure 8. Randomly selected training and test samples of for the class "Yellow-headee Blackbird". Notice that training samples all have "aircraftccariie" backgrounds while the testing samples have "sauna" backgrounds.
18

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
D. Training, Model Selection, and Hyperparameters
D.I. Training
During training. we use the standard categorical cross-entropy loss as Cask. For baselines that optimise a binary cross entropy loss between a predicted set of concepts and their corresponding ground truth labels, we use a weighted binary cross entropy loss that weights the loss of each concept's label proportionally to its representation in the training distribution (except in CelebA due to instabilities where we use unweighted version). That way, we encourage models to learn useful concept predictors in tasks with high concept imbalance (e.g., CUB).
Unless specified otherwise, all baselines are trained using Stochastic Gradient Descent (SGD) with momentum 0.9. When computing batch-level gradients, we use a batch size of 64 for all CUB-based tasks (given their large sample and concept dimensions). Otherwise, we use a batch size of 512 for all other tasks. Similarly, when possible, we fix the initial learning rate Ir to values used by previous works and decay it during training by a factor of 10 if the training loss reaches a plateau after 10 epochs. Specifically, we use 1r 0.01 for all tasks except for CelebA, where we use 1r 0.05 as in (Espinosa Zarlenga et al., 2022). Finally, based on the configuration by Koh et al. (2020), we weight decay 0.000004 for the CUB-based and AwA2-based tasks.
All models were trained for total of E' epochs, where E' = 150 for all datasets except for CIFARI0, D, where it is E = 50. We use early stopping by tracking the validation loss and stopping training if an improvement in validation loss has not been seen after (patience) x (val_freq) epochs, where patience and val_freq, the frequency at which we evaluate our model on the validation set, is val freq 5.
D.2. Base Architecture
Across all models, we use a R8 (He et al., 2016) pretrained on ImageNet as the backbone architecture &i for all tasks except for CelebA, where we use larger model (a ResNet34) as the dataset is smaller. Specifically, we use the output of the second-to-last layer in ResNet (the layer before the original logits) backbone for all models. We do not freeze the initial pretrained weights. If the backbone is required have a specific output dimension (e.g., in Vanilla CBMs), then we achieve this by adding a leaky ReLU (Maas et al., 2013) nonlinearity and a linear layer with the correct output shape to the ResNet model mentioned above.
For the label predictor fé),, we use a single linear layer for all baselines. The only exception for this is ProbCBMs, where, as proposed by the original authors, we perform inference via a distance-based layer that learns class embeddings in R Dy and compares their distance a learnt linear projection of the bottleneck è onto that space.
D.3. Model Selection
Across all tasks and baselines, we performed a hyperparameter search with the aim of representing each baseline fairly. For this, we made significant efforts to provide each baseline with a similar fine-tuning budget. Moreover, we aimed to provide all baselines with the opportunity have the same capacity as each other by including hyperparameterisations that lead to similar parameter sizes. When possible and available, we attempted to use the same hyperparameters and hyperparameter recommendations provided in the original works that proposed each baseline. Yet, it is worth noting that in some instances, our model selection process chose hyperparameters that resulted in models with fewer parameters, as they yielded better validation metrics.
After considering a model-specific selection of hyperparameters for each baseline (summarised in Table 3), we report the results of only the baseline with the highest area under its validation task accuracy vs intervention curve (the only exception being DNN, as it is unintervenable and therefore perform model selection based on its validation task). We use this area as proxy for metric that captures task fidelity, concept fidelity, and intervenability. Below, provide details of all baselines, together with the hyperparameters that our model selection yielded for each of them across our tasks.
D.4. Baseline Details and Selected Hyperparameters
Vanilla CBM All Vanilla CBM results in the main body of this work are produced from a jointly trained CBM (with concept weight loss Aa) whose bottleneck is sigmoidal (i.e., è is in [0. 1j*). Our model selection here yielded X, = 1 for all concept-incomplete tasks CUB- Incomplete, AwA2- Incompleteeeeee and Celeba In contrast, for all other (concept complete) tasks, our model selection chose A = 10.
19

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Table 3. Sets of hyperparameters considered when performing our model selection. For each hyperparameter, we indicate which bascline(s) that hyperparameter is relevant to. For clarity, we separate MixCEM's s specific hyperparameters at the bottom part of this table.
Hyperparameler
Semantics
Searched Values
Baselines Fine-tuming These Hyperparameters
A
Concept loss weight
{1, 5, 10}
CBM, Hybrid CBM, CEM. InCCEM
K
Extra unsupervised bottleneck dimensions
[0. 50, 100, 200)
Hybrid CBM, DNN
m
Concept embedding space dimension
{16, 32}
CEM, ProbCBM
Dy
Class embedding space dimension
[64, 128)
ProbCBM
7
Training intervention loss penalty
{1.1, 1.5]
IntCEM
Acoli
Intervention policy regulariser
{0.1, 5,
InCCEM
Acomples
Complexity regulariser
{0.000001 0.001,0.1]
P-CBM, Residual P.CBM
Ap
Prior loss weight
{0.1, 1)
MixCEM
Pdrop Eeal
Residual dropout probability
(0.1. 0.9.
MixCEM
Number of Platt scaling epochs
{0,
MixCEM
Hybrid CBM Hybrid CBMs (Mahinpei et al.. 2021) are variations of jointly trained sigmoidal CBMs where the concept bottleneck è [CaligeddCunalinnall
(R++k) is formed by the concatenation of a binary component Caligned E [0. 1]k, whose i-th entry is trained to be aligned with the i-th ground-truth concept, and vealev unconstrained component Cextra E RR* whose k' entries are not aligned to any known concept. Our model selection for Hybrid CBMs yielded k' = 50 for all tasks except for CelebA, which yielded K' = 200. Similarly, we selected A = 10 for CelebA, AwA2, and AwA2-Incomple:se d = 1 for CUB-Incomples CUs-Incomplete and CIFARIO, 0, and A = for CUB.
CEM When training CEMs, we intervene on a concept with probability Pint = 0.25 (as suggested by the authors (Es- pinosa Zarlenga et al., 2022)). In this setup, our model selection selected m 16 for CEM's embedding size for all tasks except for Ce CelebA, where we obtained m 32. Finally, we used the following concept loss weights: X 10 for CelebA, Ae 5 for AwA- Incomplete, and d 1 for all other tasks.
IntCEM Given the number of hyperparameters in IntCEMs, we focused on fine-tuning only the concept loss weight Ae. the training-time task intervention penalty 7. and the intervention policy regulariser A Therefore, we fixed the embedding size m to that used by the equivalent CEM in the respective task, the maximum number of training-time interventions T to 6, the initial probability of intervention as Pin. = 0.25, and the annealing rate for T to 1.005. We chose these values based on the suggestions by the original authors of this work. This resulted in the following hyperparameters being selected: (a) X = 1 for all tasks, (b) 7 = 1.5 for CUR, Ce l eba, AwA2, and AwA-Incompl ete and y = 1.1 for all other tasks, and (c) Avoll = 0.1 for AwA2 Incomplete, Aroll for CIFAR1O, and A for all other tasks. Finally, for stability, we use global gradient clipping (clipping value of 100) for CelebA.
ProbCBM We attempt to closely follow the same hyperparameters for ProBCBMs used in the original work by Kim et al. (2023). As such, we (1) always use an Adam (Kingma & Ba, 2014) optimiser, (2) use a starting learning rate of 0.001 (except for CIFAR10 where we increase it to 0.01 as otherwise the model severely underperformed), (3) fix the number of training and inference samples 50, (4) intervene on concepts during training with probability Pin = 0.5, (5) warm-up the model for 5 epochs, and (6) scale the KL divergence regulariser λKL.
10-5, as these were the hyperparameters used on the authors' original experiments. Similarly, we use weight decay 1 x 10- a learning rate 10 times smaller for the non non-pretrained weights, and clip gradient norms to 2 as the authors do in their official code base?.
As suggested by the authors, we trained ProbCBMs in a sequential manner. For this, we use early stopping for a maximum of E epochs but, for fairness, we spend at most E/2 of those epochs training the concept encoder and the remaining epochs training the task predictor. This left us with fine-tuning the dimensionality of both the concept embeddings (m) and the class embeddings (Dy), two hyperparameters which we noticed had critical role in ProbCBM's performance. Here we use (a) m 32 for CUB and CelebA, and m = 16 otherwise, and (b) Dy 128 for CUB and AwA2, and D, = 64 otherwise.
As in (Kim et al., 2023), we perform interventions in ProLCBMs by replacing sampled concept embeddings with the learnt embedding means of their corresponding ground-truth labels.
Posthoe CBM We train both the standard and residual versions of P-CBMs. Here, we first train a DNN on the downstream task y for E epochs by attaching two linear layers, with a leaky ReLU between them, to the output of the backbone sb. The first linear layer will have as many neurons as concepts in the dataset and the second layer will have as many neurons as
"See http://gww....ooo////////o/mmmme
exp.yaml#I.30.
20

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
output labels in the specific task. This is done so that we provide this model with similar capacity the CBMs we train. We train the black-box model with the same optimiser's hyperparameters used for other baselines.
Once the black box task predictor has been trained, we extract a training set of embeddings by projecting the entire training set to the space of the second-to-last layer of this model (i.e., the space of the first linear layer with k neurons that we added), Then, we learn the Concept Activation Vector (CAVs) (Kim et al., 2018) for concept e using the vector perpendicular to the decision boundary of a linear SVM, with Eg penalty C' = 1, trained to predict concept o; from the activations of the second-to- -oosddto--lat layer of the black box DNN.
When fine-tuning P-CBMs's sparse linear classifier, we fine-tune the complexity regulariser Auomplex and fix the elastic net's 4 ratio to be 0.1. Our model selection chose Acomplex = 1 x 10-6 for CUB, AwA2, and CIFAR10, O, Acomplex 0.001 for CUB-Incomp e and AwA2-Inccompl Tncompplooooooo ete, and Acomplex = for all other tasks.
When training the residual version of P-CBM, we provided the residual layer with hidden neurons to enable this model to have a closer capacity than that of competing baselines. Our tine-tunig for this model selected Asomplex = x 10 6 for CUB, Acomplex = 0.001 for CIFAR10 0, and Acomplex = 0.1 for all other tasks.
Finally, given lack of direct mechanism for performing concept interventions on P-CBMs, when we intervene on their concept predictions, we follow the same intervention process as in Vanilla CBMs, whose bottlenecks are unnormalised (e.g.. logits). That is, as suggested by Koh et al. (2020), we indicate a concept C; is active by setting the neuron aligned to its score to the 95th percentile value of that neuron in the empirical training distribution. Similarly, we indicate a concept is inactive by setting that same neuron's 's output to the 5th percentile of its empirical training distribution.
DNN As a representative of black-box models, in our evaluation, we included vanilla Deep Neural Networks (DNNs). To ensure fairness in terms of capacity with respect other baselines, we implemented DNNs using the same architecture as Hybrid CBMs in each task but setting the concept weight to O. This ensures this model only learns its downstream task using the same architecture provided to equivalent Hybrid CBMs. Therefore, as in Hybrid CBMs, we fine-tuned the number of extra dimensions k' in the bottleneck that follows the backbone, selecting a value in k' E 50, 100, 200}, and set the activation function of the entire bottleneck to a vanilla leaky ReLU function. Our model selection chose k = 200 for CUB, CUB-Incomplett AwA2, and CelebA while it chose k' = 100 for AwA2 Incomplete and CIFARIO.
MixCEM For MixCEMs, we fine-tune the weight of the prior loss Xp. the dropout probability of a concept embedding's residual during training Parep- and the number of epochs Eeal used for Platt calibration (Platt et al., 1999) on the validation set. All other hyperparameters that are inherited from CEMs, such as the probability of training intervention Pint the concept loss weight λe. and the embedding size m, are always set to the same values selected for CEMs on the respective task. As for our other hyperparameters, our model selection yielded: (a) Ap = 1 for all tasks except for CIFAR1O, where we selected Ap = 0.1, (b) Pärup = 0.1 for CUB- Incompll lete and the Trave i ngBi ras-based tasks, Paro = 0.5 for CUB, AwA2, and CIFAR10, 0, and Pdrop = 0.9 for Ce CelebA, and (c) Eeal = 30 for all tasks except for Ce Celeba, where model selection chose not to perform Platt Scaling (i.e., Ecal = 0). Finally, we always fix the number M of residual dropout samples we generate at inference to M- = 50.
In Appendix J, we show ablation study for model's hyperparameters that suggests its ability to achieve both CA and BI is preserved across several hyperparameterisations. Nevertheless, noticing that (Ap: Pirop, Eeat) = (1, 0.5, 30) were by far the most often selected hyperparameters, we recommend using these as the default values of MixCEM's hyperparameters if there are no resources for fine-tuning its hyperparameters.
D.5. Bayes Classifier
To determine whether or not all baselines achieve bounded intervenability, we wish to approximate the accuracy of a Bayes Classifier that takes as input any set of ground-truth concept labels cs and predicts arg ma .L} P(y = es). As learning a model for each possible concept subset S C {I,.... ....., .k} the Bayes Classifier via a Multilayer Perceptron (MLP) r/(c): (o, 1|k + [o, 1]4 and training it to minimise Lask ()))). For this model to support any arbitrary concept subset as an input during inference, however, we randomly mask its input concept vectors e during training by setting any input concept to 0.5 with probability p 0.25. That way, given the labels cs of any subset of concepts S, we can estimate P( cs) by predicting 7((c((S), where we let c' (S) E [0, 1] be a vector such that c'(S) = if i E S and c'(S)= = 0.5, otherwise. In practice, across all tasks, we train this masked model for EBayes = 75 epochs using an MLP with hidden layers [28, 64, 32) and leaky ReLU nonlinearity in between them.
is intractable, we approach this problem by approximating
21

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
E. Results on Variations of Vanilla CBMs
As discussed in Section 2, Vanilla CBMs can be trained jointly, sequentially, and independently. Moreover, the jointly trained version of Vanilla CBMs can use a bounded sigmoidal activation for its bottleneck, or it can use unbounded logit scores. In the latter case, where the bottleneck contains the logits of the concept they predict, these scores can be intervened on using the 5th and 95th percentile values of their respective training distributions as suggested by Koh et al. (2020). Here, we investigate how various training regimes for Vanilla CBMs impact their ID and OOD intervention performances.
CUB
CUB- CUB-Incomplete
AwA2
Aw AwwA2-looue Incomplete
CIFAR10
CelebA
3900
7V
ID Acceraay (16)
97.
850
100
DOD Accmrey (88)
36
103
20
30
10
Interveted Cancepts (80)
Intervened Conoepts (19)
Intervened Concepts (96)
Intervened Concepts (96)
Interomed Counepts (85
Incervened Concepts (5)
Vanilla CBM
Joint Logit CBM
Sequential CBM
Independeuu CBM
MisCEM (ouzs)
Bayas Classifiet
Figure 9. Task accuracy of several Vanilla CBM variants as we intervene on concepts, selected at random, for ID (top) and OOD (bottom) test samples. Wc use the same sctup as the experiments described in Figure 4.
In Figure 9, we show the results of intervening on several variations of Vanilla CBMs across all tasks and compare these to our MixCEM's results. These results, which follow the same setup introduced for Figure 4 in Section 5, show that MixCEM outperforms all CBM variations when intervened on for ID and OOD samples. In particular, we observe that MixCEMs significantly outperform all variants of Vanilla CBMs in our muncpptincoomplte tasks as, in contrast to Vanilla CBMs, MixCEMs are completeness-agnotie The closest variant to MixCEM in terms of its unintervened performance is the jointly trained logit CBMs, as their logits are leakage-enabling activations in the bottleneck. Yet, as seen in particular in the cancept-incompler tasks such as CUB- In--Iroooplett and AwA2 incomplete. these models are unable to properly react to interventions in these setups, instead decreasing their accuracy the more they are intervened on.
F. Scalability Studies
Below, we describe series of experiments to measure the computational impact of MixCEM's architectural components on MixCEM's training and inference times.
F1. Training Times
In Table 4, we show the efficiency of each method expressed as the number of wall-clock seconds taken per training epoch across all tasks. We emphasise that these results are likely biased as they depend on implementation. Moreover, they are prone to high variance due to our hardware infrastructure (we train models on shared machines whose latency may be affected by concurrent processes). Nevertheless, our results suggest that MixCEM's training times are significantly faster than those seen in IntCEMs and ProbCBMs. This is due InCCEMs incorporating an expensive sampling-bars objective, which MixCEMs avoid completely. As expected, MixCEM's times are slower than CEMs given their introduction of new mechanisms top of CEM. Nevertheless, we believe this performance hit is not significant and can be considered to be amortised at inference time if one considers that MixCEMs are much better at receiving concept interventions than CEMs both for ID and OOD samples. Finally, we observe that as the number of concepts increases, the training times of ProbCBM rise significantly, even for similar sample sizes (e.g., AwA2 and CIF AR1 This is highly suggestive that these models may not properly scale to large concept bases and suggests that MixCEMs, whose performance remains relatively stable even when the number of concepts is high, can properly scale across concept set sizes.
training
22

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Table 4. Efficiency study showing the training time per epoch (in seconds) for all embedddigg-bee baased haselines.
Method
CUB
Coco-Inomplle lete
AwA2
AwA2 Inc2-Inooeeeeee
Cifar10
CelebA
ProbCBM
67.9046.51
26.0842.67
182.38+6.00
121.07+22..0
125.622156.10
11.1440.12
CEM
30.3412.87
26.21 11.49
82.44 19.38
108.82 12.63
32.37 10.32
8.15
IntCEM
53. 53.7548.81
35.57
149.9648.94
144.18
93.85 1.58
11.574041
MixCEM
50.03
26.00
110.94
97.9616.81
59.92
7.7340.02
E2. Inference Times
To complement our discussion on MixCEM's training times with respect to competing baselines, in this section we explore the effect of MixCEM's 's components on its inference time. Table 5 shows the inference wall-clock times of all baselines in CUB. Although we observe a slight increase in inference times of MixCEMs with respect to CEMs (-..2% slower), we argue this difference not problematic, considering MixCEM is designed to be deployed together with an expert who can intervene on it. In this setup, we believe that less than millisecond of extra latency should not bear too heavy a toll, as post-intervention accuracy more important.
Table S. Efficiency study showing the inference time per sample (in milliseconds) for all baselines CJB.
Vanilla CBM
Hybrid CBM
ProbCBM
P-CBM
Residual P-CBM
CEM
InCCEM
MixCEM
Time per Sample (ms)
1.386
1.394
5.687
1.426
1.433
1.437
1.431
1.497
G. Image Noising Details
Across all of our experiments where noise is used generate OOD test samples (e.g., Figure 4 and Figure 5, left), form of "Salt & Pepper" noise. Given a noise strength factor λ E [0, 1J, this noise sets randomly selected fraction of 21/2 pixel channels, selected with replacement for efficiency, to 255, their maximum value. Then, it sets \r/2 randomly selected pixel channels, selected with replacement for efficiency, to O, their minimum value. This leads to the resulting image having at most λ of its channels corrupted and to them becoming darker as the noise level increases. If the noise level not specified, then we use \₁ = 10% as the default level. We note that we use this version of Salt & Pepper noise as allows us to efficiently test its effects on models across large datasets using noise that similar that found in real-world scenarios (Azzeh al., 2018). A visualisation of CUB images with different levels can be seen in Figure 10.
Original
= 10% de = 23% de = 50% A 75% X 90%
Figure 10. Examples of random images in CUB with our form of "Salt & Pepper" noise as we vary the noise's 's strength level.
H. Interventions on Different Forms of Distribution Shifts
In this section, we explore different forms of distribution shifts beyond those studied in Section 5. Our results below strongly suggest that the improvements in intervenability described in Section 5 for MixCEMs can be seen across multiple forms of distribution shifts.
23

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
H.1. Exploring Different Forms of Visual Distribution Shifts
First, we explore visual distribution shifts caused by different forms of transtormations besides the Salt Pepper noise studied in Section 5.2. Specifically, in this section, evaluate interventions samples that were downsampled, baussian-blurred, and subjected to random affine transformation (including rescaling and rotation). We chose these distribution shifts as they represent widespread forms of OOD shifts found in real-world deployment.
Our results on CUB- Incompl ete and our AwA2 tasks, shown in Figure 11, suggest that MixCEMs have better OOD intervention task accuracies than our baselines across different distribution shifts. For instance, MixCEMs can have up ~20% percentage points more in OOD task accuracy than CEMs and IntCEMs when all concepts are intervened on in inputs downsampled to 25% of their size. These results suggest that MixCEMs are better receiving interventions in practical scenarios and real-world forms of distribution shifts.
Salt & Pepper (10%)
CUB- NUB-Inconplete
AvA2
Ax82 Ial--Iacepplste
GOD Aceranryy (36)
Random Affine
OOD Acctrary (76)
100
OOD Accuracy (36)
Motion Blur
COD Aomrncy (06)
100
Resize (25%)
OOD Aortney (16)
Resize (50%)
Isterveed
Concepts (56)
Intervened Concepts (%)
Interveed Concepts (36)
Vanilla CBM
CEM
ProbCBM
Residual P-CBM
Hybxid CBM
1nCEM
P-CBM
MisCEM (ours)
Figure Task accuracy as we intervene on concepts, selected random, OOD test samples on CUB Incomp lete and our AwA2 variants. Out-of-distribttion samples are generated by applying different forms of visual transformations to the test set (shift type shown on the y-axis).
H.2. Exploring Domain Shifts
Next, we explore distribution shifts in the form of domain shifts. For this, we train our models addition task where 11 MNIST digits (Deng, 2012) form each training sample, and the task is to predict whether all digits add to more than 25% of the maximum sum. We provide the identity of five digits as training concepts (i.e., an incomplete task), and at test time, we swap MNIST digits for real-world sampled digits coming from the Street View House Numbers (SVHN) dataset (Netzer al., 201 1). Our results, shown in Figure 12, suggest that MixCEMs achieve better ID and OOD intervention
24

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
task AUC-ROC than our baselines, particularly for high intervention rates. For example, when all concepts are intervened, MixCEM attained ~31, ~7, and ~3 more percentage points in OOD task AUC-ROC over CEM, InCEMM, and ProbCBM, respectively, In contrast, we found it very difficult to get CEMs to perform well in this incomplete task.
ID (MNIST Digits)
OOD (SVHN Digits
Task ROC-AUC (%)
90
80
80
70
70
60
60
30
0
50
100
50
100
Intervened Concepts (%)
Intervened Concepts (%)
CEM
ProbCBM
IntCEM
MixCEM (ours)
Figure 12. Task accuracy as we intervene on concepts. selected at random. for models trained on the digit addition task. Models are trained using MNIST digits. However, when we evaluate OOD interventions, we use test samples whose digits are drawn from the SVHN dataset.
I. Extended Robustness Experiments
We complement our results in Section 5.3 by showing extended versions of those experiments. We first discuss our extended noise level ablation results on CUB-Incomplete and CUB and then our baselines' results on SravelingBiros
L.I. Extended Noise Ablation Results
Figure 13 shows all intervention curves in CUB and CUB- Tncomplteee as we vary the amount of "Salt & Pepper" noise on the test samples. As in our analysis of Figure 5 (left) in Section 5.3 suggests, we see that across noise levels, MixCEM outperforms all baselines terms of its task accuracy as interventions are made. MixCEM's placement within the other baselines appears to be independent of the dataset's concept completeness, although we observe significantly greater improvement in CUB-Incomplee eete than in CUB. Moreover, we also observe that, as discussed in Section 5.3, MixCEM's unintervened performance (left-most part of the plot) is above that of other methods for noise levels of up 10% corruption. After those levels, MixCEM's unintervened accuracy seems to be on par with that of all other baselines, but it better receives interventions than other approaches (leading to higher accuracies when intervened on across all noise levels).
1.2. Complete Spurious Study on Travel ingBirss
In Figure 14, we summarise the intervention curves for all baselines in Trave siweeiingirds Inw.rdseee.eeeeeeeeeeeee Here, as we observed in Section 5.3, we see that MixCEM's performance for spuriously correlated ID samples is on par with that of IntCEMs and CEMs, the two best-performing baselines for ID samples in both tasks. contrast, for OOD samples, we see that ProbCBMs, Vanilla CBMs, and Hybrid CBMs have better OOD unintervened performance than MixCEM only in TravelingBi in Section 5.3 and constitutes evidence for the hypothesis we discuss in Section 6 that argues that using constant/global concepts in bottleneck, such those used for Vanilla CBMs and ProbCBMs, enables them to avoid exploiting spurious correlations in the same way less constrained methods such as CEMs or IntCEMs may. These results also follow similar conclusions previously discussed in this dataset by Koh et al. (2020). Nevertheless, we observe that (1) MixCEM's 's accuracy is higher than that of all baselines when the dataset is incomplete, our main setup of interest, and (2) MixCEMs intervention accuracy is properly bounded (always above that of the Bayes Classifier), something that cannot be said of any other method in these two tasks except for IntCEMs.
and its incomplete version
This aligns with the results we reported
25

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Noise
Nuiee Levti
1065
Noise
81
(CTD look 5e (80
Edenesal Chunegis
Comppts
Intervood Choeegts (4)
(5)
Vaila CBM
Boland CDM
CEM
FradCBM
P-BB
Rosiral P-CBM
MecCEM (azzs)
Darr Clastder
Figure 13. OOD task accuracy vs intervention curves for all baselines in the CUB (top) and CCm-Incomplete tasks. Test samples are perturbed by adding "Salt & Pepper" noise with increasing levels (% pixels corrupted).
TravelingBirde
TravelingBirds -ir.-innpBiddI-cccpeeeeeee
ID Took Accurscy (%)
100
80
60
60
OOD Tosis Aucaraey (36)
80
60
60
60
29
60
120
20
100
Concepts Intervened (%6)
Concepts Intervened (%)
Vanilla CBM Hybrid CBM
InCEM
Residuud P-CBM
PrabCBM
MixCEM (ouns)
CEM
P.CBM
Bayes Clussifier
Figure 14, ' nighi rds (left) and l'ravel 1 i inghi rds- - incomp Semmplete et (right) intervention curves. We show results on a spuri- ously correlated validation set (top) and on an OOD test without the spurious correlation (bottom).
J. Hyperparameter Recommendations and Ablation Studies
In this section, we thoroughly examine the impact of MixCEM's hyperparameters on its performance. For this, we focus on the CUB-Incomplet. dataset, given that it is good representative for concept-incomplet task, the main setup of interest for this paper. Moreover, to enable a tractable exploration we train MixCEM only with 25% of the training data. The only exceptions for this are our studies of the effect of (I) random train-time interventions (Appendix J.5), and (2) concept calibration (Appendix J.4), where we conducted experiments across all tasks, as these studies could be performed efficiently. Finally, when studying specific hyperparameter, we vary its value over set of pre-defined acceptable values at different scales while fixing all other hyperparameters to the values selected by our model selection procedure for CUB Incomplete (described in detail in Appendix D).
vast number configurations for the ablations below,
Our results in this section strongly suggest that MixCEMs are robust to different hyperparamettrisation.. attaining both high intervenability and fidelities for both ID and OOD samples across all hyperparameterisations we attempted. As
26

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
such, MixCEMs do not require significant efforts fine-tune, making them practical in real-world setups where proper fine-tuning may be intractable. To facilitate MixCEM's future use, include a set of hyperparameter recommendations in Appendix J.6.
J.1. Effect of Concept Weight Loss (Ac)
First, we evaluate the effect of the concept loss weight A₂ MixCEM's performance. Although, in practice, do not fine-tune this hyperparameter for MixCEM (instead always using the concept loss weight selected for an equivalent CEM), this hyperparameter has been previously shown to significantly affect how CBM-based models perform (Koh et al., 2020; Espinosa Zarlenga et al., 2022). As such, understanding how it affects our model is an important practical consideration.
In Figure 15, we show the ID and OOD intervention curves for MixCEM in SOBB-Incompeeee across several values of de E {0, 0.01, 0.1, 1, 2.5, We see that, throughout all values of Xe MixCEM significantly increases its accuracy when one intervenes on its concepts both in ID and OOD test sets. Although these results appear be somewhat stable, we see, as one would expect, that the unintervened task loss suffers drop when de is high and the mean concept AUC drops when A is too low. This suggests that, in practice, using a midrange value for X in between [1, 2.5] yields an intervenable model with high concept and task fidelities.
A surprising result from this ablation is that MixCEM is still highly intervenable when A. = 0 (meaning no concept loss is applied during training). We believe that this is a consequence of MixCEM's prior predictive error Lusk (y, I f()) maximising the task predictor accuracies assuming all concepts are intervened on. Hence, these results provide further
evidence that La (y, (c)) has an implicit intervention-aware effect.
OOD
Thak Accurain 190
Mom Conoggt Allic (50)
Intervenrdd Auc (56
Acrmury
Intervenel
30
63 Intervecued Conrepts (96)
30
25
Intervened Cuncepts (X)
Ag
XacCEM (A_ 0.0)
MECCEM (2,
MIXCEM (Ac 1.0
M&CEM (2.
MICCEM 5.30
Figure 15. Ablation study for A in a smaller version of CUB CUB-Inee IIDD-Incoppetee On top of each subplot, we indicate whether we show results for an ID test an OOD test (generated using 10% salt & pepper noise). The right-most two plots show the task accuracy when all concepts are intervened. We highlight the model corresponding to the hyperparameter selected for our evaluation in Section 5.
J.2. Effect of Prior Error Weight (Ap)
Next, we examine how the prior error weight Ap affects MixCEM's performance. In Figure 16, we observe that small values of Xp result in both worse ID and OOD intervenability. This is expected, as the larger Ap is, the better the model's accuracy will be when making predictions based only on the global embeddings. Nevertheless, we see that for larger values of Ap (e.g., Xp 2 1), all metrics become relatively stable. As such, it is important that A, is set to a value near 1. By doing so, during training we are assigning equal weight to correctly predicting task labels using only the global embeddings (i.e., minimising Luskk (y. (()) and to correctly predicting task labels with the contextual embeddings (i.e., Lusx (v. I (g(x)))
J.3. Effect of Training Fallback Probability (Parop)
When looking at the effect of the dropout probability Purop. Figure 17 suggest that, at least in CUB SUIB-Inconn Incompleteee this hyperparameter does not have much of an effect. This true except for Parop = 1, where we see drop in unintervened task accuracy. This is because when Pdrop 1, the model essentially blocks any leaked information from x into the label predictor f. However, for all other values, MixCEM's performance remains relatively stable with purop 0.5, yielding the best overall intervenability results, albeit with slight difference.
Surprisingly, however, even when Prro = 0, meaning we do not use any residual dropout during training or testing, MixCEM achieves very high task accuracies when it is intervened on for ID and OOD samples. This may suggest that this dropout mechanism is not always needed. Nevertheless, as our results in Figure 18 comparing a MixCEM with dropout and a MixCEM without dropout in Cel CelebA show, there are clear benefits of adding this dropout mechanism, particularly for
27

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Moaz Conrge AUC (SC
Lntervaued Are (50)
Tiell Avcnnang (K)
Avcunucy
Iatervsuel
8
70
01
13
11.
01
87
03
10
03
1.
100
Intervened Cuncepts (56)
Interveuud Connrpts (36)
Ap
MicCEM (A,
MIxCRM 0.3)
MisCEM
MESCTEM
MiCCEM (A,-
Figure 16. Ablation study for A in smaller version of CUB- Incomplete. On top of each subplot, we indicate whether we show results for ID test an OOD test (generated using 10% salt & pepper noise). The right-most two plots show the task accuracy when all concepts are intervened. We highlight the model corresponding to the hyperparameter selected for our evaluation in Section 5.
OXD
13)
120
Nateovored Anc (56)
10)
10)
Mose Chnorgee AUC
Duck Armunny
20
29
00
Interwmmed Concepti (50)
Intersaned Crnorrts (96)
MrCCEM (Nin 0.0)
MixCEM (paug
MIxCEM (Ning 0.5)
MIxUTEM (riig
MIxCEM (Moig 1.0)
Figure 17. Ablation study for Prop in a smaller version of CUB- -UC-IIncoplete. On top of cach subplot, we indicate whether we show results for an ID test or an OOD test set (generated using 10% salt & pepper noise). The right-most two plots show the task accuracy when all concepts are intervened. We highlight the model corresponding to the hyperparameter selected for our evaluation in Section 5.
difficult tasks such CelebA.
OOD
OOD
(95)
(56)
Thee Morunn (90
(56)
Moar Conage ADC.
Intervened Concepts
Intervened Cancepts (%6)
MrCEEM (peve in (.))
MrCEM (pwup 0.1)
MrCEEM (p... 0.3
MiCCEM (paw U.S)
Figure 18. Ablation study for MixCEM's 's residual dropout probability Purop hyperparameter in Ce ebA. Notice that in this task, there a significant improvement in OOD intervenability when paop is greater than O. We mark red and bold the model corresponding to the hyperparameter that was selected for our evaluation in Section 5.
J.4. Effect of Calibration (Eea))
In Figure 19, we can see the intervention curves MixCEMs across all tasks with and without Platt scaling. We notice that Platt scaling brings some key benefits for OOD samples, particularly for complex datasets such as CUB Incomplete and IwIInnooopleeeee This is because the more calibrated MixCEM's concept probabilities are, the more likely it is to drop its residual embedding when that concept becomes OOD, The only instance where we saw drop in performance when using Platt Scaling was in CelebA. We believe this is due to the concepts in this task being too complex/subjective to be properly predicted in the first place, leading to a model whose concept predictions were not overconfident even before Platt scaling was done. Nevertheless, we notice that with and without Platt Scaling, MixCEMs can recover very high accuracies when intervened for OOD setups. Hence, Platt scaling is helpful but not entirely necessary for MixCEM's ability to receive interventions for OOD samples properly. Because of this, when selecting hyperparameters for MixCEMs, we include Fecl = 0 (i.e., no calibration at all) as a hyperparameter option.
28

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
CUB
CLB Insmuplle
Awdd
AncA Insoroltte
CIFAA10
CHiMA
in: Sanming (M1
(9)
Intevssal Iateween Goasgle
Insward Chenggia (8)
Liteered Caaopes [R]
Unneaal (R)
Istewadd Gaapplo 05)
beavod Chorgeo 159)
X
MicCRM (Su Callicalins)
MixlRMM (Pist Chalionia))
Figure 19. Task accuracy of MixCEMs with and without Platt scaling as intervene on concepts, selected at random, for ID (top) and OOD (bottom) lest samples. We use the same setup as the experiments described in Figure
J.5. Effect Training Intervention Probability (Pint)
Finally, we study the effect of training-time interventions on MixCEM's performance across ID and OOD tasks (so-called RandInt (Espinosa Zarlenga al., 2022)). Our results, shown in Figure 20 suggest that randomly intervening on concepts during training with Pint 0.25 is generally beneficial for ID interventions. Nevertheless, observe that the improvements from including these train-time interventions on MixCEMs are significantly less impactful than what the original CEM authors observed for CEMs (Figure 6 of (Espinosa Zarlenga et al., 2022)). This is once more evidence, as discussed in Section 6, that MixCEM's s prior error minimisation has an implicit intervention-awee bias in it that leads to models that are more receptive to test-time interventions.
CUB
CUB- B-lioompptte
Awi2
AndA2 lnsonctte
CHELID
CHiMA
Accuaay (99)
9
30 Checerte (%)
Cooepte (5)
Giveonad Conpoee [X]
Inteoradd Comppe (X)
Cooeppts
Cocopea
McXCEM (av 0j
McXCEM (Ner 0.20)
Figure 20. Task accuracy of MixCEMs with and without randomly intervening at training time with probability Pin (i.c., using RandInt). We use the same setup as the experiments described Figure
J.6. General Recommendations
Our ablation results suggest that MixCEMs can perform well both for ID and OOD instances across wide range of hyperparameters. Thus, if fine-tuning is not option, we would recommend setting A 1, Párop = 0.5. Ap 1, m = 16, Pin = 0.25, and T = 50 for obtaining already high ID and OOD intervention receptiveness. If one of these hyperparameters to be fine-tuned, our ablations suggest setting A = 1 and focusing on fine-tuning λ. as changes in this hyperparameter affect MixCEM's interpretability the most.
We note that in experiments, focus almost entirely selecting >p and Purp- All other hyperparameters (e.g., m, Pints T') were either fixed to constant value or were selected based on those used for an equivalent CEM.
K. Learning Intervention Policies In MixCEM
As discussed in Section 6, because MixCEM utilises embedding representations for its concepts, we can train MixCEMs using the same intervention-awar pipeline used for IntCEMs (Espinosa Zarlenga al., 2023a). This would enable
29

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
CUB
CUB-Incomplete
95
ID Accuracy (%)
90
90
80
75
25
50
75
100
0
25
50
75
100
100
OOD Accuracy (%)
80
60
60
40
20
20
25
50
75
100
0
25
50
75
100
Intervened Concepts (%)
Intervened Concepts (%)
CEM (Random Policy)
MixCEM + IntCEM (Random Policy)
MixCEM (Random Policy)
IntCEM (Learnt Policy v/))
IntCEM (Random Policy)
MixCEM + IntCEM (Learnt Policy vtc))
Figure 21. Effect of integrating IntCEM's training procedure into MixCEM. Intervention curves for ID (top) and OOD (bottom) test sets across our CUR-based tasks for learnt policies across intervention-as aware models. As a haseline, we include intervention curves for CEM and MixCEM using random policy. Otherwise, show the curves produced from IntCFMMsstyl learn policies the using solid lines with stars. MixlntCMM corresponds a MixCEM trained with InCEM's training-time sampling mechanism, enabling the model to learn intervention policy the as by- productt..
MixCEM to learn an intervention policy during training, which can then be used at test time express preference over interventions.
In this section, we investigate the impact of incorporating MixCEM's training procedure, which involves sampling interventions at training time from a learnable policy the into MixCEM's training objective. We note that, due to computational constraints, in this evaluation we did not perform a hyperparameter search over hyperparameters introduced when using InICEM's training pipeline for MixCEM (e.g., A and 7) or over any of MixCEM's hyperparameters (e.g., Ag- Ap). Instead, fix all hyperparameters of this hybrid model, which combines MixCEM and IntCEM, to those used for the equivalent MixCEM IntCEM in the evaluation task. This may lead to results that are skewed against MixIntCEMs, as they were not explicitly fine-tuned. We summarise the results of our best best-performing baselines in the CUB-based tasks, and show intervention curves when intervening following (1) a Random Policy and (2) the corresponding model's learnt policy t/c (if applicable).
Our results in Figure 21 suggest that incorporating InICEM's training procedure into MixCEM (a model we call MixInCEM) does indeed lead to model that learns intervention policy the that is significantly better than a Random Policy (as seen when looking at the differences between MixIntCHM's 's curves for the "Random Policy" vs the "Learnt de Policy"). More importantly, we see that this benefit comes without sacrificing random intervenability in MixCEM, both for ID and OOD test sets (as seen by the almost identical random intervention curves for MixCEM and MixInCCEM). More importantly, how- ever, we see that CEM'ss intervention policy yields better intervention curves than all other basclines for OOD test sets.
30

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Nonetheless, we also observe that, for some instances, intervening on IntCEM using its learnt policy yields better intervention curves than those on ID test sets for complete datasets (i.e., CUB), suggesting there is still some value in maintaining the original IntCEM pipeline if we know we are operating in a complete task where test samples will not go OOD. Considered together, these results suggest that, although MixCEM can learn a powerful intervention policy that works both for ID and OOD test sets, there are still benefits in using the "pure" IntCEM model in deployment if we know the model will be trained on a complete training set and evaluated uniquely on ID test sets.
when we intervene on MixIntCEM using its own learnt policy. This is seen only when intervening
L. Resources Used
Software For our experiments and evaluation, we adapted the original CEM codebase" built by Espinosa Zarlenga et al. (2022). This codebase provided the basis for our implementation of MixCEM and gave us the foundations for our implementations of CEMs, IntCEMs, Vanilla CBMs and Hybrid CBMs. Moreover, we used the data loaders for CUB and CelebA provided by this codebase, of which the former is based upon the original data loader by (Koh et al., 2020). For Posthoe CBMs (P-CBMs), we based our implementation on that used by the authors and published with the paper*. Similarly, for Probabilistic CBMs, we based our implementation on very close adaptation of the implementation made public by the authors in the codebase accompanying their paper'. Finally, for our CIFARI0 and AwA2 loaders, got inspiration from the public implementation of these loaders by Vandenhirtz et al. (2024) and Marcinkevićs et al. (2024), respectively.
Our experiments were run on PyTorch 1.1 (Paszke et al., 2019) and facilitated by PyTorch Lightning 1.9.5 (Falcon, 2019). For our plots, we used matplotlib 3.5.1 (Hunter, 2007) and the open-sourced distribution draw.io.
All the software and datasets used to build our code and run our experiments were made available to the public via open-source licenses (e.g., MIT, BSD). To facilitate and encourage the reproduction of our results, we include all of our code, including configuration files reproduce each of our experiments, in our supplementary submission for this work. All of our code, including configs and scripts to recreate results shown in this paper, can be found in CEM's official public repository found at https//ggghub.coommmttttessppppe
Resources We executed all experiments on a shared GPU cluster with four Nvidia Titan Xp GPUs and 40 Intel()) Xeon(R) ES-2630 v4 CPUs (at 2.20GHz) with 125GB of RAM. All of our experiments, including early development experiments, expensive ablation studies, and fine-tuning for all baselines, took approximately 500 compute hours.
M. Tabulation of Intervention Results and Computation of Area Under the Intervention Curves
In Table 6, we show a tabulation of Figure using representative subset of the points shown in that figure. We also include an estimation of the area under each intervention curve (computed using a Riemann sum), which shows how MixCEM's intervenability is significantly better than that of competing methods, particularly for OOD setuss.
"haps://gituu..com//n http:///ittbbboommmacccceiisseee
31

---

Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts
Table 6. Task accuracy (%) for ID (bluc) and OOD (red) samples as we intervene on larger fraction of randomly selected concept groups. These results show the same data as Figure 4 but in more accessible form. Each task's best result, and those not significantly different from it (paired t-test, = 0.05), are in bold. We note that MixCEM's unintervened accuracies (i.c., 0%) may differ very slightly from those Table as its inference is non-dcterministic (sce Section 4). We show the estimated area under each intervention curve in the
"AUC" column.
Mechod
0%
20%
60%
89%
Vanila
07.27_1.4
27.38_...3
Mirhbacas
Hybrid
PoOCEM
08.1822..7
70.88
90.1111..
100.00
90.55.139
P-CBMM
15, 15.t5a
51.00 St.llacm
54
TT
83.55 53.51.ra1
K3
Rocikad P-CBM
4:35aam
KS S5.50a
738 28.50
(I 96.lasum
50.30a
23
Biges Clasafiar
0.50 / /3.5
8.81
9.33aean
40.5)
5t St.lluus 7 (55.00 55.40laomm
56 38.08amm::
588 G8
100 na3.
100 100000,,0
55 55.62amm
CEM
76 11.07 8.0821.0
55 10
12 332.8:2541
52.00 22.00un11/
56000go0 555 10.077200
07 R1.0aat
/ 803 10
98, 11
St. Se.ooo
02.00ur
06 06.50e3
IaCEM
73 23.33amm., / m1
No 23
57 05
87 20
90.0773
DS Ti 2 86 24
00 Gl.zmm/ 88 98-97ax
101 90200
94 s4.08.nm//
50 19
MicCEM (oan)
70 64 o.22 / 17:25 (17:25.0.3
59.19 or
02.24...43
Di i1
30
n6.4bacer
99.77
99.61.011
99 10
/ 99.20 13
94.69
52 52.53.111
Yanilla CBM
56. Me e 3.47aa
61.Maaun7
20 Slas
22 Ma
To 10:30
66 41
7a Sa
35 Ranee
Hybrid CBM
72.882380 57 4.8 t0
70.311a79
55 S3.3ions/
55 G04
/ 18 18.08
34.88ooua
ProSCBM
00.56g 1 4.20g 30
67.50
7150
20.87.104
20.00
47
20
88 20,000
P-CBMM
2220
ao-thussan
Resisaal P-CBM
24.2 4.82..m
12.25am
Bayes
con
7.82
1846
Hyliid CBM
54.30rm0
50 36
00 78
100
100
03P52.30 24:22 1412-Inc- Inc.rrioottt 24-22
P-CBMM
19 29.7
(91
t9
13.35
Resddall P- P-CBM
00.00
/ 19.44 119449...1
4224
T1 11.721155
p5,35
09.3
99.75
99
Biges Clasitiar
1.38.100 / 11
12.08
12258.00
55 16
50.30
00.00
100 oo.
100 100,002000
CEM
91. 1T
20 22
56 Maacn
20 22
3t Baala
10
MixCEM loan)
80.00
17.70.
05.45
45.51 0.3
Us 95
00.02
99.95 1935
99.00
99 us
100,00
100 100.00.com
97.92
Vanile CBM
10.0 100
11.00 11:00
Hybiid CBM
30.23ga.a
18:1
50.75
22
1888
20 10
28 13
21.45
38
20
P-BBM
25. Tanta
14.20
Residuil P.CBM
7
37
05
9020.0
in
70000
Bayes
BMC
58.88
MiCCEM
26.77.000
45.88...
74:44:044
50
Vanill CBM
30.3
Ryhrid CBM
70.12a
20.3
10. 10.iias
ProCCRM
27.
57 22
T8.38
5t7basas
50.9
50 18
80 20
P-CBM
79.80
/3321.m17
55.0
5S
7822
24424
Reidaal P- P-CBM
79.90
83.91inss
55.48
56 54
55.00
T0.57
TRBLioos 14122
75,00
74388.04
Bages Clasciter
63388
50 20.
13 (B 12.522n1
70 MS5aaan 76 15.-aaa
CEM
M0.15 00.
32 33
1222
st than
T1 32
3yCEM
2 32.41.aa0
85 camm
d5-Baass
an 1.000.mm (G.Daaas
90.57 an.57
Thil as
02 02
02 30
MicCEM (ours)
mo 52 T0 es
63
(5:75 41
82 5l
97
Vantia
Uyyris
13 22 10 73,5 es
36 a0
36
coldbd
P-CBM
17.18
/ 1.700.100
17
160722 1900123
23.5 2317
2000:20 10
42.27
Residual P-CBM
n
10
16:1:: 7717ass
21
CEM
22222.14
IxCEM
14.35
50.05
50.30 22.88ia
09.541175/
59.57
MicITM (rrr))
11
51.17mass
18
22 22
42.55
B2 Shan
53.21.
31
32


Reviews:
[{"review_id": "t5cWRSRpJT", "reviewer": "Reviewer_At2h", "summary": "This paper proposes a novel concept-based model called mixCEM. The authors point out the problem of leakage poisoning, where as concept interventions become out of training distribution, the task prediction accuracies reduce. The authors propose mixCEM which uses residual embeddings for positive and negative concept labels. These embedding are then mixed to create the concept predictor which further predicts the task.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for taking the time to go over our work and provide feedback. Your comments have helped us identify areas where we can improve our manuscript.\n\nWe are glad you found our paper “well-written” and its evaluation sound. Below, we reply to your feedback. If you have further questions or concerns, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your score after considering our replies.\n\n### **(Concern 1) CEM vs MixCEM’s performance (what is the trend?)**\n\nAs we point out in Section 5.1, “we do not expect MixCEM to be the best-performing baseline in terms of its in-distribution (ID) task and concept fidelity.”  This means that, in some datasets, CEM’s **unintervened performance on ID samples** is slightly higher than that of MixCEMs (at most 1.5% percentage points in task accuracy and 2% in concept accuracy). However, this slight drop in MixCEM’s unintervened performance is **statistically significant in only 3/6 tasks** (as determined by a paired t-test in Table 1). As we argue below, we see this as a reasonable trade-off considering MixCEM’s improvements in intervenability.\n\nSpecifically, our work aims to improve OOD intervenability while remaining competitive against strong baselines such as CEMs in ID cases. Our results demonstrate that this is the case: across **all** datasets, MixCEMs achieve (statistically) significantly better accuracies than CEMs when intervened on, both in ID and OOD test sets. For example, in CUB alone, when one intervenes on 20% of the concepts on the OOD test set, MixCEM achieves a task accuracy of 62.24 ± 3.43% vs CEM’s 32.58 ± 3.12% (**an absolute improvement of nearly 30% points!**). Even in the ID test set, at an intervention rate of 20%, MixCEMs attain more than 3% of absolute gains w.r.t. CEMs. This gap is even wider for other datasets: in AwA2, MixCEM **has an OOD task accuracy ~55% higher (in percentage points!) than CEM’s** when all concepts are intervened on both models. Similar results can be seen across all datasets (see Figure 4 or Table 5, where we also provide statistical significance tests). We believe these improvements are worth a potential hit of 1-2% in unintervened accuracy if one expects the model to be intervened on at test time, our deployment setup of interest.\n\nHence, the **observed trend** is the following: **the area under MixCEM’s intervention curves is significantly higher than CEM’s across all tasks, both in ID and OOD test sets (i.e., MixCEM is much more “intervenable” while remaining competitive without any interventions)**. You can see this by noticing that the red-dashed line (MixCEM) in Figure 4 **is always above** the green-triangle line (CEMs) when we perform one or more interventions.\n\nTo clarify our results for future readers, we will update Section 5 and the caption of Figure 4 to explicitly make these points.\n\n\n### **(Concern 2) Argument for why MixCEM mitigates leakage poisoning**\n\nIntuitively, MixCEM’s entropy gates work as OOD detectors that only let the residual component into an embedding when the sample is ID. Therefore, the embeddings used when intervening on OOD samples will have no residual components, meaning they will be “global” and not sample-specific embeddings. If no sample-specific information is allowed in the label predictor after an intervention, then leakage can't exist. This means that **by blocking leakage/residual information when a sample goes OOD, MixCEMs mitigates leakage poisoning**. Further theoretical motivation is provided in Appendix B.\n\n### **(Q1) Can you explain crisply why mixing of residual embeddings is helpful?**\n\nCrisply put, we do mixing at two levels: (1) we construct final concept embeddings $\\mathbf{c}_i$ by mixing *\"positive\" and \"negative\" semantic embeddings* $\\mathbf{c}_i^{(+)}$ and $\\mathbf{c}_i^{(-)}$, and (2) we construct each semantic embedding itself  $\\mathbf{c}_i^{(+/-)}$ by mixing a *global embedding* $\\bar{\\mathbf{c}}_i^{(+/-)}$ and a corresponding *residual embedding* $r_i^{(+/-)}(\\mathbf{h})$.\n\nThese two levels of mixing allow one to (1) perform interventions by simply hardcoding a concept’s final embedding at test-time to $\\mathbf{c}_i^{(+)}$, if the expert believes $c_i = 1$, and to $\\mathbf{c}_i^{(-)}$ otherwise, and (2) avoid leakage poisoning by dropping residual components of each embedding when the sample is OOD (therefore completely blocking any leakage from influencing the downstream predictor). Notice that by allowing the residual compontent of each semantic embedding to be partially dropped (as it is mixed using a continuous value), this architecture allows MixCEM to use these residual embeddings to communicate useful information to the label predictor **both** for ID and OOD samples (leading to models that are completeness agnostic).", "claims_and_evidence": "The authors claim the proposed model mitigates the leakage posing problem. However, the theoretical argument towards this is not clear to me. Also, the experimental results in table 1 and figure 4 show that performance of CEM is comparable to mixCEM. The performance of mixCEM is sometimes better for OOD samples. However, I don't see a clear pattern.", "methods_and_evaluation_criteria": "I am not completely sure, but looks like the evaluation method is standard.", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "The experiments look sound to me.", "relation_to_broader_scientific_literature": "The authors cite and connect with the literature in the area.", "essential_references_not_discussed": "None that I know of.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "The writing is good.", "questions_for_authors": "Can you explain crisply why mixing of residual embeddings are helpful?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l8AZ1FWzRF", "reviewer": "Reviewer_KTVL", "summary": "The authors present the first study examining the effectiveness of concept interventions under distribution shifts in interpretable concept-based models introducing the concept of \"leakage poisoning\", a phenomenon that hinders models from accurately improving when intervened upon for out-of-distribution inputs. To address this challenge, they propose the Mixture of Concept Embeddings Model (MixCEM), a new model architecture built upon recent concept embeddings models (CEMs). The proposed architecture is designed to adaptively leverage leaked information missing from its concepts only when this information is in-distribution.  Through a comprehensive evaluation covering both concept-complete and concept-incomplete tasks, the authors illustrate that MixCEM enhances accuracy for both in-distribution and out-of-distribution samples, regardless of the presence or absence of concept interventions. Moreover, it bridges the performance gap in state-of-the-art CEM models when handling out-of-distribution samples during concept interventions.\n\n\n## update after rebuttal\nI acknowledge the authors' response to my concerns, including the additional distribution shift experiment. Their clarifications show that the proposed methods effectively address performance gaps for out-of-distribution samples during concept interventions, enhancing explainability applications. Considering the contributions and the rebuttal response, my score is changed from \"weak accept\" to \"accept\", though something may escape my evaluation since I'm not an expert in concept-based models and their applications.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback! Your comments really helped us improve the quality of our manuscript. We are glad you found our work novel, “well-written”, and “well-structured”. Below, we answer your main concerns. If you have further questions, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your score given our replies below.\n### **(Q1/W2) Evaluation of other distribution shifts**\n\nIn our evaluation, we focused primarily on salt and pepper noise as it is a common real-world form of noise [[1](https://arxiv.org/abs/1903.12261), [2](https://link.springer.com/article/10.1007/s11042-016-3622-9)]. However, our methodology makes no assumptions on the type of distribution shift, as seen in our TravelingBirds experiments. To provide further evidence for this, **we carried out these new experiments**:\n1. **Other Noise Forms**: We evaluated interventions on samples that were *downsampled*, *gaussian blurred*, and applied a random *affine transformations* (rescalings and rotations). These are widespread noise forms in real images. Our [results](https://imgur.com/a/8SPvTd2) on CUB-Incomplete and our AwA2 tasks suggest that MixCEMs have better OOD intervention task accuracies than our baselines across different noise forms. For instance, MixCEMs can have up to ~20% percentage points more in OOD task accuracy than CEMs and IntCEMs when all concepts are intervened on in samples downsampled to 25% of their size. These results suggest that MixCEMs are better at receiving interventions in practical scenarios.\n2. **Domain/Context Shifts**: We followed your insightful suggestion of testing our baselines when a domain shift occurs. We trained our models on an addition task where 11 MNIST digits form each training sample, and the task is to predict whether all digits add to more than 25% of the maximum sum.  We provide the identity of five digits as training concepts (i.e., it is an incomplete task), and at test time, we swap MNIST digits for SVHN digits. Our [results](https://imgur.com/a/Duoto5p) show that MixCEMs achieve better ID and OOD intervention task AUC-ROC than our baselines, particularly for high intervention rates. For example,  when all concepts are intervened, MixCEM attained ~31, ~7, and ~3 more percentage points in OOD task AUC-ROC over CEM, IntCEM, and ProbCBM, respectively. In contrast, we found it very difficult to get CEMs to perform well in this incomplete task.\n\nWe will incorporate the results of (1) in a new Appendix summarised in §5.2 (where we will better motivate our use of S&P noise). We will discuss the results of (2) in §5.3, where they will complement our TravelingBirds experiments by showcasing MixCEM’s utility on different distribution shifts.\n### **(W1) Energy-based and Label-free CBMs**\n\nOur evaluation focuses on baselines that cover key directions in concept learning: we include methods that are embedding-based (CEM/ProbCBM), intervention-aware (IntCEM), scalable (Posthoc CBM), and “traditional” (CBM variants). Moreover we **include label-free annotation pipelines** via our CIFAR experiments. We believe our **ten** evaluation baselines and diverse training sets cover an overview of competing methods. Because of this, we decided not to include Energy-based CBMs as they were not designed with intervention performance in mind and are outperformed by approaches such as CEMs (see Figure 2 of the [ECBM paper](https://arxiv.org/abs/2401.14142)).\n\nLabel-free CBMs were not explicitly included in our evaluation for three reasons: (1) **their main contribution, i.e., using LLMs and VLMs for concept extraction, was included as part of our evaluation in our CIFAR dataset**; (2) in datasets where we have concept annotations (e.g., CUB), it is difficult to fairly compare label-free CBMs and concept-supervised methods as we do not have ground-truth labels for label-free concepts; and (3) Label-free CBMs were not designed with interventions in mind. This is exemplified by the original label-free CBM paper, which did not evaluate concept interventions.\n\nWe will update our Baselines section to justify these decisions.\n### **(Q2) Failure modes of MixCEMs**\n\nAlthough we discuss some limitations in §6, we agree that this section could examine potential failure modes. We foresee at least two limitations: (1) when a concept goes OOD and the shift renders the concept incomprehensible for an expert, MixCEMs may fail to completely block leakage poisoning as one cannot intervene on such a concept. Hence, future work can explore mechanisms for blocking all unwanted leakage without knowing a concept’s label.\n\nSecond, in incomplete tasks, intervened MixCEMs do not always recover the full ID performance in OOD inputs. Therefore, future work could explore how to better limit leakage, rather than entirely remove it, so that information about unprovided concepts can still be exploited after an intervention.\n\nWe will include this discussion in our Limitations section.", "claims_and_evidence": "The authors assert that existing CEM models struggle to manage both concept incompleteness and test-time interventions when dealing with out-of-distribution inputs, whereas MixCEM remains unaffected by these challenges. To confirm this claim, they evaluate the proposed architecture alongside different concept-based models (including CEMs) across varying proportions of intervened concepts in both in-distribution and out-of-distribution settings.", "methods_and_evaluation_criteria": "The proposed evaluation criteria make sense for the problem, but a more comprehensive evaluation of different out-of-distribution settings would strengthen the paper’s impact. The analysis encompasses both concept-complete and concept-incomplete datasets, as well as two additional datasets, one synthetic and one real-world, where concept attributes are derived from either CLIP-based classification or subjective human annotations. However, the paper lacks a broader range of distribution shifts (which is the focus of the paper) beyond salt-and-pepper noise. Regarding the method’s comparison, the study evaluates various models, including Deep Neural Network (DNN), Concept Embedding Models (CEMs), and Concept Bottleneck Models (CBMs).", "theoretical_claims": "The theoretical foundation builds on prior work in interpretability and concept-based models, so there are no specific theoretical proofs that require verification. The focus is primarily on methodology and experimental outcomes.", "experimental_designs_or_analyses": "The experimental design and analysis reported in the main paper are sound, though a more extensive evaluation across a wider range of out-of-distribution settings would further strengthen the study.", "relation_to_broader_scientific_literature": "The paper positions its contribution in the context of literature on interpretability and distribution shift in concept-based models. Specifically, the authors claim to be the first to examine the impact of concept interventions on out-of-distribution samples, introducing the term \"leakage poisoning\". They then propose a novel architecture, comparing it with existing concept-based models and highlighting how it advances prior work by directly addressing leakage poisoning within CMs.", "essential_references_not_discussed": "The paper provides a well-structured overview of various categories of CMs, including CBMs, their extensions, and CEMs.", "other_strengths_and_weaknesses": "Strengths:\n- The paper is well-written and well-structured, with clearly articulated contributions.\n- The paper proposes a novel approach to handling distribution shifts within CMs maintaining competitive performances across different scenarios.\n\nWeaknesses:\n- The experimental comparison lacks the inclusion of label-free CBMs and energy-based CBMs mentioned in the \"CBMExtensions\" paragraph.\n- The study does not extensively evaluate the method’s robustness across a broader range of distribution shifts beyond salt-and-pepper noise. The discussion on a different distribution shift (TravelingBirds) is very limited.", "other_comments_or_suggestions": "Adding a diagram illustrating how concept interventions work could enhance understanding.", "questions_for_authors": "1. How does your approach handle more complex and varied distribution shifts beyond those tested in the paper? Is it possible to test other types of out-of-distribution datasets or samples? For example by using different domains and contexts, like training on MNIST and testing on SVHN or using shared object classes from Imagenet and CIFAR-10 datasets. \n\n\n2. Under what conditions might your approach fail, and how could these limitations be addressed? Acknowledging potential weaknesses and offering solutions would strengthen the paper's robustness claims.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bY0xmqJPTw", "reviewer": "Reviewer_NChs", "summary": "The paper proposes MixCEM (Mixture of Concept Embeddings Model), which uses an entropy-based gating mechanism to control the leakage of information from the feature extractor. MixCEM is designed to dynamically adjust the influence of residual (leaked) embeddings so that they are beneficial for in-distribution samples while being suppressed for OOD samples. The authors back their claims with a thorough experimental evaluation on multiple datasets using diverse baseline CBM architectures.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your insightful review! Your comments, particularly those regarding OOD detectors, have helped us improve our manuscript.\n\nWe are glad you found our identification of leakage poisoning novel and telling of a “previously overlooked issue in the design of CBMs.” Moreover, we are glad you appreciate the novelty behind our entropy-based gating mechanism and that you believe this paper was “clear and easy to read.”\n\nBelow, we reply to the concerns raised as part of your review. If you have further questions or concerns, please let us know. Otherwise, we would appreciate it if you would consider updating your score.\n\n### **(W1) Limited Novelty of Architecture**\n\nAs discussed in our “Summary of Contributions” subsection, we see the MixCEM architecture as just one of our contributions. In particular, we believe the study of concept interventions in OOD setups and the identification of “leakage poisoning” (something not previously identified in the 3+ years these architectures have existed) are equally important contributions to this work.\n\nWe see the simplicity of MixCEM’s additions vis-a-vis CEMs as a “feature” rather than a “bug”. This work shows that a simple yet well-motivated modification to the CEM architecture can lead to models that are not only significantly better than CEMs at receiving interventions in OOD samples but **they are better than CEMs at receiving interventions even for in-distribution test sets**! This was achieved by introducing a simple yet novel entropy gating mechanism (as pointed out by this review). Considering all of this, we believe this work provides a series of novel contributions and describes an important yet previously unknown design consideration for concept-based models.\n\n### **(W2) CBM + OOD Detector Approaches**\n\nRegarding GlanceNets, we opted against including them for two reasons: First, they are not completeness-agnostic models, as their label prediction is based only on the concept-aligned latent factors. Second, although GlanceNets can detect leakage via their Open Set Recognition module, once this leakage is detected, the architecture does not provide a solution that allows operations like interventions to work in that instance. For example, in the leakage experiments (Section 5.2) of the [GlanceNet paper](https://arxiv.org/abs/2205.15612), the authors “implement rejection by predicting a random label“, meaning that if the OOD detector is triggered (i.e., a sample is “rejected”), GlanceNet simply outputs a random label.\n\n\nNevertheless, we believe your point about using a CBM and an OOD detector is a good suggestion and something one may want to try. As seen in the GlanceNet example above, however, even if we have a perfect OOD detector, we need to know how to act given the knowledge that an input might be OOD. In architectures like CEMs, one could, in theory, act on this knowledge by intervening using average “global” concept-level embeddings rather than the dynamic sample-dependent embeddings CEMs use for interventions (as this will “destroy” all leakage). We attempted this by **running new experiments on our CUB and AwA2 tasks, and we report the results of our strongest baselines, for visual clarity, [in this plot](https://imgur.com/a/k3nAzME)**. Here, we assume we have an “oracle” OOD detector that is always right for CEM, and use the mean active/inactive concept embedding computed in the training set to perform an intervention whenever the OOD oracle is triggered. Our results suggest that **even when we have a perfect OOD detector, interventions on CEMs are significantly worse than those in MixCEMs when samples are OOD**. More importantly, this strategy can lead to even worse intervention performance than using CEM’s original dynamic embedding when intervening. These results strongly suggest there is a significant benefit in jointly learning global and residual embeddings for interventions, as MixCEM does, and in including an OOD-detector-like gating mechanism as part of the inference path (as done by our entropy-based gates).\n\nGiven the above, we will (1) update Section 5’s “Baselines” subsection to clarify why GlanceNets are not an ideal baseline for our evaluation, and (2) discuss in Section 5.2 why OOD detectors, on their own, may not work unless architectural changes are made, with the results of the new experiments included in a new Appendix.\n\n\n### **(W3) MixCEM vs CEM**\n\nThank you for bringing up this point! We would appreciate it if you could refer to our reply to a similar question by reviewer At2h (our reply to “*Concern 1*”). In summary, we argue that MixCEM’s performance is significantly better than CEM’s when **intervened on** for ID and OOD datasets (**up to 55% percentage points in task accuracy!**). We hope that this discussion, together with the discussion above on why it is not enough to have an OOD detector with CEM to make interventions work for OOD samples, clarifies why MixCEMs bring significant improvements over CEMs.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "It relates well with current state of research.", "essential_references_not_discussed": "All references discussed.", "other_strengths_and_weaknesses": "Strengths:\n1. The identification of leakage poisoning is novel and highlights a critical, previously overlooked issue in the design of CBMs.\n2. The proposed solution, MixCEM, introduces an innovative gating mechanism that adapts to the uncertainty in concept predictions, thereby improving both ID and OOD performance.\n3. Paper is clear and easy to read.\n\nWeakness:\n1. Limited Novelty: The architecture clearly builds on top of CEM [1] with the presence/absence concept embeddings ($\\hat{c}$) contextualized with residual embeddings ($r$) that correspond to residual embeddings and global embeddings ($c$) which are learnable. The entropy-based gating mechanism controls ID and OOD samples' residual embeddings. This introduces more parameters and alternative reasoning pathways for OOD samples - which is overall limited in novelty.\n2. Comparisons to other CBM+OOD detectors - Approaches like GlanceNets [2] also fix leakage using a OOD detection mechanism. Hope the authors can provide a comparison to such approaches.\n3. Results: In most of the datasets, CEMs actually perform better than MixCEM. If this is the case, can a OOD detector as used in [2] be useful in discarding them to improve performance?\n\n[1] Concept Embedding Models, NeurIPS 22\n[2] GlanceNets, NeurIPS 22", "other_comments_or_suggestions": "Refer Weakness.", "questions_for_authors": "Refer Weakness", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kHoapLLqk7", "reviewer": "Reviewer_iEkK", "summary": "This paper investigates the behavior of concept-based models (CMs), particularly under distribution shifts, and introduces a novel model called MixCEM to address a critical limitation termed *leakage poisoning*. The authors demonstrate that existing state-of-the-art CMs, which rely on bypass mechanisms (e.g., residual connections) to handle incomplete concept annotations, suffer from degraded performance when intervened on out-of-distribution (OOD) samples. Leakage poisoning arises when leaked feature information becomes OOD, rendering concept interventions ineffective. MixCEM mitigates this by dynamically mixing *global* (sample-agnostic) and *contextual* (input-dependent) concept embeddings, using an entropy-based gating mechanism to suppress harmful residual information for OOD inputs. Experiments across four datasets (CUB, AwA2, CelebA, CIFAR10) and their concept-incomplete variants show that MixCEM outperforms baselines in both in-distribution (ID) and OOD accuracy, maintains intervenability, and avoids leakage poisoning. Key contributions include the formalization of leakage poisoning, the introduction of MixCEM, and the empirical validation of its robustness to distribution shifts.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to give us very insightful feedback. We are glad you found our architecture a “sensible” solution to solve the task at hand and its gating mechanism “novel”. Moreover, we are happy to read that you found our evaluation “extensive.” Below, we focus on addressing some of the concerns you raised in your review. If you have further questions or concerns, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your score accordingly.\n\n### **(W1) Inference Overhead**\n\nIn Appendix F, we mainly studied the training cost of MixCEM’s architectural components w.r.t. competing methods. However, we agree that this could also discuss inference costs.  We note that the MC sampling in MixCEM is a relatively cheap operation as it does not require rerunning the bottleneck generator.  We see this [$\\underline{\\textbf{here}}$](https://imgur.com/a/D2bEjg4) when we look at the inference wall-clock times of all baselines in CUB. Although there is a slight increase in inference times of MixCEMs w.r.t. CEMs (~4.2% slower), we argue this difference is not problematic considering MixCEM is designed to be deployed together with an expert that can intervene on it. In this setup, less than a millisecond of extra latency should not bear too heavy of a toll, as post-intervention accuracy is more important. Nevertheless, we will update Appendix F to include this discussion.\n\n### **(W2) Hyperparameter Sensitivity**\n\nWe agree that MixCEM has some hyperparameters that need fine-tuning. However, we show in Appendix I that MixCEM’s performance is very robust to a series of hyperparameters and, more importantly, **in Appendix I.6 we provide recommendations for selecting hyperparameters if one does not have the time to fine-tune the model**. There, we suggest focusing only on $\\lambda_c$. This means that future work could easily use our model (whose implementation as a standalone layer, seen in our code submission, will be made public) by trying only a few values for $\\lambda_c$ and fixing all other hyperparameters to our recommended default values. We hope this will encourage the ease of use and accessibility of our proposed methodology.\n\n### **(W3) Evaluation of more complex distribution shifts**\n\nThank you so much for bringing up this excellent point! **We have now performed new experiments that show that our results hold across several forms of realistic image noise/shifts (e.g., downscalings, affine transforms, etc.) as well as in different forms of noise like domain shifts (e.g., an MNIST-trained model deployed on real-world SVHN digits)**. Please refer to our reply to Q1 of Reviewer KTVL for these results.\n\n### **Improvements in CelebA**\n\nAs argued in our reply to “Concern 1” of Reviewer At2h (please refer to this reply for details), we are interested in improving task intervention accuracy in ID and, especially, OOD test sets rather than unintervened accuracy. We want to achieve this without significantly dropping unintervened accuracy. So, while we agree that the improvements of unintervened accuracy in CelebA are modest w.r.t. IntCEM and CEM, we highlight that, when all concepts are intervened on in the OOD CelebA test set, MixCEM’s task accuracy is 62.53%. **This is about 29% and 51% percentage points (!) above the accuracies of IntCEM and CEMs, respectively, when all concepts are intervened on those models too**.\n\n### **Bayes Classifier**\n\nWe used our masked MLP evaluation mostly because we needed a tractable way of testing a model that takes as an input **any** concept subset and predicts a downstream task from it. Given the limited compute and data we had for each task, it is intractable to calculate the true Bayes error optimally. Nevertheless, we point out that even if this baseline is not accurate and therefore underperforms w.r.t. the true Bayes Classifier, we still observe that several key concept-based models (e.g., CEMs, IntCEMs, and P-CBMs) significantly underperform against this approximation when intervened on in OOD test sets (our main claim in Section 5.2). This means they are unlikely to perform better than the true Bayes Classifier.\n\nWe will clarify this important point in our updated manuscript by better motivating using a masked MLP approximation for the Bayes Classifier in Section 5.\n\n### **Theoretical Guarantees for BI**\n\nOur main goal was to highlight a key design limitation in modern CBMs (that of interventions not working when inputs go OOD) that had not been pointed out in the 3+ years that residual and embedding concept-based architectures have existed. BI serves as a way to formalize a target that can help us study this design consideration. As such, we opted to provide a guideline for a research direction that may lead to achieving BI using a novel architectural gating mechanism. Nevertheless, we will make sure to point out that future work could explore formal guarantees for BI in MixCEM and similar models as we agree these are important.", "claims_and_evidence": "The paper derives MixCEM’s objective function as a maximum likelihood estimation (MLE) under a probabilistic graphical model (Appendix B). While the derivation aligns with the proposed architecture, the proof is not explicitly detailed, and assumptions (e.g., independence between global embeddings and inputs) are stated without rigorous justification. The bounded intervenability (BI) property is introduced as a desirable criterion but lacks formal theoretical guarantees. Further analysis of how the entropy-based gating ensures OOD robustness would strengthen the theoretical foundation.", "methods_and_evaluation_criteria": "**Methods**: MixCEM’s design is sensible for addressing leakage poisoning. By decoupling global and contextual embeddings, the model dynamically adjusts reliance on leaked features, which aligns with the goal of balancing completeness-agnosticism and intervenability. The entropy-based gating mechanism is a novel and logical approach to suppress OOD residuals.\n**Evaluation**: The benchmarks (CUB, AwA2, CelebA, CIFAR10) are standard in concept-based XAI, and the inclusion of concept-incomplete variants and synthetic/OOD shifts (e.g., TravelingBirds, salt-and-pepper noise) is appropriate. However, the evaluation focuses on synthetic noise and spurious correlations; testing on natural distribution shifts (e.g., domain adaptation datasets like PACS) would strengthen validity.", "theoretical_claims": "I did not check the details of the theoretical proof.", "experimental_designs_or_analyses": "The experiments are extensive, covering ID/OOD scenarios, concept-complete/incomplete tasks, and multiple noise levels. Key strengths include:\n\n- Comparisons against strong baselines (CEMs, IntCEMs, Hybrid CBMs, ProbCBMs).\n- Use of synthetic OOD shifts (e.g., salt-and-pepper noise) and real-world distribution shifts (TravelingBirds).\n- Ablation studies on hyperparameters (Appendix I) and bottleneck visualizations (Figure 5).\n\nHowever, some aspects warrant clarification:\n\n- The OOD noise injection method (Appendix G) uses pixel-level corruption, which may not fully represent real-world shifts (e.g., semantic changes).\n- The Bayes classifier approximation (Appendix D.4) relies on a masked MLP; its fidelity to the true Bayes optimal performance is not validated.\n- The reported improvements for CelebA are modest (e.g., ~35% task accuracy), suggesting potential limitations in highly noisy or subjective concept settings.", "relation_to_broader_scientific_literature": "The work builds on concept-based XAI (CBMs, CEMs) and addresses gaps in handling distribution shifts—an understudied aspect in interpretable ML. It connects to broader literature on OOD generalization and intervention-aware models (e.g., IntCEMs). The leakage poisoning concept parallels issues in robust representation learning (e.g., disentanglement of spurious correlations). However, the discussion could better contextualize MixCEM’s contributions relative to recent advances in OOD detection and causal intervention frameworks.", "essential_references_not_discussed": "No essential references are not discussed.", "other_strengths_and_weaknesses": "**Weaknesses**:\n\n- **Computational Overhead**: MixCEM’s residual dropout and Monte Carlo sampling (Appendix I.3) increase inference time, which is not quantified.\n- **Hyperparameter Sensitivity**: While ablations show robustness, MixCEM requires tuning $\\lambda_p$$p_{drop}$ , and $E_{cal}$, which may limit accessibility.\n- **Limited Real-World Shifts**: Experiments rely on synthetic noise; testing on natural OOD data (e.g.,  ImageNet-A[1]) would bolster claims.\n\n[1] Benchmarking Neural Network Robustness to Common Corruptions and Perturbations", "other_comments_or_suggestions": "No other comments.", "questions_for_authors": "No further questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper currently has three Weak Accept and one Accept recommendation. 

Among the identified strengths, the paper is generally evaluated as well-written, easy to read, and structured. Some reviewers consider the contributions well-articulated, and acknowledged how this work successfully identifies some gaps and potential issues in how Concept-Based Models (CBM) interventions respond to out-of-distribution inputs, proposing an effective method based on the identification of  “leakage poisoning”, which is benchmarked against baselines and using both synthetic and real-world OOD shifts.
 In the first reviews, some reviewers raised concerns about computational overhead, the need for more experiments with a broader range of distribution shifts or real-world OOD shifts (e.g., ImageNet-A), the limited novelty of the proposed architecture, and comparisons to other CBM+OOD detectors.

In their rebuttal, the authors provide new experiments covering more distribution shifts beyond salt-and-pepper, more insights on CBM+OOD detectors, and clarifications on inference overhead and the main contributions and novelty.

Considering the reviewers’ scores and the rebuttal and evaluating strengths and weaknesses of the paper, the AC agrees on the identified strengths and recognizes the novelty in the identification of “leakage poisoning”; therefore, they recommend the acceptance of this work.
Decision: Accept (poster)