Counterfactual Concept Bottleneck Models
Authors: Gabriele Dominici
Pietro Barbiero
Francesco Giannini
Martin Gjoreski
Giuseppe Marra
Marc Langheinrich
Venue: ICLR 2025 Poster (2025)
Keywords: Concept Bottleneck Models
Concept Based Model
Counterfactuals
Explainable AI
Interpretability
Primary area: interpretability and explainable AI
TL;DR: CounterFactual Concept Bottleneck Models predict, simulate, and generate alternative scenarios in a single step, without needing post-hoc analysis.

Abstract: Current deep learning models are not designed to simultaneously address three fundamental questions: predict class labels to solve a given classification task (the "What?"), simulate changes in the situation to evaluate how this impacts class predictions (the "How?"), and imagine how the scenario should change to result in different class predictions (the "Why not?"). While current approaches in causal representation learning and concept interpretability are designed to address some of these questions individually (such as Concept Bottleneck Models, which address both ``what'' and ``how'' questions), no current deep learning model is specifically built to answer all of them at the same time. To bridge this gap, we introduce CounterFactual Concept Bottleneck Models (CF-CBMs), a class of models designed to efficiently address the above queries all at once without the need to run post-hoc searches. Our experimental results demonstrate that CF-CBMs: achieve classification accuracy comparable to black-box models and existing CBMs (“What?”), rely on fewer important concepts leading to simpler explanations (“How?”), and produce interpretable, concept-based counterfactuals (“Why not?”). Additionally, we show that training the counterfactual generator jointly with the CBM leads to two key improvements: (i) it alters the model's decision-making process, making the model rely on fewer important concepts (leading to simpler explanations), and (ii) it significantly increases the causal effect of concept interventions on class predictions, making the model more responsive to these changes.

Published as a conference paper at ICLR 2025
COUNTERFACTUAL CONCEPT BOTTLENECK MODELS
Gabriele Dominici
Pietro Barbiero
Università della Svizzera italiana gabriele. domin cicicii @usi.
IBM Research"
pietro. warbieroo ibm com
Francesco Giannini
Martin Gjoreski
Scuola Normale Superiore francesco giannini@sns. it
Università della Svizzera italiana martin. gwwreskioikekeooo seuresiiiusi....
Giuseppe Marra
Marc Langheinrich
KU Leuven giuseppe marra@kuleeuen..bl
Università della Svizzera italiana marc wwwgheinnichheeeeeeeeeeeee
ABSTRACT
Current deep learning models designed simultaneously address three fundamental questions: predict class labels to solve given classification task (the "What?"), simulate changes in the situation evaluate how this impacts class predictions (the "How?"", and imagine how the scenario should change result in different class predictions (the "Why not?"). While current approaches causal representation learning and concept interpretability are designed to address some of these questions individually (such as Concept Bottleneck Models, which address both "what" and "how" guestions), no current deep learning model is specifically built to answer all them at the same time. To bridge this gap, introduce CounterFactual Concept Bottleneck Models (CF-CBMs), class of models designed to efficiently address the above queries all at once without the need run post-hoc searches. Our experimental results demonstrate that CF-CBMs: achieve classification accuracy comparable to black-box models and existing CBMs ("What?"). rely on fewer important concepts leading to simpler explanations ("How?")), and produce interpretable, concept-based counterfactuals ("Why not?"). Additionally, we show that training the counterfactual generator jointly with the CBM leads to two key improvements:
(i) it alters the model's decision-making process, making the model rely on fewer important concepts (leading to simpler explanations), and (ii) it significantly increases the causal effect concept interventions on class predictions, making the model more responsive to these changes.
INTRODUCTION
Original (Pneumothorax)
Counterfactuals (No Pneumothorax)
To calibrate human trust and hance human-machine interactions, deep learning (DL) models should learn how to master three fundamen- tal questions: predicr class labels for new inputs (the "What is the diagno- sis for the X-ray shown in Figure """. simulate changes in the situation to evaluate how this impacts class pre- dictions (the "How would the absence of collapsed lung impact the presence of pneumothorax?"", and imagine al-
Why not No Phoumothorax?
Input level No intulive
Why not Pnoumothorax?
lepaeedd Plurn
Go Collepedd
Concept level
Interpratablee
Val Flevra
Figure 1: CF-CBMs generate counterfactuals at the concept level rather than the input level, the changes the contrary, identifying the changes in input level require significant more effort from the user. The images are sourced from the SIIM Pncumothorax dataset (Zawacki ct al. 2019).
interpretable.
Work conducted while employed Università della Svizzera italiana.

---

Published as a conference paper at ICLR 2025
Bive world Certu goood Tagg Impprtt Slaguedo
Noo Photasetheraz
Chllapeed
Parzmotiiona
WHYY NOT
1. WHAT What the dlagpoota for &ha
Why the dhapoott Sor the Jonay not Nos Portno
topot
concopé-hanee
dierrbetion
locer coppertterraa dueesbattio.
coumtortectral
Xocodax
Conorpt Prodinoe
Varia Hirmal
No
Prebets
Vadija
Line
4,
Collaged Coru
sheusder tal sloi
Figure 2: Counterfactual CBM. For given input sample, the task predictor answers "what?" queries predicting class labels. The concept predictor answers "how?" queries simulating changes in the scenarios through interventions. When taking the counterfactual latent distribution z', the concept predictor answers "why not?" queries via concept-based counterfactuals, which are easier to understand then counterfactuals at the input level.
ternative scenarios that would result in different class predictions (the "Why is the patient not having pneumothorax?") also known as counterfactual explanations (Wachter et al. (017). Currently, no existing DL model is designed to answer all these questions by design.
Concept Bottleneck Models (CBMs) (Koh et al. 2020) are DL architectures designed to answer both "what?" and "how?" questions. They first predict human- unrrannndnnsttaaaaelee concepts (Kim et al., 2018), allowing experts to evaluate how intervening on such concepts influences the model's class predictions. However, CBMs fall short in answering "why not?" queries as they are not trained to produce concept-based counterfactuals. More broadly, counterfactuals have been explored in concept-based models, e.g., for concept discovery (Stammer al., 2024), while causal inference addresses this statistically (Louizos et al. 2017 Xia et al., 2022), seeking to discover and/or exploit the true causal relationships in the data. In contrast, we focus on addressing these questions from the model's 's perspective, shedding light on its decision-making the-art counterfactual generation methods (Wachter et al.) 2017:) 2 Pawelczyk et al. 2020 Guyomard et al. 2023; Nemirovsky et al., 2021) answer "what?" and "why not?" but not "how?", especially when working with unstructured data such as images. Moreover, in these cases, counterfactual methods may generate image that either looks nearly identical to the original, or ii) incorporates pixel-level modifications of difficult interpretation from the human perspective across thousands of pixels throughout the entire image (as shown in Figure [ This scattering of modifications makes it hard to understand the answers to the "why not" questions. In addition, is difficult to answer "how" questions because semantically coherent pixel-level interventions are hard define and perform for domain experts (e.g., radiologists) (Abid et al. 2002]. In summary, input-level counterfactuals often lack actionable insights, while concept- models cannot answer counterfactual queries.
process. From this viewpoint, ot
Contributions. To bridge these gaps, we introduce CounterFactual Concept Bottleneck Models (CF-CBM), class of deep learning models designed to jointly master three fundamental questions for a classification problem "What?", "How?" and "Why not?", generating interpretable concept-based counterfactuals. Our key innovation a latent process that generates two similar concept vectors: one predicts the target class label (as in standard CBMs), while the other predicts an alternative class. Our experiments show that CF-CBMs achieve classification accuracy comparable to black-box models and existing CBMs ("What?")), rely on fewer important concepts leading to simpler explanations ("How?""), and produce interpretable, concept- based counterfactuals ("Why not?"). Additionally, training the counterfactual generator jointly with the CBM leads to two key benefits: (i) it alters the model's Docision-making making it rely on fewer important concepts and (ii) significantly class predictions, making the model more
increases the causal effect of concept interventions responsive to these changes. The code of this paper is publicly available
BACKGROUND
Concept Bottleneck Models (CBMs). Concept Bottleneck Models (Koh et al. 2020) are interpretable architectures that explain their predictions using high-level units of information (i.e., "concepts").
2

---

Published as a conference paper at ICLR 2025
Given sample's raw features X S R (e.g., image's pixels), set of concepts C' {0,1}" (e.g.. "collapsed lung", "visible pleural line"), and a set of class labels yj E Y S {0, 13 (c.g., labels "pneumothorax" or "no pncumothorax").
CBM estimates the conditional distribution II, P(y; c,,..., G)) II, p(c; x) where p(c; x)) is set of independent Bernoulli distributions. At test time, human experts may intervene mispredicted concept labels to improve CBMs' task performance, simulating different scenarios. In probabilistic perspective (Bahadoni & Heckerman 2020, Misino et al. 2022), (ideal) concepts represent key factors of variation (Kingma Welling 2013
Zarlenga et al.
2023 of observed data æ and as illustrated in the graphical model
Counterfactual Explanations. A counterfactual is hypothetical statement in contrast with actual events that helps us understand the potential consequences of different choices cireumstances (Pearl et al., 2016). In the context of machine learning, Wachter et al. (017)) define counterfactual explanations as an optimization problem representing an answer
"why not" question. The optimization aims to find for each observation the closest datapoint x' such that classifier m produces class label m(x') that different from the original label m(z).
COUNTERFACTUAL CONCEPT BOTTLENECK MODELS
CounterFactual Concept Bottleneck Models (CF-CBMs, Figure is novel class of interpretable models designed to generate counterfactuals via variational inference at the concept level. In the following, motivate CF-CBMs' architecture and optimization objective (Section 3.1 Sections 3.2 3.3 3.3 show how CF-CBMs answer the different questions and how users can act upon generated counterfactuals.
3.1
GENERATING CONCEPT-BASEL COUNTERFACCUUALS
To realize concept-based counterfactuals, extend the graphical model two additional variables: c', representing concept- based counterfactual labels; and y', representing the counterfactual class label. We represent the counterfactual dependency the actual concepts with an arrow from c to e', while y' depends on d':
introducing
We also notice that multiple counterfactuals can be used to explain any given fact (Wachter et al. 2017. Pearl 2000). To address this, an ideal solution would be to use a generative approach modeling the discrete distributions over and c'. However, modeling such distributions directly might be unfeasible in practice (Richardson & Domingos] (006) as requires either modelling complex concept dependencies with discrete distributions which scale exponentially with the number of concepts, or assuming that concepts are independent of each other, an often unrealistic assumption. For this reason, we use latent variable approach (Kingma & Welling 2013) model concept dependencies in a continuous latent distribution, which allows us to model discrete concept values as independent of each other given the latent variable.
Architecture. CF-CBMs are latent variable models generating counterfactuals via variational inference. To this end, they add two random variables and z' Diagram [ These variables represent latent factors of variation whose probability distributions are easier to model and sample compared those for e and d' We also include arrows from e and y to the counterfactual latent distribution z' in order to explicitly model the dependency of z' on the symbolic values of actual concept e and class labels y, resulting in the following overall probabilistic graphical model:
(2)
This way. the generative distribution factorizes as: u/,. p(c, plc.:llappddddllll/oltttappaaa p(c.vlz)= p(ule)p(cl))
(3)
p(zlc.v) p(2))(2/1e,c.)))
(4)
3

---

Published as a conference paper at ICLR 2025
In approach, p(yle) and p(y"lc') are modeled categorical distributions parametrized by the same task predictor f: (cc]z) and p(c' (²") as sets of Bernoulli distributions parametrized by the same assume that the input is always observed at test time, making
concept predictor 6.- In practice, the p(xiz) term irrelevant. This allows z and z/ to encode only the most salient information about the concepts and ct, rather than all the details needed to reconstruct As result, the optimization process is significantly simplified, enabling efficient counterfactual generation for complex data like images. Finally, p(z) a normal prior distribution and p(='|z, c, y) is learnable normal prior whose mean and variance are parametrized by neural networks 6pu and &pe- The added complexity introduced with these additional black-box components limited the stages before concept extraction, enabling the encoding of information about possible counterfactuals. Thus, our model retains the interpretability of CBMs while providing the additional capability to explain decisions through counterfactuals.
Amortized inference. CF-CBMs amortize inference needed for training by introducing two approxi- mate Gaussian posteriors (==(x)) and <=''2z, y, 3/) whose mean and variance are parametrized by pair of neural networks (o Φa) ((0)', Par), respectively). The corresponding inference graphical model (i.e. the encoder) is shown in Appendix
Optimization problem. CF-CBMs are trained to optimize the log- log-likelihood of tuples (c. y.y'),)) y²). where y' is randomly sampled from uniform distribution for each observation of x, simulating scenario in which a user requests the generation of a counterfactual for a specific label. Following a variational inference approach, optimize the evidence lower bound of the log-likelihood, which results (see Appendix A in the following objective function to maximize:
reconstruction of and
prior regularization on
C = X E [==o( [log + logp(ulc) X2 (xDx((((2(1) [p(=)]
recomsimaction of
prior regulatization on
+ AeBessseeoeeeeerrrrrrooeeooooooo
-----aaaaa DeDaal((a a))lp(2'|2,c,,.)
where DEL is the Kullback-Leible divergence, Moreover, in order to enforce concept-based counterfactuals to be as close as possible to the current concept labels, we add an additional term to the objective (Kingma & Welling 2022):
",,,y',y') and X; are hyperparameters.
posterior distance
prior distance
Ld= = As Dxz (a(=\=)lla(="(aa) DED (v(=))lp(e/le,c.)))
(5)
3.2 ANSWERING "WHAT?", "HOW?" AND "WHY NOT?" QUERIES
CF-CBMs design allows them to answer "what?", "how?" and "why not?" queries through the following steps (Figure 2):
1. What? Predict concept and class labels:
(a) Sample from latent concept posterior: 2 ~ (d)(z))
(b) Sample to predict concept and class labels: ÿ ~ Cat(f (C)), ė (er(
2. How? Simulate changes in the situation and evaluate the impact on the outcome. Similar to standard CBMs, is possible to intervene at the concept level-since these concepts are inherently interpretable-and analyze how these modifications impact the model's prediction:
(a) Simulate changes intervening on concept values: è == 2 Vi E I where ē, is the intervened value for each concept i in the set I of the intervened concepts.
(b) Sample to predict class labels based on the intervened concepts: ÿ ~ Cat(f(())
3. Why not? Imagine interpretable counterfactuals via the generation of new concept tuples:
(a) Sample from latent counterfactual posterior:
(() Por (a)), a (2,ê,0.1)), 5/~ ~ Categorical()), v v~ U(0 lo]
(b) Sample to predict counterfactual labels: v/ (e)), &l Ber(d>)='))
4

---

Published as a conference paper at ICLR 2025
The following example illustrates a concrete scenario.
Example. Consider a lung X-ray sean with the task of classifying if a patient has pneu- mothorax not. Additionally, we have access to two key concepts: "collapsed lung" and "visible pleural line", which are critical in determining if a patient has pneumothorax not. A CF-CBM addresses the three interpretability questions follows: Predict: the model predicts concept labels = LoollopeedIongg = 1, Pieuralle = 1 and class labels UNo Pnoumatthhraa 0, Pneumothorea 1, indicating the presence of both concepts and Pneu- mothorax classification. Simulate: it is possible to intervene on the concept "collapsed lung" CoolapsedLanng 0, simulating different scenario, and observe that the model still pre- diets the same class, presence of Pneumothorax, showing the model's robustness to changes in one concept. Imagine: generate the counterfactual where the desired class changes to "NNonneumthoras 1, U/ Pnnemmtthhrrrr 0. The corresponding concept-level counterfactual would be "CollppedLangg 1, y Plerallee O, demonstrating how modifying these concepts could lead to different classification.
3.3 TEST-TIME FUNCTIONALITIES
In contrast to standard CBMs, CF-CBMs can either sample or estimate the most probable counterfac- tual to: (i) explain the effect of concept interventions tasks, (ii) show users how to get desired class label, and (iii) propose concept interventions via "task-driven" interventions.
Counterfactuals explain the effect of concept interventions on downstream tasks. Plain concept- based explanations indicate the presence or absence of a concept for given class prediction. However, the complexity of plain explanations grows quickly with the number of concepts. Concept-based counterfactuals, instead, induce simpler, sparser explanations (via Eq. 5) representing minimal modifications of concept labels that would have led different class prediction.
Act upon concept- counterfactuals. Concept- based counterfactuals can guide users towards achieving desired outcomes (indicated by class labels), especially when users do not know how to alter their status (represented by concepts). In this scenario: (i) a user specifies a desired class label y for the downstream task representing their goal (e.g.. get a loan), (ii) CF-CBMs generate a concept-based counterfactual p(c'lp/ ().) representing minimal concept modifications changing the class label from the predicted ý to the desired y' (e.g., save 10% more each month), (iii) the user can act upon the counterfactual c' accomplish the goal represented by y'.
Task-driven interventions fix mispredicted concepts. A key feature of CBMs is enabling human-- in-the-loop loop interventions: at test time, users correct mispredicted concept labels to enhance downstream task performance (c -> y intervention). Concept interventions can be useful when intervening concepts is easier than intervening easier to identify if the lung is collapsed (c) in a lung X-ray scan than to provide an accurate prediction if a patient has or not pneumothorax (y). While still supporting concept interventions, CF-CBMs may also invert this mechanism via task-driven interventions (y + c), exploiting its counterfactual generation abilities. Task-driven interventions can be used to correct mispredicted class labels when intervening on the downstream task is casier than intervening on concepts. For instance, it might be easier to identify mispredicted Parkinson's 's disease (y) rather than intervening in the genetic pathways (c) leading to the disease. In this scenario, users can intervene on the class label (rather than on the concepts) suggesting a correct y/. Considering this additional information, CF-CBMs propose more accurate set of concept labels è' representing potential concept intervention previously unknown to the user.
the downstream task. For instance, it might be
EXPERIMENTS
Our experiments aim to answer the following questions:
What? prediction generalization: Do CF-CBMs attain similar task/concept accuracy as standard CBMs?
How? intervention impact: Does optimizing counterfactual generation end-to-end with the predictor alter the importance assigned by the model each concept?
5

---

Published as a conference paper at ICLR 2025
Why not? counterfactual actionability: Can CF-CBMs produce valid, plausible, efficient counterfactuals? Can CF-CBMs generate accurate task-driven interventions?
This section describes essential information about experiments. We provide further details in Ap- pendix C
Data & task setup In our experiments we use five different datasets commonly used to evaluate CBMs: dSprites (Matthey et al. 2017), where the task is to predict specific combinations of objects having different shapes, positions, sizes and colours; MNIST addition (Manhaeve et al.) (2018), where the task to predict the sum of two digits; and CUB (Welinder et al.J (010)). where the task is to predict bird species based on bird characteristics; CIFARIO (Krizhevsky et al.), where the task is to classify the object in the image, and SIIM Pneumothorax (Zawacki et al. 2201)) where the task is to determine whether the X-ray scan indicates Pneumothorax. These two datasets do not include concept annotations, so we extract them following the method proposed in Oikarinen et al. (2023). In all experiments, images are encoded using pre-trained ResNetl 88 (He et al. 2015) (dSpties, MNIST Add, CUB), CLIP V1TB16 (Radford et al. (CIFARIO) or CXR-CLIP Pneumothorax).
You et al.
2023 (SIMM
Evaluation In our analysis we use the following metrics. What? (prediction generalization): we compute the Area Under the Receiver Operating Characteristic Curve (Hand & Till 200T) for concepts and tasks (ROC AUC ( How? (intervention impact); we assess the influence of concepts on the downstream task using the Causal Concept Effect (CaCE) (Goyal et al. 2019), which quantifies the degree to which each concept is critical for class. This allows us to evaluate the impact of optimizing counterfactuals end-to-end versus using post-hoe methods. Additionally, in scenarios where confounders are present, CaCE can help determine whether the impact of these confounders is high or low. Why not? (counterfactual actionability): drawing from previous works on counterfactuals, we compute: (i) the validity (Wachter et al. 2017) by checking whether the model predicts the desired class labels based on the generated counterfactual; (ii) the proximity (L) (Pawelczyk et al. 2020) which evaluates counterfactuals" "reliability" as their similarity w.r.t. training samples; (iii) the time (L) (Romashov et al. 2022) to generate counterfactuals (see Appendix D for the results on this metric). In addition, we propose the following metrics to assess the quality of counterfactuals: (i) the A-Sparsity (L) which evaluates user's "wasted efforts" by counting the
number concepts changed (Guo et al. (023) w.r.t. the minimal number of changes that would have generated a counterfactual according to the dataset; (ii) the "plausibility" (Wachter et al.) (017) as the Intersection over Union (loU (t)) (Jaccard] 1912) between counterfactuals and ground -truth concept vectors; (iii) the variability (1) which evaluates counterfactuals" "diversity", as the cardinality ratio between the set of counterfactuals generated and the set of training concept vectors; (iv) finally, we measure the concept accuracy of generated interventions (Acc Int. (1)) w.r.t. ground-truth optimal interventions that evaluate the model's 's ability to generate potential concept interventions. Following Espinosa Zarlenga al. (2022). we inject noise predicted concepts to reduce task accuracy, then we sample counterfactuals, conditioning on the ground-truth label, and use them to fix mispredicted concept labels automatically. All metrics are reported using the mean and the standard сгror over five different runs with different initializations. Additionally, in Appendix E. we provide examples of counterfactuals generated by our models and the baselines across various datasets.
Baselines In our experiments, we compare CF-CBM with Black Box model, a standard CBM and a more powerful CBM as Concept Embedding Model (CEM) (Espinosa Zarlenga et al. 2022 in terms of generalization performance. To ensure a fair comparison, adapting post-hoc methods to standard CBMs, allowing for a direct comparison between our approach and other state-of-the-an art models. We propose this experimental design for two main reasons. First, standard CBMs cannot generate counterfactuals by design, hence they need an external generator. Second, the standard, direct application of post-hoc methods on unstructured data types (e.g.. image pixels rather than concepts) would have significantly compromised counterfactuals' interpretability and difficult comparison between the generated results. We remark that this already represents an undocumented procedure in the current literature, as post-hoc methods typically derive counterfactuals from input features, not in the concept space. However, this also enables us to compare post-hoc
create counterfactual baselines by
²(t): the higher the better. ({): the lower the better.
A...a. "correctness" (Abid al. 2022).

---

Published as a conference paper at ICLR 2025
counterfactual methods with our model, which incorporates counterfactual constraints during training.
Among post-hoc approaches we select: (i) the Bayesian Counterfactual (BayCon) (Romashov et al.)
2022), which generates counterfactuals via probabilistic feature sampling and Bayesian optimization;
(ii) the Counterfactual Conditional Heterogeneous Variational AutoEncoder (C-CHVAF) (Pawelezyk)
a.. 2020), which iteratively perturbs the latent space of a VAE until it finds counterfactuals; (iii) the
VAE CounterFactual (VAE-CF) (Mahajan et al.) 2220)), which improves C-CHVAE by conditioning counterfactuals on model's predictions; (iv) the Variational Counter Net (VCNet) (Guyomard et al.
2023 which conditions latent counterfactual sampling on a specific target label y'
KEY FINDINGS & RESULTS
5.1 WHAT? PREDICTION GENERALIZATION
CF-CBMs achieve generalization performance that is close to standard CBMs. CF-CBMs attain both concept and task ROC AUC similar to Black Box, standard CBM and CEM, a more expressive CBM (details in Appendix DD. This shows that generating counterfactuals does not have a negative impact on classification performance on the datasets we considered. Our goal was not to improve performance but to improve interpretability. This outcome highlights how CF-CBMs generalise standard CBM architectures by matching their predictive accuracy while introducing new fundamental functionalities (i.e., counterfactual explanations and task-driven interventions).
5.2 HOW? INTERVENTION IMPACT
Jointly training counterfactuals generator and CBMs makes the model rely on fewer important concepts leading to simpler expla-
Table I: CF-CBMs ignore the confounding concept on the modified dSprites dataset.
nations (Table Following Koh et al. (2020).
CF-CBMs allow users to intervene directly on
Post-hoc
Joint (ours)
the concepts, enabling simulation of diverse sce- narios and observation of their effects the model's final prediction, as quantified by the Causal Concept Effect (CaCE) (Goyal et al. 2019). To assess the impact of CF-CBMs joint training design on how the model perceives the importance of each concept we conducted exper- iments using a modified dSprites dataset, follow- ing Goyal et al. (2019), where high correlation was introduced between an object's color and its class label. While standard CBM explanations weighted shape and color (confounders) equally for class prediction, CF-CBMs correctly priori- tized shape as the primary predictor, as shown in Table Specifically, altering color in standard CBMs resulted in a CaCE score of 0.17, demonstrating a 17% chance of prediction change due to this confounding factor. This reveals that standard CBMs susceptible relying on shorteuts in decision-making (Marconato al. 2023). In contrast, CF- CBMs showed minimal sensitivity to color changes, with a CaCE score of just 0,02, demonstrating significantly higher robustness against confounding features. Additionally, CF-CBMs assigned higher CaCE scores to shape concepts, indicating a stronger reliance on these critical features, unlike the host-hoe version. This trend was consistently observed across the other datasets: MNIST (p-value 0.003) and CUB (p-value- = 0,000), where CF-CBMs and CBMs exhibited distinct CaCE distributions as confirmed by the Kolmogorov-Smine Smiirooy----oovvoooooo test (the distributions are shown in App- D. Table 2 further illustrates that CF-CBMs achieved a higher mean CaCE for the top 10% of concepts, signifying that the model concentrates on a smaller subset of highly relevant concepts. This focus results in more con- cise and interpretable explanations by engaging reduced number of concepts. Moreover, CF-CBMs exhibited a lower mean CaCE for the bottom 50% of concepts, suggesting that the model learned to disregard less critical features, thereby enhancing robustmess against concept perturbations. The Mann- Whitney U test results reveal that three out of four comparisons were statistically significant, underscoring the impact of joint training. This shift in decision-making comes from the CF-CBM's
Shape
0.16 = 0.03
0.30 2 0.15
Shape 2
0.13 = 0.03
0.18 1 0.08
Shape 3
0.16 1 0.03
0.32 1: 0.09
Two obj
0.04 = 0.01
0.16 1.6:00 0.07
Colours (confounder)
0.17 a 0.03
0.02 4: 0.01
Table 2:
Compared to CBMs, CF-CBMs assign
greater importance to a small fraction of concepts, while the majority of concepts receive less importance on CUB.
Perceatile
Joint
p-value
MINIST
Top 10% (1)
0.15 +: 010203
0.1827
+ 0.0676
0.0001
Buttum 50%
0.0175
0.0257
0.0143
0.2437
CUB
Top 10% (h)
0.0059
0,0029
0.0131 =: 0.0044
0.0
Bottom 50% (L)
0.0066 0.0066
0.0013 0.0008
0.0
7

---

Published as a conference paper at ICLR 2025
Table 3: CF-CBMs outperform all baselines in counterfactuals" validity across all datasets.
dSprites (?)
MNIST add (1)
CT
(IFARID
SIM Prcumothurax (7)
arg. (1)
TRM-BayCon
100.0
94.6 0.5
66.2 0.0.5
91.0
100.0 0.0
90.4 0.7
CBM-OCHVAE
0..4::0.6
04.7 0:2.
78.7 1:04
9.16::0.1
90.0 0.6
91.7 1: 0.2
CBM-VARCF
100.0 0.0
82.0 + 5.9
96.2 10.2+0..
91000088
100.0 + 0.0
53.4 + 1.2
CBM-YCNET
100.0
4:0.5
100.0::00
100.0
CF-CBM (ours)
100.0 0.00
96.4
0..0
100.0.1.00
99.9 0.1
99.06 0.1
objective of generating counterfactuals that alter only the minimal subset of concept labels required to change the class prediction- c.g. adjusting shape but not color. Consequently, the model's 's task predictor forced focus on the minimal set of decisive features. For additional details on the CaCE score distributions, refer to Appendix [ Overall, our findings suggest that jointly training CF-CBMs not only improves model interpretability but also yields more robust decision- maainn process.
5.3
WHY NOT? COUNTERFACTUA ACTIONABILITY
CF-CBMs outperform baselines in counterfactuals' validity. (Table 3 CF-CBMs attain the highest counterfactual validity across all datasets when compared to counterfactual generation baselines applied to the concept space of standard CBMs. This result shows that CF-CBMs can effectively generate concept vectors that lead to user-requested class labels. The gap w.r.t. baselines increases in complex datasets involving a vast search space over multiple concepts and classes. In the MNIST addition dataset (20 concepts, 19 classes), CF-CBMs achieve +14 percentage points (p.p.) higher validity than VAECF and +7 p.p. higher than VCNet. In CUB (1 18 concepts, 200 classes), CF-CBMs find solutions with a significantly higher validity w.r.t. iterative methods: -33 p.p. higher than BayCon and +0 p.p. higher than C-CHVAE. Lastly, in the CIFARIO and SIIM Pneumothorax datasets, CF-CBMs continue achieve the highest validity.
CF-CBMs balance counterfactuals/ reliability with minimal user effort. (Figure 3
High- qualiiyaltty
counterfactuals should satisfy two key properties simultaneously: be reliable and to require the least amount of users' effort to act upon. However, minimizing user's 's effort alone might lead to learning shorteuts producing less reliable counterfactuals, while optimizing reliability might generate counterfactuals that change the values of multiple concepts and represent significant burden for users to take actions. For this reason, these two properties should be evaluated together:
CF-CBMs generate reliable counterfactuals: When processing large datasets, the training distribution covers a large portion of the feasible action space. For this reason, counterfactual solutions close to training samples are likely to be more reliable, as discussed by Pawelczyk et al. 22000); Laugel et al. (2019); Wachter et al. (2017). Our results show that CF-CBMs generate highly reliable counterfactuals across all datasets. Baseline models instead struggle produce counterfactual close to training samples in at least one dataset.
CF-CBMs generate counterfactuals requiring minimal user effort to act upon: CF- CBMs consistently generate counterfactuals close to the optimal sparsity i.e., the fewest possible concept changes alter the class prediction. This represents significant result in terms of actionability because each concept modification requires user effort. For this reason, should a counterfactual require many concept label changes compared the original concept vector, it would lead to substantial unnecessary effort. Across all datasets, CF-CBMs stand out for their consistency in this aspect: their counterfactuals are either the sparsest or very close to the sparsest. This makes them highly effective and user-friendly. avoiding the burden of unnecessary actions.
CF-CBMs calibrate the taabbbiityypauubbiliii trade-off. (Figure 4 Another key trade-off in generating counterfactuals corresponds to the tension between producing counterfactuals with high diversity, allowing users to pick among different roadmaps action, and producing plausible counterfactuals corresponding to user actions that are likely to be feasible in practice. However, trade-off. For
learning shortcuts might lead to trivial solutions with a poor sariability-plausbbbie instance, once plausible counterfactual is found, trivial solution could be to generate always the same counterfacual, thus leading to minimal diversity, while maximizing the variability increases the chance of generating less plausible counterfactuals. In practice, these two properties variability and IoU between counterfactuals and ground-trume truth concept vectors. A higher variability implies that our model can produce a wider range of counterfactuals. A higher loU value suggests that
measured by
8

---

Published as a conference paper at ICLR 2025
eSprices
MNIST add
CUB
CIFAR19
SIIM PNEUMOTHORAX
815151
REST (1)
BEST 13
915T (1)
BEST
03ST (++)
DEST (-)
BEST [-]
83ST
BEST
Posinityy (%)
Proceit [S]
Prwinety
CF CBM
CBM
Bay
CBM COAME
CBM WAEFF
CBM VONET
Figure 3: CF-CBMs balance the trade off between counterfactuals reliability (proximity) and user effort (A Spaassiiyyy, The arrw (-) points towards optimal values. Pareto-optimal models form the frontier of the shaded region, whereas dominated solutions are located within the shaded region.
dprittes
MNIST add
CUB
CIFARIO
SIJM
PNEUMOTHORAX
14
155
10
25
RIST
8EST
MA 182
ou (4)
Id0 (5)
IAU 1%2
1e0 (4)
CF-CEM
CEM WRECF
CF-CBME
100
COHNEE
CBM
Figure 4: CF-CBMs calibrate the trade off between counterfactuals" diversity (variability) and plausibility (IoU). The arrow (-) points towards optimal values. Parcto-optimal models form the frontier of the shaded region, whereas dominated solutions are located within the shaded region. "CF-CBM n" samples n counterfac- tuals from the latent distribution as described App.
generated counterfactuals are also representative of the training distribution which contains plausible solutions. Our results show that CF-CBMs balance the optimization of these two properties better than existing methods. The gap w.r.t. BayCon and VCNet is already significant on dSprites where CF-CBMs produce counterfactuals with +5 p.p. higher IoU (plausibility) and x3 more variable. In MNIST addition the VAECF's performances significantly deteriorate, while C-CHVAE mainly struggles in CUB. On CIFARIO, all baselines perform poorly on the IoU metrie, while on the SIIM Pneumothorax dataset, they also struggle with variability, with the exception of C-CHVAE.
CF-CBMs generate accurate task-driven interventions. (Figure CF-CBMs enable task-driven interventions, showing users how to modify concepts to get the desired class label. This represents novel crucial feature when users know the desired outcome but lack the knowledge to intervene on concepts. CF-CBM significantly outperforms other baselines in concept accuracy of task-driven interventions by up to p.p. Such performance can be explained as the generation of raikenriven interventions is conditioned on a user- gper-provided target class label which provides fundamental infor- mation for the model to search in the latent concept distribution. Appendix shows that CF-CBMs attain similar results while applying different levels of noise on predicted concept labels.
DISCUSSION & CONCLUSION
Limitations. CF-CBMs combine features from CBMs and generative models, carrying forward their strengths and limitations. From generative models, CF-CBMs inherit the ability to efficiently approximate the data distribution. However, this approximation does not guarantee counterfactuals optimality when compared to exact methods (Wachter et al. 2017). Besides, CF-CBMs require concept annotators (humans or machines (Oikarinen et al. 2023 )) to ground explanations, as standard CBMs. Reasoning shortcuts (Marconato et al. 2023). concept impurities (Zarlenga et al., (023). and information bottlenecks (Espinosa Zarlenga et al. 2022)
also typical limitations of CBM architectures, especially when the concept bottleneck is not complete (Yeh et al. 2020).
Relations with concept-based models. CF-CBMs share common ground with existing CBMs in modeling concepts in latent spaces. However, current models use the latent concept space to improve
task accuracy (Espinosa Zarlenga et al.] [2022] Barbiero et al.] [223]] Debot et al. (024). promote concept disentanglement (Misino et al.
et al. (002). model concept interactions (Xu
et al. 2024 Barbiero et al. 2024), estimate task confidence intervals (Kim et al. 2023). Differently,
9

---

Published as a conference paper at ICLR 2025
eSprices
MNIST add
CUB
CIFAR19
SIIM PNEUMOTHORAL
-10
-30
-17
180
100
180
0S5T (-3
BEST (-))
EEST [-]
055T (-)
BEST
Prosintly
(4)
Powrrety (S)
Proeeity (%)
CF-CBM
CBM CCHWRE
CBM WAFFF
CBM
VONET
Figure 5: CF-CBMs generate reliable (proximity) and accurate interventions (ROC AUC Int.). The arrow (-) points towards optimal values. Pareto-optimal models form the frontier of the shaded region, whereas dominated solutions are located within the shaded region.
Table 4:
CF-CBMs efficiently address "what?". "how?", and "why not?" questions while supporting both concept and task-interventions, as opposed to existing CBMs and counterfactual generators.
CF-CBM (ours)
CBM
Black Box
BayCon
C-CHVAR
VAE-CF
VCNet
What? How?
Why not?
Cuncept interventions
Task driven interventions Real-time
4
CF-CBMs leverage the latent concept space to extend CBMs' capabilities by introducing two key functionalities (Table 4 counterfactual explanations and task-driven interventions.
Relations with generative structured models. CF-CBMs are latent generative models characterized
by a structured latent space. This charactorization aligns them with other methodologies utilizing
hierarchical priors (Sonderby et al. (2016); Maalee et al. (2019); Vahdat & Kautz (2020); Klushyn et al. (2019)), mixture priors (Bauer & Mnih (2019)), ( and autoregressive priors (Chen et al. (2017):
den Oord et al. () Razavi al. (2019)) However, the structure of CF-CBMs' latent variables
explicitly captures both the concept and the counterfactual latent space, thus promoting structural interpretability, unlike existing structured latent variable models.
Relation with counterfactual models. Existing DL methods generating counterfactuals typically offer post-hoc explanations (Ghandeharioun et al. 2021 Abid et al., (022) or require interpretable input features (Guyomard et al.) [2023] Pawelczyk et al. [020 Mahajan et al.) 2020). However, applying post-hoc methods to unstructured data (e.g., pixels instead of concepts) significantly reduces counterfactuals' interpretability (Kim et al. (0188. In contrast, CF-CBMs: (i) are data-agnostic, ex- tracting interpretable counterfactuals regardless of data type, (ii) provide answers to "how?" questions by allowing interventions, (iii) modify the decision-making process by optimizing counterfactuals through joint optimization leading to relying on fewer concepts and (iv) support both concept and task-driven interventions (Table 4 As side effect, concept- baasept-based based counterfactuals may also introduce benefits in terms of privacy as sensitive and unique details embedded in input data (c.g.. the image of radiography) remain concealed, thus mitigating the risk of data extraction through repeated model queries (Goethals et al., 2023 Naretto et al. 2022 Pawelczyk et al.) 2022).
Broader Impact. The ability to answer all the three questions represents fundamental step to calibrate human trust and enhance mamane interactions. In this work, we show how combining existing interpretable models (such CBMs) and post-hoc counterfactual generators may represent possible solution to address "what?", "how?", ; and "why not?" queries. Experimental results show that training the counterfactual generator jointly with the CBM leads to two key improvements: (i) it alters the model's decision decision-making process, making the model rely on fewer important concepts (leading to simpler explanations), and (ii) significantly increases the causal effect of concept interventions on class predictions, making the model more responsive to these changes. Besides, the ability to perform task-driven interventions may provide actionable insights to improve CF-CBMs² predictions whenever users observe mispredicted class label but do not know how to intervene on concepts. Finally, the proposed latent variable approach enables to efficiently model concept dependencies while generating multiple counterfactuals for given input. This represents key advantage w.r.t. iterative methods in practical applications where proposing a variety of interpretable counterfactuals on the fly represents a fundamental advantage.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
GD acknowledges support from the European Union's Horizon Europe project SmarCHAAGE (No. 101080965). PB acknowledges support from Swiss National Science Foundation projects TRUST-ME (No. 205121L, (51211__214911 and IMAGINE (No. TMPFP2 (RPP2__ FG has been supported by the Partnership Extended PE00000013- "FAIR Future Artificial Intelligence Research" Spoke "Human-centered AI". MG acknowledges support from Swiss National Science Foundation projects XAI-PAC (No. PZ0OP2 216405). GM acknowledges support from the KU Leuven Research Fund (STG/22/021, CELSA/24/008) and from the Flemish Government under the "Onderzocksprogrammn Artificièle Intelligentie (AI) Vlaanderen" programme. This work was also supported by the EU Framework Program for Research and Innovation Horizon under the Grant Agreement No 101073307 (MSCA-DN LeMuR).
REFERENCES
Abubakar Abid, Mert Yuksekgonul, and James Zou. Meaningfully debugging model mistakes using conceptual counterfactual explanations. In International Conference on Machine Learning, pp. 66-88. PMLR, 2022.
Mohammad Taha Bahadori and David Heckerman. Debiasing concept-based explanations with causal analysis. arXiv preprint 2xXi:2007.115000 2020.
Pietro Barbiero, Gabriele Ciravegna, Francesco Giannini, Mateo Espinosa Zarlenga, Lucie Charlotte Magister, Alberto Tonda, Pietro Lió, Frederic Precioso, Mateja Jamnik, and Giuseppe Marra. Inter- pretable neural-symbolic concept reasoning. In International Conference on Machine Learning, pp. 1801-1825. PMLR, 2023.
Pietro Barbiero, Francesco Giannini, Gabriele Ciravegna, Michelangelo Diligenti, and Giuseppe Marra. Relational concept bottleneck models. Advances Neural Information Processing Systems 37 (NeurIPS 2024), 2024.
Matthias Bauer and Andriy Mnih. Resampled Priors for Variational Autoencoders. In AISTATS, pp. 66-75, 2019.
Xi Chen, Diederik P. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, and Pieter Abbeel. Variational Lossy Autoencoder. In ICLR, 2017.
David Debot, Pietro Barbiero, Francesco Giannini, Gabriele Ciravegna, Michelangelo Diligenti, and Giuseppe Marra. Interpretable concept-based memory reasoning. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, (9(6): 141-142, 2012.
Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini. Michelangelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, Pietro Lio, and Mateja Jamnik. Concept embedding models: Beyond the accuracy-xxplinnablity trade-off. Advances in Neural Information Processing Systems, 35, 2022.
Asma Ghandeharioun, Been Kim, Chun-Liang Li, Brendan Jou, Brian Eoff, and Rosalind W Pi- card. Dissect: Disentangled simultaneous explanations via concept traversals. arXiv preprint arXiv:2105.15164, 2021.
Sofie Goethals, Kenneth Sörensen, and David Martens. The privacy issue of counterfactual ex- planations: Explanation linkage attacks. ACM Trans. Intell. Syst. Technol, 14(5), 2023. ISSN 2157-6904. doi: 10.1 145/3608482. URL btttp::///ooii.. http//i.orgg/../o/ wtw.oo///......1..1. 1145111555500 1145/3666482
Yash Goyal, Amir Feder, Uri Shalit, and Been Kim. Explaining classifiers with causal concept effect (cace). arXiv preprint arXiv: (it::1907. 1907,07777770 006650 2019.
Hangzhi Guo, Thanh H Nguyen, and Amulya Yadav. Counterne: End-to-end training of prediction aware counterfactual explanations. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 577-589, 2023.
11

---

Published as a conference paper at ICLR 2025
Victor Guyomard, Françoise Fessant, Thomas Guyet, Tassadit Bouadi, and Alexandre Termier. Venet: self-explainngg model for realistic counterfactual generation. In Massih-Rezz Amini, Stéphane Canu, Asja Fischer, Tias Guns, Petra Kralj Novak, and Grigorios Tsoumakas (eds.), Machine Learning and Knowledge Discovery in Databases, pp. 437-453, Cham, 2023. Springer International Publishing. ISBN 978-3-031-22387--5
David J. Hand and Robert J. Till. simple generalisation the area under the roc curve for multiple class classification problems. Mach. Learn., 45(2):171-186, oct 2001. ISSN 0885-6125, doi: 10.1023/A:10100228199881
doi 1ttp://0..002/::11..2 httppp/....o...o........///01111
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
J. D. Hunter. Matplotlib: A 2d graphics environment. Compuling in Science & Engineering, 9(3): 90-95, 2007. doi: 10.1 109/
Paul Jaccard. The distribution of the flora in the alpine zone. 1. New phytologist, 11(2):37-50, 1912.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference machine learning, pp. 2668-2677. PMLR, 2018.
Eunji Kim, Dahuin Jung, Sangha Park, Siwon Kim, and Sungroh Yoon. Probabilistic concept bottleneck models. arXiv preprint 230.0...0.. 2023.
Diederik P Kingma and Max Welling. arXiv:1312.61 14, 2013.
Auto-encoding variational bayes.
arXiv preprint
Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2022. URL https
7/arxiv. www.v//rggabbb//bb //2 6114
Alexej Klushyn, Nutan Chen, Richard Kurle, Botond Cseke, and Patrick van der Smagt. Learning Hierarchical Priors in VAEs. In NeurIPS, pp. 2866-2875, 2019.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference machine learning, pp. 5338-5344 5348. PMLR, 2020.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research). URL. httt:httt:///ttt //wt:: cs wwwottorooooooo......oooo .corontoo.dd//kkii......... edu/=kriz. z/ far.html
Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, and Marcin Detyniecki. Issues with
post-hoc counterfactual explanations: discussion, 2019.
Christos Louizos, Uri Shalit, Joris M Mooij. David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL httpss://proceedinngs ptoppp///pppppeeeieeeeeeeeeooooo file/ (ttp://beeeeeeeeooeeeeeeeeeeeee 2ecereeaddddeddee7eeeeeedddddddd wwt.ss22a1--aappp..a..eaee Pwppp.ppppp........ .pff
files/ s1les//pperr20117
Lars Maalse, Marco Fraccaro, Valentin Liévin, and Ole Winther. BIVA: Very Deep Hierarchy of Latent Variables for Generative Modeling. In NeurlPS, pp. 6548-6558, 2019.
Divyat Mahajan, Chenhao Tan, and Amit Sharma. Preserving causal constraints counterfactual explanations for machine learning classifiers, 2020.
Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Lue De Raedt. DeepProbLog: Neural Probabilistic Logic Programming. In NewrlPS, pp. 3753-3763, 2018.
Emanuele Marconato, Andrea Passerini, and Stefano Teso. Glancenets: Interpretable, leak- concept-bard based models. Advances in Neural Information Processing Systems, 15521222221222 2022.
12

---

Published as a conference paper at ICLR 2025
Emanuele Marconato, Stefano Teso, and Andrea Passerini. Neuro-symbolic reasoning shortcuts: Mitigation strategies and their limitations. arXiv preprint arXiv:2303 2023.
Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentanglement testing sprites dataset. https://github..om//eeee http:///w////eeeeii///ssttsstaaa 2017.
Elconora Misino, Giuseppe Marra, and Emanuele Sansone. Vacl: Bridging variational autoencoders and probabilistic logic programming. Advances in Neural Information Processing Systems, 35: 4667-4673 2022.
Francesca Naretto, Anna Monreale, and Fosca Giannotti. Evaluating the privacy exposure of interpretable global explainers. 2022 IEEE 4th International Conference on Cognitive Machine Intelligence (CogMI), pp. 13-19, 2022. doi: 10.1 10110000C0MM155444222220000
Daniel Nemirovsky, Nicolas Thiebaut, Ye Xu, and Abhishek Gupta. Countergan: Generating realistic counterfactuals with residual generative adversarial nets, 2021. URL /tl/atttp////rx..o00////
abs/2009 05199
Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, and Tsui-We Weng. Label-free concept bottleneck models, 2023.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019. URL URLL_tttpp//pppes.eeziiisss.se
901 5
an http::/aaa.a....aa...e...a..aaa.
Library.
pdf
Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci. Learning model-agnoste counterfactual
explanations for tabular data. In Proceedings The Web Conference 2020, WWW '20. ACM, April 2020. doi: 10.1 00.1445/36642333388000
URL http: dww.dxxooooo.oooooo org httpg//1.111533666410
3380087
Martin Pawelezyk, Himabindu Lakkaraju, and Seth Neel. On the privacy risks of algorithmic recourse, 2022.
Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, 2000.
Judea Pearl, Madelyn Glymour, and Nicholas Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830 2011.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https
7/arxiv.
00020
Ali Razavi, Aäron van den Oord, and Oriol Vinyals. Generating Diverse High-Fidelity Images with VAE-2. In NeurIPS, pp. 14837-14847, 2019.
Matthew Richardson and Pedro Domingos. Markov logic networks. Machine learning, 62:107-136. 2006.
Piotr Romashov, Martin Gjoreski, Kacper Sokol, Maria Vanina Martinez, and Marc Langheinrich. Baycon: Model-agnostic bayesian counterfactual generator. In Lud De Raedt (ed.), Proceedings of the First International Joint Conference Artificial Intelligence, IJCAI-22, pp. 740-746.
International Joint Conferences on Artificial Intelligence Organization, 2022. doi: 10.24963/ Main Track.
2jc2i.2022/104. URLptttss: //dp:s::///d.i.o http:////ooooo///////// 20.....29433.... htt.:/aaai..22210000 2022/104
13

---

Published as a conference paper at ICLR 2025
Casper Kaae Sønderby, Tapani Raiko, Lars Maalde, Søren Kaae Sønderby, and Ole Winther. Ladder Variational Autoencoders. In NeurIPS, pp. 7738-3744 2016.
Wolfgang Stammer, Antonia Wüst, David Steinmann, and Kristian Kersting, Neural concept binder, 2024. /ttppp////aa/...o/////aa//////// 0994499999999
Arash Vahdat and Jan Kautz. NVAE: Deep Hierarchical Variational Autoencoder. In NeurIPS, 2020.
Aäron van den Oord, Oriol Vinyals. and Koray Kavukcuoglu. Neural Discrete Representation Learning. NeurIPS, pp. 6306-6315, 2017.
Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv. JL. & Tech., 31:841, 2017.
Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Be- longie, and Pietro Perona. Caltech-uesd birds 200. Technical Report CNS-TR-20 TR-201, Caltech, 10. CUB-200
2010. http://////ogoooeee///////////// pdf, http: www. vi sion. wwt.w./ww...aclccccceooooooo edu vi wtt.:sssspppaaaaaaaaaaaaa C/UB 200 html
Kevin Xia, Yushu Pan, and Elias Bareinboim. Neural causal models for counterfactual identification and estimation, 2022. URL htLphhtp:::aarat http:/::///xxxv.o//////./ http:/bb//2211.00.3
Xinyue Xu, Yi Qin, Lu Mi, Hao Wang, and Xiaomeng Li. Energy-based concept bottleneck models: unifying prediction, concept intervention, and conditional interpretations. arXiv preprint arXiv:2401. 14142, 2024.
Chih- Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Raviku- mar. On completenes--wware concept-based explanations in deep neural networks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, pp. 20554-20565. Curran Associates, Inc., 2020. URL httpss://proceeddnngg nwwrips.n.ipp..ee... wctpppppp.ppppppppppeppepppppppe files//aperr2220/ FLL-0222EE77Se25555422222EEEE222
pdf
Kihyun You, Jawook Gu, Jiyeon Ham, Beomhee Park, Jiho Kim, Eun K. Hong, Woonhyuk Back, and Byungseok Roh. CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-training,
doi: 10.1007) 978- 378-3-031-43895-0 URL 0tt.a//////////..././///00000000 10
pp. 101-111. Springer Nature Switzerland, 2023. ISBN 9783031438950,
doi
Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. In ICLR 2022 Workshop on PAIR*2Struct: Privacy, Accountability, Interpretability, Robustness, Reasoning on Structured Data, 2022. URLhtttps: /att:p::/openzzevie..
qg
Mateo Espinosa Zarlenga, Pietro Barbiero, Zohreh Shams, Dmitry Kazhdan, Umang Bhatt, Adrian Weller, and Mateja Jamnik. Towards robust metrics for concept representation evaluation. arXiv preprint arXiv:2301.10336 2023.
Anna Zawacki, Carol Wu, George Shih, Julia Elliott, Mikhail Fomitchev, Mohannad Hussain, Paras Lakhani, Phil Culliton, and Shunxing Bao. Siim-acr pneumotho- rax segmentation 2019, 2019. URL hhtpps: http://ww.kgggllle..// wwt. conm/competitionss
siim cer-ppeumttthoax horax segmentation/
14

---

Published as a conference paper at ICLR 2025
A AMMORTIZED INFERENCE AND ELBO DERIVATION
To derive the ELBO objective function defined in Section log-likelihood of the tuple (c, y,y'):
wc start from the maximization the
logplc, U.V. log
plc, d v. 2'dd dadæ'
where 2,22 factorizes as in Section g(2, z', ,,,,,,,,,,,,.. (,,,,,,,, = ( infer
3.1
We consider the variational approximation
thus exploiting the observation a to better
y
We have:
v((cdc))))))))) (, ple v... d' dzdz
logp(c,8. = log
and, given the Jensen's 's inequality, we obtain:
Dogp(c,u.3/) 2 f /
log plc.d....l..,22))
de dd dzdz'
that can be rewritten as:
log v') 2Rg[log[cl=]]+
logp(y)c)+
E oe (log(("(cc))-
DeL [x))
B TAKE THE BEST BET OR CONTEMPLATE MULTIVERSE?
All these functionalities can be used in two ways: the "best bet" mode and "multiverse" mode. the "best bet" mode, CF-CBMs provide maximum posteriori estimates by taking the mode of posterior distributions. Using this mode, users will be provided with the most probable counterfactual for each input. In "multiverse" mode instead, CF-CBMs sample from posterior distributions. This allows the model to generate multiple counterfactuals for each input, thus allowing users to choose among variety of actions/interventions. For instance, in a healtheare scenario, the "best bet" mode might suggest the most probable alternative treatment plan based on patient data, while the "multiverse" mode could present range of treatment strategies under varying clinical conditions, each representing different plausible future.
C EXPERIMENTAL DETAILS
C.I DATA TASK SETUP
In experiments five different datasets commonly used to evaluate CBMs:
15

---

Published as a conference paper at ICLR 2025
dSprites (Matthey et al. 2017) It is composed of images of one of three objects (square, ellipse, heart) in different positions and of different sizes. Starting from it, we design a binary task, which is to predict if in the image there is at least square combine initial images to obtain new ones with more than one object. The concepts for this task are: (1) the presence of square, (2) the presence of an ellipse, (3) the presence of heart, (4) the presence of two objects, (5) objects are red, (6) objects are green, (7) objects are blue.
heart. We also
The task of this dataset is to predict the sum of two MNIST digits (Deng 2012). The concepts of this task are the one-hot encoding of the first and the second digits concatenated together.
MNIST addition
CUB (Welinder al. 2010) It contains pictures of birds and the final task is to predict their species (among 200 species). The concepts are 118 bird features that human annotators selected.
CIFAR10(Krizhevsk) et a This dataset contains images of objects, with the task of classifying them into one of 10 classes. Concepts are automatically extracted using the method proposed in Oikarinen et al. (2023), resulting in 143 concepts.
et al. 2019) This dataset contains X-ray scans of lungs, with the task of determining whether the patient has pneumothorax. Concepts are automat- ically extracted following Oikarinen et al. (2023), leveraging CXR-CLIP, resulting in 19
SIIM Theumothorax(Zawaat
concepts.
C.2 EVALUATION METRICS
In our analysis we use the following metrics. What? (prediction generalization): we compute the Area Under the Receiver Operating Characteristic Curve (Hand & Till 200T) for concepts and tasks (ROC AUC (f). How? (intervention impact): we compute the impact of each concept on the downstream task, inspecting the task predictor f (Yuksekgonul et al. (022). For each class, we look at the related weights of the task predictor Linear layer), one weight for each concept. In this way, it is visible the impact of each concept on each specific class. Then, we evaluate the impact of intervening on confounding concepts for class predictions (Koh et al.] 2020). We modify the dSprites task to highly correlate the colors with the shapes and therefore the task label. For instance, 85% of samples with positive label (presence at least one square or heart), the objects are green, while in 85% of the samples with a negative label, the objects are red blue. Why not? (counterfactual actionability): drawing from previous works on counterfactuals, we compute: (i) the (t) (Wachter al. (017) by checking whether the model predicts the desired class labels based on the generated counterfactual; (ii) the proximity (4) (Pawelczyk et al. 2020) which evaluates counterfactuals" "reliability" their similarity w.r.t. training samples; (iii) the time (4) (Romashov ct al. 2022) to generate a counterfactual for set of samples.
In addition, we propose the following metrics assess the quality of counterfactuals: (i) the A- Sparsity (L) which evaluates user's "wasted efforts" by counting the number concepts changed (Guo)
al. 2023) w.r.t. the minimal number of changes that would have generated counterfactual according to the dataset:
A ASparsity = (OptimalSparsity - Sparsityl
where OptimalSparsiin represents the mean Hamming distance between each concept vector in the test set and the closest concept vector of drawn random class y/, found using "brute-force" approach and Sporsity represents the mean Hamming distance between the predicted concepts and the predicted concept counterfactuals 2: (ii) the "plausibility" (Wachter et al. 2017) as the Intersection over Union (loU (1)) (Jaccard] 1912) between counterfactuals and ground-truth concept vectors; (iii) the variability (t) which evaluates counterfactuals "diversity", as the cardinality ratio between the of counterfactuals generated and the set of training concept vectors
V Variability
where |d/| represents the cardinality of the set composed by all é' while |c| represents the cardinality of the set composed by all e (iv) finally, measure the concept accuracy of generated interventions
16

---

Published as a conference paper at ICLR 2025
Table 5: Model Hyperparameters shared across all models. During the training, we select the best checkpoint for each model.
Hyperparams
dSprites
MNIST add
CUB
CIFARIO
SIIM Pneumothorax
Epochs
75
150
150
30
100
Learning rate
0.005
0.005
0.005
0.005
0.005
Hidden size
128
128
128
128
128
Batch size
1024
1024
1024
1024
1024
(Ace Int. (1)) w.r.t ground-truth optimal interventions that evaluate the model's ability to generate potential concept interventions. Following [Eppnnoaa Zarlenga et al. (2022), inject noise predicted concepts ė to reduce task accuracy (randomly flip the value of some concepts), then we sample counterfactuals, conditioning on the ground-truth label y, and use them to fix mispredicted concept labels automatically.
Eiec la-e
AccInt.
[c]
where c, represents the concept ground truth for the sample while 27 represents the counterfactual generated for the sample that tries to recover c.
C.3 BASELINES AND IMPLEMENTATION DETAILS
Hyperparameters To train our models and the baselines we selected the best hyperparameters according to our experiments. Table 3 the number of epochs, learning rate, and embedding size in the latent space, batch size for each dataset. They are shared among the baselines, and we took the best checkpoint out of the entire training for each model. In addition, Table Table/olillun Gillustrates the parameters used to weight each term in the loss for all the methods. These hyperparameters were chosen through validation on subset of the training to achieve the best possible trade-off across all metrics important for counterfactual evaluation, such as validity, proximity, and sparsity, for both our method and the baselines. While our method introduces more hyperparameters, it provides greater flexibility to specify the desired objective. As counterfactual metrics often conflict, creating inherent trade-offs (as shown in our experiments), there is one-size-fits-al solution. Thus, optimizing the counterfactual generator based on the metric interest crucial, and our method facilitates this. Each hyperparameter in the loss has specific role in steering the model toward particular behavior:
A (task-related) and X (concept-related) are important in all configurations as it remains the main goal of the model.
À prioritizes validity, encouraging counterfactuals with predictions matching yr, though potentially compromising proximity or sparsity.
Aa and As emphasize generating more realistic counterfactuals, potentially reducing proxim- ity, which may trade off against validity and sparsity.
As and A promotes sparsity by reducing the number of changes from the input, which may trade off against validity. However, we observed that è has minimal influence on the optimization process, with 77 being the key hyperparameter driving this behaviour.
This flexibility makes our model more versatile and practical compared to baselines, which have limited or partial support for these trade-offs (e.g., VAECF has the possibility be flexible on validity). To show these behaviours we perform an ablation study on MNIST add that is shown in Table 7
Counterfactual CBM implementation details To improve the training process of our models we decided to relax some assumptions we made in Section 3.1 Employing a Bernoulli distribution to predict concepts and Categorical distribution for the class prediction would make the training process more difficult, ruining the gradient in the backpropagation process. Therefore, we predict them directly with the output of the concept predictor and the task predictor applying Sigmoid function and a Softamx on top of them, respectively. To optimise them, we choose to use a Binary Cross Entropy loss for the concept loss, and Cross Entropy Loss for the task loss. All this is
17

---

Published as a conference paper at ICLR 2025
Table 6: Loss weights
Method
Dutuset
Loss Term
dSprites
MNIST add
CUB
CIFARIO
SIM Pneumothorax
Concept
1.0
1.0
1.0
1.0
1.0
Task
0.1
0.1
0.1
0.1
0.1
Reconstruction
1.0
1.0
1.0
1.0
1.01
CCHVAB
KL
0.5
0.5
0.5
0.5
Reconstruction
3.0
1.3
1.0
1.3
VABCF
KL
1.0
1.0
2.0
1.0
1.0
Validity
15.0
15.0
15.0
20.0
Concept
1.0
VCNET
Task
0.5
0.5
0.5
0.5
0.5
Reconstruction
0,5
0.5
0.5
0.2
0.8
Concept
10.0
10.0
10
10.0
10.0
Task
0.7
1.0
0.1
1.0
1.01
Validity
0.3
0.02
0.2
CF-CBM
KL
1.2
0.2
2.0
KL
1.2
2.0
0.2
2.0
2.0
Prior distance
1.0
1.7
0.2
1.7
Posterior distance
0.6
0.55
0.03
0.4
Table 7: Performance comparison for different objective focuses with varying values on the MNIST add dataset.
Objective Focus
A
04 & Às
Àς
Validity (%)
Sparsity (%)
Proximity (%)
Higher Jocus validity
0.4
2
0.55
99.1 0.2
19.0 0.4
4.3
Higher Jocus proximity
0.2
4
0.55
93.6 1.6
14.2 0.4
3.6 3.6:0..0
Higher focus on sparsity
2
0.70
74.4 0.01
=
2.6 0.0
This means that all component are trained together. However, it is also possible train the model in a post-hoc manner-firg training the concept encoder and predictor, followed by the counterfactual generator, or performing a warm-up phase for the encoder and predictor to better initialize their weights before continuing with joint training.
done in jointly training to fully leverage the advantages outlined in Section 5.2
Code, licenses and hardware For our experiments, we implement all baselines and methods in Python 3.9 and relied upon open-source libraries such as PyTTrrch 2.0 (Paszke al. 2019) (BSD license), PytorchL.ghtning v2. 1.2 (Apache Licence 2.0), Sklearn 1.2 (Pedregosa et al. 2011 (BSD license). In addition, we used Matplotlib (Hunter 2007) 3.7 (BSD license) to produce the plots shown in this paper. The datasets we used are freely available on the web with licenses: dSprites (Apache 2.0) MNIST (CC 3.0 DEED) and CUB (MIT License). We will publicly release the code with all the details used to reproduce all the experiments under an MIT license. The experiments were performed on a device equipped with an M3 Max and 36GB of RAM, without the use a GPU. Approximately 80 hours of computational time were utilized from the start of the project, whereas reproducing the experiments detailed here requires only 10 hours.
D FURTHER METRICS AND RESULTS
In addition to the result present in Section 5 Table shows the task and concept performance of our model and the baselines. Moreover, Figure with different levels of noise injected at the concept level. is visible how CF-CBM achieves stable Acc Int. results with different level of noise injected at the concept level. This is not the case for the other baselines. Figure] shows the distribution of the CaCE score obtained in MNIST add and CUB in the post-hoc and joint fashions. It allows to directly see the distributions used to compute the metrics in Table 2 Finally, Figure g shows the time efficency of all the method in generating counterfactuals. Generation efficiency respect, CF-CBMs significantly outperform iterative models such as C-CHVAE and BayCon. These models quickly struggle in large search spaces involving high number of concepts: generating
o illustrates the Ace Int. achieved by all models
key feature for real-tine rall-tmme applications. In this
18

---

Published as a conference paper at ICLR 2025
Table 8: CF-CBM attains generalization performance of standard CBMs concepts and class labels.
dipittes
MNNST add
CUI
CHARII
SIM Phoumothona
(1)
Corsupt
Conaggt
(5)(C)
Chencopt
(1::41)
Cancpp
Blak Rox
13.00 C-00
96.44
81.1 4 0.0
09.7 +0.0
95.1 012
CBM
05.82:0.11
58, 28.4
20.7-2:100 [1.
50.0 :: 0.0
25
11.3 20000
08.1
88.4 4 D.6
99.6 i: 0.1
CEM
99.5-2-0.1
98.01-0.1
$8.4 0.0
09.3 010
10.3 2 0.1
85.7 2
00.5 0..0
98.2 3-0.0
91.2-1 0.1
99.5 it 0.0
Boytin
09.5-200.
98..2:0.1
98. 28.4
39.7 2: [1)
10.1 =
55.3 20.2200.
05.4 2:0.
2:0.0
89.3 4 08
98.6 2: B.1
0.1
50.5
VESET
98.0 0.1.1
68.24:000
0.0.
OFCRM
49.7.4000
9..74-01
97.04.011
98.35-00
09.4 9.0
96.6 16.66.9.0 9.00
0.5
98.1 0.0
dSprites
MNIST add
CUB
(1)
111
1857
Acc. (%)
-20
-90
Noise (%)
Noise (%)
Noise (%)
Figure 6: CF-CBMs attain similar results while applying different levels of noise on predicted concept labels.
counterfactual takes more than minute dSprites (7 concepts) and up to few hours on CUB (118 concepts). CF-CBMs instead scale well with the size of the data set by requiring less than a second on dSprites and few seconds on CUB, on par with VAECF and VCÑet. This efficiency enables wider range of applications requiring high number of concepts and quick feedbacks.
E COUNTERFACTUALS
Tables Jand 10 present the top-3 most common counterfactual examples for each method on dSprites, MNIST addition, and SIIM Pneumothorax, respectively. The tables display only the active concepts to maintain clarity, as including all concepts would make them difficult to interpret. All other concepts are considered inactive. Examining these three tables especially the first two, which are casier to understand without requiring Nomain-specific knowledge-in it is clear that the counterfactuals generated by CF-CBM are the most reasonable. In Table CF-CBM consistently generates feasible states by modifying the minimum number of concepts. In contrast, VCNET fails to activate any color concepts, leading to suboptimal changes and unfeasible states. Similarly, CCHVAE activates the "Two obj" concept, which irrelevant and fails to influence the prediction, effectively wasting change. For MNIST addition, CF-CBM once again generates valid counterfactuals, while other baselines occasionally produce counterfactuals with zero, one, or three active concepts- an impossible scenario, as each sample must always contain two digits. This issue is particularly prominent in VCNET and VAECE, which rely heavily on the fuzziness of concept values for gecision-making allowing scenarios where no active concepts still predict different labels. Lastly, we include single example of counterfactuals per model for CUB and CIFAR 10 for completeness. Due to the large number of concepts in these datasets, presenting multiple examples in organized format would be challenging.
CUB:
Baycon:
Factual: Bill Shape Hooked (Seabird), Wing Color - Grey, Underparts Color White, Breast Pattern Solid, Breast Color White, Throat Color White, Eye Color Black,
Bill Length- Same as Head, Nape Color- White, Belly Color- White, Size Mediium
(9-16 in), Back Pattern Solid, Tail Pattern Solid. Belly Pattern Solid, Wing Pattern
Solid - Task: Brandt Cormorant
Coonterfactual: CF: Bill Shape Hooked (Seabird), Wing Color- Grey, Upperparts
Color Black, Underparts Color Black, Breast Pattern Solid, Breast Pattern Striped,
Upper Tail Color White, Head Patter Plain, Breast Color Black, Throat Color-
19

---

Published as a conference paper at ICLR 2025
CUB
MNIST add
0.25
0.15
0.20
Frequency
0.15
0.10
0.10
0.05
0.00
0.00
0..0
0.02
0.04
0.0
0.2
0.4
Causal Concept Effect
Causal Concept Effect
Joint (ours)
Post-hoc
Figure 7: The distribution of the CaCE score of the CUB and MNIST add dataset obtained by the end-to-end
(Joint) approach and by the post-hoc one.
dSprites
MNIST add
CUB
CIFAR10
SIIM PNEUMOTHORA
12
37
10
122
10
27
0177
31
10
CF-CRM CRM RayCon
CBM CCHVAE
CRM VAECF
CBM VENET
Figure 8: CF-CBMs efficiently generate counterfactuals on the fly, on par with the fastest generators.
Yellow, Eye Color Black, Bill Length Same as Head, Forehead Color White, Under Tail Color Black, Nape Color Yellow, Belly Color Grey, Belly Color - Black, Size Medium (9- 16 in), Back Pattern Solid, Tail Pattern Solid, Belly Pattern Solid, Leg Color Black, Wing Pattern Solid - Task: Black Footed Albatross
CCHVAE:
Factual: Bill Shape Cone, Wing Color Brown, Upperparts Color - Brown, Upperparts Color Buff, Underparts Color White, Breast Pattern Striped, Back Color Brown, Tail Shape - Notched, Upper Tail Color Brown, Upper Tail Color Buff, Breast Color White, Throat Color White, Eye Color Black, Bill Length Shorter than Head, Forehead Color Brown, Belly Color White, Wing Shape Rounded, Size Small (5-9 in), Body Shape Perching- like, Back Pattern Striped, Primary Color Brown, Crown Color Brown, Wing Pattern Striped, Task: Purple Finch
Counterfactual: Bill Shape Hooked, Wing Color Black, Upperparts Color Black, Underparts Color Black, Breast Pattern Solid, Back Color Black, Upper Tail Color Black, Head Pattern Plain, Breast Color Black, Throat Color Black, Eye Color Black, Bill Length Shorter than Head, Forehead Color Black, Under Tail Color Black, Nape Color Black, Belly Color- Black. Wing Shape Rounded, Size Small (5-9 in), Body Shape Perching- like, Back Pattern Solid, Tail Pattern Solid, Belly Pattern Solid, Primary Color Black, Leg Color Black, Bill Color Black. Crown Color Black, Wing Pattern Solid, Task: Shiny Cowbird
VAECF:
Factual: Breast Pattern - Solid, Eye Color Black, Bill Length Shorter than Head, Size Small (5-9 in), Size Very Small (3-5 in). Back Pattern Solid, Tail Pattern Solid, Belly Pattern Solid, Leg Color Black, Bill Color Black, Wing Pattern
Multi-Colored Task: Rufous Hummingbird
20

---

Published as a conference paper at ICLR 2025
Counterfactual: Wing Color Black, Upperparts Color Black, Underparts Color
Black, Breast Pattem Solid, Back Color Black, Upper Tail Color Black, Head Pattern Plain, Breast Color Black, Throat Color Black, Eye Color- Black, Bill Length Same as Head, Forehead Color - Black, Under Tail Color Black, Nape Color Black, Belly Color Black, Back Pattern Solid, Tail Pattern Solid, Belly Pattern
Solid, Primary Color Black, Leg Color Black, Bill Color - Black, Crown Color-
Black, Wing Pattern Solid Task: American Crow
VCNET:
Factual: Underparts Color White, Breast Pattern Solid, Eye Color Black, Bill
Length Shorter than Head, Belly Color White, Size Small (5-9 in), Body Shape
Perching-like, Back Pattern Solid, Belly Pattern Solid Task: White Pelican
Counterfactual: Bill Shape Dagger, Wing Color Grey, Wing Color White, Up-
perparts Color Grey, Upperparts Color- White. Underparts Color White, Breast Pattern Solid, Back Color Grey, Back Color White, Upper Tail Color White,
Head Pattern Plain, Breast Color - White, Throat Color White, Eye Color Black,
Bill Length Same as Head, Forehead Color White, Under Tail Color- White, Nape Color White, Belly Color White, Size Medium (9-16 in), Back Pattern Solid,
Back Pattern Multi-Colored, Tail Pattern Solid, Belly Pattern Solid, Primary Color
Grey, Primary Color White, Crown Color White, Wing Pattern Solid Task: Ivory Gull
CF-CBM:
Factual: Bill Shape Cone, Wing Color Brown, Wing Color Buff, Upperparts Color Brown, Upperparts Color - Black, Upperparts Color Buff, Underparts Color White, Breast Pattern Striped. Back Color Brown, Back Color- Buff, Tail Shape Notched, Upper Tail Color Brown, Upper Tail Color Buff, Breast Color Buff, Throat Color Buff, Eye Color Black, Bill Length- Shorter than Head, Under Tail Color Buff, Wing Shape Rounded, Size Small (5-9 in), Body Shape Perching-like, Back Pattern Striped, Belly Pattern Solid, Primary Color Brown, Bill Color Buff, Wing Pattern Striped Task: Harris Sparrow
Counterfactual: Bill Shape Dagger, Wing Color Grey, Wing Color White, Upper- parts Color - Grey, Upperparts Color White, Underparts Color White, Breast Pattern Solid, Back Color- Grey, Back Color White, Upper Tail Color White, Head Pattern Plain, Breast Color - White, Throat Color White, Eye Color Black, Bill Length-
Same as Head, Forehead Color White, Under Tail Color White, Nape Color White, Belly Color White. Size Medium (9-16 in), Back Pattern Solid, Belly Pattern Solid, Primary Color Grey, Primary Color White, Crown Color White Task:
Western Gull
CIFAR10:
Baycon:
Factual: Hunter, beak, bed, birdfeeder, birdhouse, bit, branch, cage, captain, cat bed, cat food dish, cat toy, collar, crew, dashboard, deck, dog bowl, fly. food bowl, forest, four-legged mammal, gear shift, green or brown body, grille, hitch, lead rope, leaf, litter box, long neck, mane and tail, mast, mosquito, nest, net, nose, pedal, person, pointed front end, pond, port, reddish-brown coat, rifle, scratching post, seat, seatbelt, short, stocky build, small, lithe body, spider, steering wheel, tail, tire, tough, durable frame, tree, wet nose, wheel, wide mouth, worm, an engine, animal, cargo, engines, engines on the wings, feathers, food, four legs, four round, black tires, fur, grass, hooves, insects, large sails or engines, large, brown eyes, large, bulging eyes, lily pads, living thing, long ears, machine, mammal, multiple decks, multiple sails, nature, object, organism, passengers, pointed ears, quadruped. side windows, the ocean, trees, two headlights, vertebrate, vessel, webbed feet, whiskers, white spots on the coat, wings, woods Task: Cat
Coontterfactual: Hunter, beak, bed, beetle, bird feeder, birdfeeder, birdhouse, bit, boat, branch, captain, cat food dish, cat toy, collar, copilot, crew, dashboard, deck, dog bowl, field, flat back end, flat front and back, flight attendant, fly, food bowl, gear
21

---

Published as a conference paper at ICLR 2025
shift, grille, hitch, large size, leaf, leash, lily pad, litter box, long neck, mast, mosquito, net, nose, passenger, pedal, person, pilot, pointed front end, port, rider, rifle, road, rudder, runway, saddle, seat, seatbelt, spider, steering wheel, suitcase, tail, tire, toy, tree, wet nose, wheel, wide mouth, windshield, worm, an engine, animal, antlers, cargo, engines, engines the wings, feathers, food, four round, black tires, fur, grass, hooves, insects, landing gear, large sails or engines, large, brown eyes, large, bulging eyes, lily pads, living thing, machine, mammal, multiple decks, multiple sails, object, organism, passengers, pointed ears, quadruped, side windows, the ability to fly, the ocean, transportation, trees, two headlights, vertebrate, vessel, watercraft, webbed feet, whiskers, white spots on the coat, wings Task: Airplane
CCHVAE:
Factual: boat, captain, deck, dock, port, large sails or engines, multiple decks, multiple sails, the ocean, vessel, watercraft Task: Ship
Coonterfacttall: None Task: Horse
VAECF:
Factual: beak, bed, bed for carrying cargo, beetle, bird feeder, birdfeeder, birdhouse, bit, boat, branch, cage, captain, cat bed, cat toy, collar, copilot, crew, dashboard, deck, dock, dog bowl, field, flat back end, flat front and back, flight attendant, fly, gear shift, grille, halter, hitch, large body, large, metal body, lead rope, leash, litter box, long neck, mast, mosquito, net, pedal, person, pilot, pointed front end, port, rifle, road, rudder, runway, saddle, scratching post, seat, seatbelt, short, stocky build, small, lithe body, spider, steering wheel, suitcase, tail, tire, tough, durable frame, toy, tree, wheel, windshield, worm, an engine, cargo, engines, engines on the wings, four legs, four round, black tires, four wheels, fur, grass, hooves, insects, landing gear, large sails or engines, large, bulging eyes, long hind legs for jumping, long, thin legs, machine, mammal, multiple decks, multiple sails, object, organism, passengers, quadruped, the ability to fly, the ocean, transportation, trees, vertebrate, vessel, watercraft, webbed feet, wings- Task: Airplane
Counterfactual: runway, the ability to fly Task: Horse
VCNET:
Factual: barn, bed, bed for carrying cargo, beetle, cab for the driver, copilot, dashboard, deck, driver. flat back end, flat front and back, gear shift, green or brown body, grille, hitch, large body, large, metal body, large, rectangular body, passenger, pointed front end, road, seatbelt, short, stocky build, steering wheel, suitcase, tire, tough, durable frame, trailer, wheel, windshield, an engine, cargo, engines, four round, black tires, four wheels, multiple decks, passengers, side windows, taillights, transportation, two headlights, vessel Task: Cat
- Counterfactuall bed for carrying cargo, boat, captain, crew, deck, dock, flat front and back, mast, port, rudder, engine, cargo, engines, large sails engines, multiple decks, multiple sails, passengers, side windows, the ocean, transportation, vessel, watercraft Task: Ship
CF-CBM:
Factual: bed for carrying cargo, boat, captain, deck, dock, mast, port, rudder, engine, engines, large sails or engines, multiple decks, multiple sails, passengers, side windows, the ocean, transportation, vessel, watercraft- Task: Ship
Counterfactual: barn, bridle, field, four-legged mammal, halter, lead rope, leash, mane and tail, pasture, reins, rider, saddle, animal, four legs. hooves, long ears, long hind legs for jumping, long, thin legs, quadruped, white spots on the coat Task: Horse
22

---

Published as a conference paper at ICLR 2025
Table Comparison the most three common counterfactuals different models on dSprites dataset. are showing just the active concepts.
Model
Factual
Counterfactuals
Concepts
Task
Concepts
Task
Hcart Green
Ellipse Heart Green
0
BayCon
Square Green
Ellipse Heart Green
Ellipse Blue
Square Ellipse Blue
Heart Green
Ellipse Red
CCHVAE
Heart Green
1
Ellipse Green
leart Green
1
Ellipse Two obj Green
0
Heart Green
1
Ellipse
0
VAECF
Heart Green
I
Ellipse Red
0
Square Blue
1
Ellipse Blue
0
Heart Green
I
Ellipse
0
VCNET
Heart Red
I
Ellipse
0
Ellipsc Red
0
Square Heart
Heart Green
Ellipse Green
CF-CBM
Heart Red
Ellipse Red
Square Heart Two obj Green
Ellipse Two obj Green
Table 10: Comparison of the most three counterfactuall for different models the MNIST add dataset.
We showing just the active concepts.
Model
Factual
Counterfactuals
Concepts
Task
Concepts
Task
10
4
BayCon
2, 0
2
0, 4, 0
8,7
15
4, 0,
1,2
3
0
CCHVAE
2.9
11
9, 9
18
1,1
2
17
1,
2
16
VAECF
7.3
10
2,
3
Nonc
10
2,9
10
None
12
VCNET
0,6
7
9
3.8
7
None
15
6.
12
9, 9
18
CF-CBM
5.0
5
7. 8
15
0.
1
2
23

---

Published as a conference paper at ICLR 2025
Table 11: Comparison of the most three common countorfactuals for different models on the X-ray dataset. We are showing just the active concepts.
Midell
Fartall
Contrefacteol
Czncapo
Crecepos
tryphyeraaesedunne.. Ryputrriedd Utllapee kny. Mettwerul
Dasphrggei:: Pleara oftvcionn
BayCos
Callapeod Iang, Midzantral dstr, Lang conutitaton Atr dunteren purters
Dippiigrenti
Exppyeerrtns
Satestmene erptysers, Plaari citaaen Emptrpeerraees changes
Cantas saacow atecmmatiy, Deep vaice age, Hadandenuty changes
Vieiie piaral
ling makrgg Lang theuss Matuetral
bymeerr
OCIIAA
Nore
Nose
Day allas ign, Hepcislaaal lotg, Seauinti anlyio
Emplosoraions chango, Hepnisleed lang
Plezral clitsces
Enpbyyenaava Air dosiaaiie
VNBCF
Sebettatora orgtrpens, trppoyeenatess chanec-
sakse agg
Collgpol Ins,, Plond elioa Lasz uxoolalaiis
Explosensices chango, Rasinicasity clarges Air doniiiataion
Plezral clissce
Emploperaices Hepecstladd
VONET
Celigoal lee,,
cnnelabiios
Sdisataeees erplicicuaa Explescusaioo clagges Heperstiaal leng
Emplo diagoo, Hoponullted
cocsoliaioss
Emptramaatn duzgo
Nac
CF CBM
Sclcutancssa caplesctus Explecctsiion changes
Abott lug nerkings lang Rhess Calla stalow Fnsmallty
Sdvarareces engliseeuu Exphysenanss charges Hypeisfaasling long
24


Reviews:
[{"review_id": "bnIlrMB3Tj", "reviewer": "Reviewer_hqCZ", "summary": "The authors extend the concept bottleneck model framework to allow simulation of counterfactual concepts and labels. The method is based on a variational approach that models concepts as latent confounders, and approximates this posterior using concept labels at training time. This extension opens the door to explaining prediction at a finer level of detail, focusing on intermediate concepts rather than pixel values.", "questions": "Additional comments/questions: \n* [L042-044] This is a bold claim....what about the field of XAI? Pearl's ladder of causality has been quite influential in deep learning (including XAI), so I would assume that any methods aimed at counterfactual explanations would address these questions to some extent.\n* I find the abstract a bit strange. Causal inference is a broad field attempting to answer (from a statistical perspective) the how and why questions, but it is not explicitly mentioned. Also, \"existing CBMs\" are mentioned in passing but the authors don’t say what they are or how they work. Maybe it would be better to more precisely situate the paper in the context of recent attempts to integrate causal inference with deep learning.  \n* [Sec 2] I don't insist, but it would be nice to see a citation acknowledging early efforts of using VAEs for causal inference [https://proceedings.neurips.cc/paper/2017/hash/94b5bde6de888ddf9cde6748ad2523d1-Abstract.html]\n* [Eqn 1, L123] I don't understand why sampling a counterfactual concept c' does not also require sampling a fresh set of covariates x'. It seems to me that for concepts that matter in the real world, you would need to update the observed features as the concept is manipulated. To ground this in the x-ray example, if the concept is changed from \"narrow joint space\" to \"bone spurs\", would the distribution over pixels not also need to be updated? Clarifying this point seems important to justify why the $p(x|z)$ term is ignored in the optimization [L152].\n* [L169] how are specific values for $\\lambda_i$ [Table 6] chosen? Was a validation set used for tuning? Is the method sensitive to these values? I'm not satisfied with \"these details are in the released code\" [L850]\n* [L260-265] The experiments inherit some datasets from prior papers, such as Koh et al 2020. Is there a reason why the OAI x-ray dataset from that paper was not used? X-rays are used as a working example throughout the paper [Fig 1, Fig 2, Sec 3.2] so this omission surprised me.\n* [Sec 4, Sec 5] How does the method do when asked to simulate concept pairings that are OOD w.r.t. its training data? I would imagine for the CUB dataset there will be combinations of bird characteristics that are not represented by real-life birds; is it possible to examine the predictions for these concepts?. The strength of human counterfactual reasoning comes in part from OOD generalization, e.g. imagining a pink elephant. Replicating this using ML seems quite difficult. In the end I still have questions around whether what is being done here is meaningfully different from structured generative/discriminative modeling [L503-509] with interpretable features that can be adjusted on the fly. I think this is certainly related to counterfactual reasoning, but may not encompass it completely.\n* [L516] If I express the counterfactual as E[Y|do(X)=X', Y=Y'] and the interventional as E[Y|do(X)=X'], then it would seem that counterfactual explainability methods can answer the \"how\" question by just marginalizing out the specific observation, i.e. E[Y|do(X)=X'] = E_{Y'}[ E[Y|do(X)=X', Y=Y']. You could replace the observation being a label (Y=Y') with the observation being a concept or other covariate as needed. Is there something wrong with this line of reasoning? Modeling counterfactuals seems more challenging than modeling interventions, so I am wondering if the authors have thought of how to go from one to the other in a post-hoc manner without designing a method that explicitly models all three levels of Pearls' ladder.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The authors tackle an extremely challenging problem (counterfactual reasoning using deep learning); making meaningful progress along this direction would have a big impact.\n\nThe authors show how training using a structured objective allows for test-time manipulations that shed light on how predictions are made\n\nThey extend CBM to include counterfactual sampling, which in principle allows a single model to estimate quantities at all three levels of Pearl’s ladder of causation.\n\nThe proposed method relies on concept labels, which may be difficult to collect in realistic settings.", "weaknesses": "The training objective is fairly complex and introduces many hyperparameters. The authors do not discuss how to tune these.\n\nThere is a disconnect between the working example of x-ray imaging and the types of datasets considered in the experiments.\n\nI felt that details of the SCM design and subsequent training (e.g. ignoring p(x|z)) were not justified", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bisQ1LgWWf", "reviewer": "Reviewer_nCvr", "summary": "This paper presents a method that combines counterfactual and concept bottleneck that can answer \"What?\", \"How?\" and \"Why not?\" all at once. It leverages latent variable models to generate counterfactuals via variational inference. It achieves comparable classification accuracy to standard CBMs and black-box models and outperforms in generating interpretable, concept-base counterfactuals.", "questions": "Could you provide additional visual examples of counterfactuals generated by your model and baseline methods for each dataset? Visualizing these comparisons would be highly intuitive and beneficial for readers, helping them clearly see the improvements.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The method is straightforward and intuitive, combining two existing approaches, counterfactual reasoning and concept bottleneck modeling with moderate novelty. The experiments provide solid evidence of its effectiveness.\n2. This paper is well organized, and abstract and introduction effectively set the stage for the paper's major content.\n3. The method enhances the interpretability of counterfactuals, making it potentially valuable for many real-life settings like medical applications, as the authors suggest, beneficial for clinical decision-making processes.", "weaknesses": "1. While the paper uses 3 datasets, they are relatively simple. In this paper, authors use examples of medical images (fig 1) but did not actually experiment with any medical dataset. Using more diverse and complex real-life dataset (e.g., CIFAR-100 or MIMIC-CXR) would strengthen the paper and demonstrate broader applicability.\n2. The paper lacks visualizations of counterfactuals. For example, when discussing improvements in counterfactual quality, it would be helpful to show visual comparisons with baselines. Providing real counterfactual comparisons would better substantiate the claims and help readers understand the model's advantages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fjs37SWnMQ", "reviewer": "Reviewer_9mnf", "summary": "The paper introduces a concept bottleneck model with the ability to generate counterfactual explanations. The paper aims to answer three questions: what, how and why not. The what questions are answered by showing the concepts used to make predictions. The how questions are answered by changing the concepts and observing how the prediction changes. Finally, why not questions can be answered by providing an alternative outcome and observing how the concepts change to understand what-if situations. The method also enables what the authors call “task-driven” interventions that allow users to correct wrongly predicted concepts. The models are tested on three datasets: dSprites, MNIST add CUB, and tested on various metrics. In general, based on the experiments, the proposed model demonstrates good performance across multiple tasks and datasets.", "questions": "- Please refer to the weaknesses.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is easy to read, and the figures are well described and informative.\n- Explanations seem to be easy to use.\n- The method is tested against many baselines.", "weaknesses": "- I am concerned regarding the number of hyperparameters that need to be set for the loss function. It is not clear how important they are, and how much tuning is needed. How easy is it to tune these hyperparameters? How do they affect the results? Is it easier to have separate systems rather than one big as the one proposed?\n- The method requires handcrafted concepts, and it might not be easy to come up with these concepts. Further, there might be a lot of labor involved in creating these concepts. Moreover, the predictive performance depends on how well these concepts describe the input features.\n- Figure 1: Is it only for illustration or is this actually the result from trained models on x-ray images?\n- I might have missed it, but I cannot find the citation for the dataset from which the x-ray images come. If I did not miss it, please cite it so it becomes easier for readers to find it. Moreover, are there any restrictions on showing these images, that is, what is the license of these images?\n- I believe the x-ray images to be from the Osteoarthritis Initiative dataset. It is used to motivate the paper, but why is it not used in the experiments? Are there any weaknesses with the method making the dataset unsuitable for experimentation and showcasing results?\n- Line 156: Why does z' need to be conditioned on both c and y? Don't we already know y if we know c?\n- Line 224: How do we know Equation 5 gives us sparser explanations? Is it because the posterior is moved towards the prior distribution? But this is defined in the latent space. How can we know that the concepts c will be sparse even though the posterior of z is close to the prior?\n- How are the models trained? Are certain parts frozen, or are all components trained jointly?\n- Is it possible to show readers the explanations themselves for the datasets used and not only Figure 1 for a dataset not used? We have seen many quantitative results, but part of XAI also involves qualitative qualities. How do these explanations perform qualitatively with end-users? Are they useful to human end-users? And who are the intended end-users for these explanations?\n- Although we do not need to run post-hoc searches, new black-box components are introduced into the model. How does that affect the overall interpretability of the model? I believe that can also have negative side effects and not only positive influences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mnol1TSqWZ", "reviewer": "Reviewer_RbDp", "summary": "This work introduces the framework of counter factual CBMs, aimed specifically to increase the variability and depth of inspection question that can be asked towards a CBM. In other words, CF-CBM aims to increase the level of interpretability and interactability of standard CBM models without losing performance.", "questions": "- 5.1 I find it important to have the results of this question also in the main paper, e.g., average values, as it is an important one. I would additionally suggest the authors change the wording of the conclusion. It seems that on average CF-CBMs are slightly lowerd in predictive performance. While I don’t think this is a drawback of the proposed emthod, I think stating the method is on par is slightly overstated. That CF-CVM is not necessarily on par is an important information for the reader/user. Again stating this won’t decrease the contribution. \n\n- 5.2. I don’t understand the intuition behind why CF-CBMs should be less prone to confounding factors? What mechanism in their learning is beneficial for this? Maybe an ablation study would be intersting to underscore these findings.\n\n- Please provide code to the paper, e.g., an anonymized github link.\n\n- Lastly, this [1] might be an interesting paper to add to the Introduction as it tackles learning concepts (e.g., for CBMs) focussing on learning inspectable concepts, e.g., via counterfactual querying. Of course [1] focuses on counterfactual concepts, whereas this work focuses on counterfactuals for a model decision.\n\n[1] Wolfgang Stammer, Antonia Wüst, David Steinmann, and Kristian Kersting. “Neural Concept Binder.” In: Advances in Neural Information Processing Systems (NeurIPS) (2024)", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is very well written and structured. It is well motivated and the methodological contribution as far as I can tell is clear and important.", "weaknesses": "I just have a few minor remarks which I have added to the questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)