Stochastic Concept Bottleneck Models
Authors: Moritz Vandenhirtz
Sonia Laguna
Ričards Marcinkevičs
Julia E Vogt
Venue: NeurIPS 2024 poster (2024)
Keywords: Concept Bottleneck Models
Interventions
Interpretability
Concepts
Primary area: interpretability_and_explainability
TL;DR: We propose a method for modeling concept dependencies that improves intervention effectiveness.

Abstract: Concept Bottleneck Models (CBMs) have emerged as a promising interpretable method whose final prediction is based on intermediate, human-understandable concepts rather than the raw input. Through time-consuming manual interventions, a user can correct wrongly predicted concept values to enhance the model's downstream performance. We propose *Stochastic Concept Bottleneck Models* (SCBMs), a novel approach that models concept dependencies. In SCBMs, a single-concept intervention affects all correlated concepts, thereby improving intervention effectiveness. Unlike previous approaches that model the concept relations via an autoregressive structure, we introduce an explicit, distributional parameterization that allows SCBMs to retain the CBMs' efficient training and inference procedure. 
Additionally, we leverage the parameterization to derive an effective intervention strategy based on the confidence region. We show empirically on synthetic tabular and natural image datasets that our approach improves intervention effectiveness significantly. Notably, we showcase the versatility and usability of SCBMs by examining a setting with CLIP-inferred concepts, alleviating the need for manual concept annotations.

Stochastic Concept Bottleneck Models
Moritz Vandenhirtz, Sonia Laguna; Ričards Marcinkevičs, Julia E. Vogt Department of Computer Science ETH Zurich Switzerland
Abstract
Concept Bottleneck Models (CBMs) have emerged promising interpretable method whose final prediction is based on intermediate, human lumaan-nndestandaalle concepts rather than the raw input. Through time-consuming manual interventions, user can correct wrongly predicted concept values to enhance the model's down- stream performance. We propose Stochastic Concept Bottleneck Models (SCBMs), novel approach that models concept dependencies. In SCBMs, single-concept intervention affects all correlated concepts, thereby improving intervention ef- fectiveness. Unlike previous approaches that model the concept relations via autoregressive structure, we introduce an explicit, distributional parameterization that allows SCBMs to retain the CBMs' efficient training and inference procedure. Additionally, we leverage the parameterization to derive an effective intervention strategy based the confidence region. We show empirically synthetic tabular and natural image datasets that our approach improves intervention effectiveness significantly. Notably, we showcase the versatility and usability of SCBMs by examining a setting with CLIP-inferred concepts, alleviating the need for manual concept annotations.
1 Introduction
today's world, machine learning plays a crucial role in making important decisions, from healthcare to finance and law. However, as these algorithms become more complex, understanding how they arrive at their decisions becomes increasingly challenging. This lack of interpretability is significant concern, especially in situations where trustworthiness, transparency, and accountability are paramount (Lipton 2016 Doshi- Velez Kim! 2017). Recent studies have focused on Concept Bottleneck Models (CBMs) (Koh et al. |020] Havasi et al.) 2022 Shin et al. 2023), class of models that predict human-unde uamaa-nnndesttndablle concepts upon which the final target prediction is based. CBMs offer interpretability since user can inspect the predicted concept values to understand how the model arrives at its final target prediction. Moreover, if they disagree with a concept prediction, they intervene by adjusting it to the right value, which in affects the target prediction.
For example, consider the yellow warbler in Figure T (a). where a user might notice that the binary concept 'yellow primary color' is mispredicted. Upon this realization, they can intervene on the CBM by setting its value to 1, which increases the probability the class yellow warbler. This way of interacting allows any untrained user to engage with the model to increase its predictive performance.
However, if the user input is that the primary color is yellow, should not the likelihood of yellow crown increase too? This adaptation would increase the predicted likelihood of the correct class even more, as yellow warblers are characterized by their fully yellow body. Currently, vanilla CBMs do not exhibit this behavior as they do not use the intervened-on concepts to update their remaining concept predictions. This indicates that they suboptimally adapt to the additional knowledge gained.
'Equal contribution. Correspondence fmoritz. vandenhirtz, slaguna) ethz. ch
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Concepts Cp.,,
Target
Mollllo Warbler') -032
wing Yellow crowa Yellow peimary
fisellow czowe") 0.39
Mtaliforiia ouil")=u113
d[Yellow peinacy") 00.00
('Ycllow headed Blackeisd") 0.36
Intervention.;
e'Bervvn wing"] -.11
(Telllow Norble=]
eTfelllow crownn1-99
I ea1l")0007
efWeellow prinsry]-1
P'Peellww headed Mlackbi -035
- (MO0, Koo e- (ern)))
x
The Po- - (R. X)
Tels (3)
Overview of the proposed method for the CUB dataset. (a) A user intervenes on the concept of 'primary color: yellow'. Unlike CBMs, our method then uses this information adjust the predicted probability of correlated concepts, thereby affecting the target prediction. (b) Schematic overview of the intervention procedure. user's intervention c's is used infer the logits 7,s of the remaining concepts. matrix for the 112 concepts of CUB (Wah et al. 201T). Characterization of concepts on the left.
Figure 1:
(c) Visualization of the learned global dependency structure as a correlation
To this end, we propose extend the concept predictions with the modeling of their dependencies, as depicted in Figure T
The proposed approach captures the concept dependencies by modeling the concept logits with learnable non-diagonal normal distribution, which enables efficient, scalable computing of the effect of interventions on other concepts. By integrating concept correlations, we reduce the time and effort of having to laboriously intervene on many correlated variables and increase the efficacy of interventions on the downstream prediction. Thanks to the explicit distributional assumptions, the model is trained end-to-end, retaining the training and inference speed of classic CBMs as well as the benefits of training the concept and target predictor jointly. Morcover, show that method excels when querying user interventions based on predicted concept uncertainty (Shin et al.) 2023), further highlighting the practical utility of our approach such policies spare users from manually sifting through the concepts to identify necessary interventions. Lastly, based on the distributional concept parameterization, propose novel approach for computing dependency-awer interventions through the likelihood-based confidence region.
Contributions This work contributes to the line of research on concept bottleneck models in several ways. (i) We propose to capture and model concept dependencies with multivariate normal distribution. (ii) We derive a novel intervention strategy based on the confidence region of the normal distribution that incorporates concept correlations. Using the learned concept dependencies during the intervention procedure allows for stronger interventional effectivenesss. (iii) We provide thorough empirical assessment of the proposed method on synthetic tabular and natural image data. Additionally, we combine our method with concept discovery where we alleviate the need for annotations by using CLIP-inferred concepts. In particular, we show the proposed method (a) discovers meaningful, interpretable patterns the form of concept dependencies, (b) allows for fast, scalable inference, and (c) outperforms related work with respect to intervention effectiveness thanks to the proposed concept modeling and intervention strategy.
Background & Related Work
Concept bottleneck models (Koh et al. 2020: Lampert et al.) 2009 N. Kumar et al. 2009) are typically trained on data points (ær, C, y), comprising the covariates & X. target y E V, and annotated binary concepts e E C. Consider neural network fo parameterized by 0 and a slice (1p) (Leino al. 2018) s.t. i := fo (a) = Swp (he (m)). CBMs enforce concept bottleneck à (ar such that the model's final output depends on the covariates solely through the predicted concepts č.

---

While Koh et al. (2020) propose the soft CBM, where the concept logits parameterize the bottleneck,
Havasi et al. (2022) argue that such a representation leads to leakage, where additional unwanted information in the concept representation is used to predict the target (Margeloiu et al., [02T
Mahinpei et al., (02T). Thus, they parameterize the bottleneck by binarized concept predictions and call it the hard CBM. Then. Havasi et al. (2022) equip the hard CBM with an autoregressive structure of the form ej|2; Cci, which is supposed to learn the concept dependencies. As such, the
implicit autoregressive modeling of concept dependencies by |Havasi et al. (2022) is the most related to the current work. Complementary to our work, Heedemmnn et al. (2023) analyze how a CBM's performance is affected by concept correlations. Unlike approaches that restrict the bottleneck to prevent leakage, Concept Embedding Models (CEM) (Espinosa Zarlenga et al. 2022) represent each concept with embedding vector from which the concept probabilities can be inferred. E. Kim et al. (2023) model the embedding with a normal distribution, assuming diagonal covariance matrix, which prevents them from capturing concept dependencies. Therefore, their intervention performance is not expected to differ from that of CEMs. Recent works explored how a CBM--iie structure can be enforced even without concept-annotated training set. YYuksekgonul et al. (2023) transform pre-trained model into a CBM via a concept bank from concept activation vectors and multimodal models (B. Kim et al., 2018), while Oikarinen et al. (2023) query GPT-3 (Brown et al. 2020) for the concept set C. and assign the values of the concept activations to each datapoint æ with CLIP (Radford
al. 202T) similarities. Similarly, Panousis et al. (2023) uses CLIP to probabilistically discover a sparse set of concepts for each input, which could be used in our model for fully probabilistic pipeline. Lastly, Marciinevvis et al. (2024) instead relax the need for a concept labeled training set to smaller validation set by fine-tuning pre-trained model.
Intervenability (Marcinkevičs et al. 2024 is crucial element of CBMs it allows the user to correct wrongly predicted concepts è to c², which in turn affects the target prediction of the model g'. If multiple concepts are intervened on sequentially, the order of interventions is important. To this end. SSethh et al. (2022) and (Shin et al. (2023) explore multiple policies according to which the order of concepts is determined. Chauhan al. (2023) propose combine predefined policies with learnable weighting parameters, while Espinosa Zarlenga et al. (2024) learn the policy itself. Concurrently, (iinhh al. (2024) learn a realignment module align concept predictions. [Steimmann it al. (023) argue that instance-specific interventions are costly and store previous interventions in a memory to automatically reapply them for similar data points. Lastly, Collins et al. (2023) explore the advantages of including uncertainty rather than treating humans oracles.
Our work models concept dependencies by parameterizing the bottleneck with a distribution. In similar vein, Variational Autoencoders (Kingma & Welling 2014) parameterize the bottleneck with a normal distribution model and generate new data. Stochastic Segmentation Networks (Monteiro) et al. 2020) parameterize the logits of a segmentation map with a non-diagonal normal distribution to capture the spatial correlations of pixels and model the aleatoric uncertainty. The modeling of uncertainty with a distribution is also explored by Bayesian Neural Networks (Neal 1995) that learn a probability distribution over the neurons of a neural network.
3 Methods
We propose Stochastic Concept Bottleneck Model (SCBM). novel concept- based method that relaxes the implicit CBM assumption of independent concepts. SCBM captures the concept dependen- cies by learning their multivariate distribution. As a result, interventions become more effective and scalable, as a single intervention can influence multiple correlated concepts. A schematic overview of the proposed method is depicted in Figure (b).
3.1 Model Formulation
To capture the concept dependencies, we model the concept logits v) with learned multivariate normal distribution. Modeling logits with a normal distribution has proven to be effective in the context of segmentation (Monteiro et al.) 2020). While Monteiro et al. (2020) use it to capture the spatial dependencies of pixels, we, instead, model the relatios between concepts, where the properties of the normal distribution will prove useful. A neural network is trained to predict the distribution's parameters 17 æ ~ N (μ(z)), I(x)), where μxx) E R, and ((:) E RCCC Thus, the traditional
'The code is available tttps:///itthbb htt
3

---

assumption of independent concepts a TL Cj x, ş is relaxed to TL Cj Vi * j. where the assumed normal distribution induces linear concept dependencies. The inductive bias of linearity is useful in practice as it is more robust to overfitting and computationally more scalable with respect C' compared to nonlinear alternative (Havasi et al. 2022). as we will show in Section
To learn the distribution, we minimize the negative log-likelihood
- log p(c x) = -log log p(c n)pp(n x)dn,
(1)
where & are the parameters of a neural network that predicts the distribution "7 x ~ N (u(x)), E())). This integral is intractable due the softmax operation applied in p(c m). Thus, the integral is approximated by M Monte Carlo samples
-log / p(cl m)po(n æ)dn ss - log M
1 Σ plc n (m)),
(m)) x ~ N (m(œ)). E(x)) (2)
In order to learn Φ, we make use of the parameterization as normal distribution and employ the reparameterivation trick f7(m) x = (a) + (()))(m)) L(x) = {xx), e(m) V (0. I)
such that gradients can be computed with respect to the parameters. Lastly, we incorporate the new relaxed conditional independence assumption
C
logp(c 17) log II p(c ma)
= logp(c mp).
where p(c: m)) describes Bernoulli distribution parameterized by the sigmoid- tranoo--eras logits (1)). Combining the above considerations results the following reformulation the negative log-likelihood:
M p(c n (m))
- llopp(c x)) = -log
Σ
M
m=
α log
exp
log p(c; m (m))
(4)
m==1
1=1
(() (m)
log
exp
where BCE stands for Binary Cross Entropy, and the logsumexp trick used for numerical stability.
The distribution-ban based modeling procedure allows for efficient sampling, thus, enabling SCBM to train concept and target predictors jointly, sequentially, or independently. In contrast, the autoregressive alternative (Havasi et al. 2022) requires independent training due to the computational complexity. We adopt joint training scheme to obtain the benefits of end-to-end learning where concept and target predictors can adjust to each other. To prevent leakage, we follow Havasi et al. (2022) and train the model with the hard (0, } concept values as bottleneck rather than the logits used the original CBM (Koh et al. (2200). To this end, we employ the straight-through Gunbel-Softmax trick (Jang et al. 2017] Maddison al,, (017) that approximates Bernoulli samples while being differentiable. The
target predictor Pup then learned by minimizing the negative lig-likelihood
log Dogp(y) xx) = log Σ Pv c)p(c ax)
oEC
= log 1 M (w c(m)),
(5)
(m) ~ p(c æ).
M ===1
Lastly, the learned dependencies regularized by following Occam's 's razor and to prevent overfitting. We take inspiration from the Graphical Lasso (Friedman et al. 2008) and penalize the off-diagonal elements of the precision matrix E-1
By combining concept, target, and precision loss with weighting factors d and A2, we arrive the final loss function
M
log
exp
BCE ((())( + MICE
1 y,
Σ gu (c()))))
+ do [E(T) (6)
M
1=1
m=1

---

3.2 Covariance Learning
The introduced amortized covariance matrix [xx) provides the flexibility tailor its predicted concept dependencies to each data point, making it adaptable to many data-generating mechanisms. For example, the commonly used CUB (Wah et al., 2011; Koh et al. (020), it can learn the class-wise concept structure present in the dataset. The explicit dependency representation inferred by the learned covariance matrix is useful as it provides insights into the learned correlations among the concepts, which is important for understanding and interpreting the model behavior.
However, an amortized covariance matrix comes at the price of not being able visualize and interpret a unified concept structure on dataset level. Depending on the need of the application, such global structure might be preferable. Thus, we propose a variation of SCBM, where the covariance matrix is not amortized (E(x)), but learned globally (E). An example of the global concept structure learned on CUB shown Figure (c). This variation has the inductive bias of assuming a constant covariance matrix, whose utility depends on the underlying data- generating mechanism. We recommend using the more flexible, amortized version by default and only utilizing global covariance if the strong assumption of fixed dependencies is reasonable. We will explore this empirically in more detail in Section 5
3.3 Interventions
distinguishing property of CBM-like methods is the user's capacity correct wrongly predicted concepts, which in turn affects the target prediction (Marcinkevičs et al. 2024). For a big concept set, this intervention procedure can become quite laborious as user has to inspect and manually intervene on each concept separately. SCBMs are designed to alleviate this need by utilizing the learned concept dependencies such that single intervention affects all related concepts as modeled by the multivariate normal distribution.
The parameterization procedure. Given a set S C (1..... eis is computed via their logits 7hs by conditioning on the intervention logits v's, utilizing the known properties of the normal distribution
multivariate normal distribution allows for quick, scalable intervention
......,) C} of concept interventions, the effect on the remaining concepts
mys V, n's N (ū(x), ((x)),E(x))
p M.s + S (T's ps):
Σ = E.sAs S,s.s Esyss-
In standard CBMs, intervention affects only the concepts which the user intervenes. As such, Koh et al. (0200) set v/! to the 5th percentile of the training distribution if c; = and the 95th percentile if a 1. While this strategy is effective for SCBMs too, Appendix C.5 the modeling of the concept dependencies warrants a more thorough analysis of the intervention strategy. We present two desiderata, which our intervention strategy should fulfill.
p(c: m/)) 2 p(c: (4)
The likelihood of the intervened-on concept e; should always increase after the intervention. If SCBMs used the same strategy CBMs, it could happen that the initially predicted μa was more extreme than the selected training percentile. Then, the interventional shift vy 14 in Eq. would point in the wrong direction. This would cause m,s to shift incorrectly.
ii In/ mel should not be "too large".
We posit that the interventional shift should stay within a reasonable range of values. Otherwise, the effect on s7 would unreasonably large such that the predicted pas would be completely disregarded.
To fulfill these desiderata, we take advantage of the explicit distributional representation: the likelihood-baat based confidence region of μ4 provides a natural way specifying the region of possible n's that fulfill our desiderata. Informally, confidence region captures the region of plausible values for a parameter of a distribution. Note that the confidence region takes concept dependencies into account when describing the area of possible n's- To determine the specific point within this region, we search for the values V which maximize the log- likelihood of the known, intervened-on concepts cs. implicitly focusing concepts that the model predicts poorly:
5

---

i's - arg max Cogp(es nss)
ns
s. - 2 plogp(ns uss; Es,s) - log us, Es.s)) $ Xd,1-a
(8)
T/ μi 2 0f = ViE
7/ nr S 0f 0, E
where d= |S]. The first inequality describes the confidence region. it based on the logarithm of the likelihood ratio, which, after multiplying with 22, asymptotically follows a x² distribution (Silvey 1975). The last two inequalities restrict the region to the desired direction. Note that 7/s is computed to determine the conditional effect of the interventions on 7,s using Equation 7 When predicting j under interventions, the logits 77,s are then used for sampling the binary concept values C,s while the intervened-on concepts c's are directly set to their known, binary value.
4 Experimental Setup
Datasets and Evaluation We perform experiments on a variety of datasets to showcase the validity of our method. Inspired by Marcinkevics et al. (2024), we introduce a synthetic tabular dataset with a data-generating mechanism that contains fixed concept dependencies we can regulate. In particular, the concept logits 7) are sampled from a randomly initialized positive definite covariance matrix and generate æ. Binary concept values are inferred from 17 and generate the target We refer to for a more detailed description.
Appendix -..
As a natural image classification benchmark, we evaluate on the Caltech-UCSD Birds- 200-2011 dataset (Wah et al.) 2011), comprised of bird photographs from 200 distinct classes. It includes 112 concepts, such as wing color and beak shape, shared across the same class instances as revised in the original CBM work (Koh et al., (2020). Additionally, we explore another natural image classification task on CIFAR-10 (Krizhevsky et al. (2009) with 10 classes. To mitigate the concept annotations requirement, the concepts are synthetically acquired in similar fashion the concept discovery literature. We adopt the 143 concept classes generated via GPT-3 (Brown et al. 2020) in prior work OOiarinnnn al., 2023). To obtain the binary concept values, we use the CLIP model (Radford et al. 2021) to compute the similarity between each instance of an image with the text embedding of the similarity of its negative counterpart, i.e. not the concept. Appendix A.2 contains further details about the natural image datasets.
specific concept and compare
To compare methods, we evaluate the model performance based the concept and target accuracy. We compute test performance before and after intervening on an increasing number of concepts. The order of concepts in the intervention determined by an uncertainty-baat that selects the concept whose predicted probability is closest to 0.5. We also show results for random policy in Appendix C.33 Additionally, we evaluate the calibration of the predicted concept uncertainties that are being used for the uncertainty-basee policy, with the Brier score (Brier 1950)
policy (Shin al., (203)
and the Expected Calibration Error (Naeini et al. 2015) A. Kumar al. 2019).
Baselines We evaluate the performance of our method in comparison with state-of-the-ar models. Namely, we focus on the vanilla concept bottleneck model (CBM) by Koh al. (2020) in its hard version (Havasi et al. 2022), trained jointly using the straight-through Gumbel-Softman trick (Jang al. 2017| Maddison et al. 2017), as a sensical baseline our binary modeling of concepts. Additionally, we explore the concept embedding model (CEM) by Espinosa Zarlenga et al. (2022) that learns two concept embeddings, दे and ह These representations are used to predict the final concept probability with a learnable scoring function P s(ē) è,
o(W, e, + b.) and are then combined into final concept embedding é (p,c + (1 p,)c,) ) that is passed to the target predictor. Interventions are modeled by altering the concept probabilitiess ft. Note that Espinosa Zarlenga et al. (2022) optimize for intervention performance during training, which we omit, to ensure a fair comparison where no method was explicitly trained for intervention performance. Finally, we evaluate the autoregressive CBM structure proposed by Havasi et al. (2022), where concept dependencies are learned with an autoregressive structure. Here, each concept is predicted with a separate MI.P that takes as input latent representation of the input fo(a)) and all previous concepts C12 ..., G-1- To obtain good initialization of the autoregressive structure, it is pretrained

---

Table 1: ses concept and target accuracy (%) prior to interventions. Results are reported as averages and standard deviations of model performance across ten seeds. For each dataset and metric, the best-performing method bolded and the runner-up is underlined.
Dataset
Method
Concept Accuracy
Target Accuracy
Hard CBM CEM
61.42 = 0.07
58.38 0.39
61,42 = 0.12
58,01 + 0.49
Synthetic
Autoregressive CBM
62.17 = 0.11
59.60 0.62
Global SCBM
61.57 = 0.05
58.39 + 0.53
Amortized SCBM
62.41 = 0.20
58.96 + 0.38
Hard CBM
94.97 + 0.07
67.72 + 0.57
CEM
95.12 0.07
69.60 0.30
CUB
Autoregressive CBM
95.33 = 0.07
69.24 + 0.44
Global SCBM Amortized SCBM
94.99 0.09
68.19 0.63
95.22 = 0.09
69.87 + 0.56
Hard CBM
85.51 = 0.04
69.73 + 0.29
CEM
85.12 0.14
72.24 + 0.33
CIFAR-10
Autoregressive CBM
85.31 = 0.06
68,88 + 0.47
Global SCBM
85.86 = 0.04
70.74 + 0.29
Amortized SCBM
86.00 0.03
71.66 + 0.25
for 50 epochs. As the Monte Carlo sampling from the autoregressive structure is time-consuming, the target predictor Pe is trained independently using the ground-truth concepts as input. At intervention time, normalized importance sampling algorithm is used to estimate the concept distribution.
Implementation Details The model architectures comprise a backbone for concept prediction followed by linear layer as head for an interpretable target prediction. More details can be found in Appendix To ensure the positive definiteness of the concept covariance matrix Σ, we parameterize it via its Cholesky decomposition Σ LL Thus, directly predict the lower triangular Cholesky matrix L. We will evaluate two options for SCBMs: using a global (Σ) or an amortized covariance set the weighting terms Xı and X2 of Equation 6 1. For the global version, we initialize it with the estimated empirical covariance matrix and set X2 = O, we did not observe big differences when varying Ae. In Appendix C.4 we provide ablation study, demonstrating that SCBMs are not very sensitive to the choice of X2 At intervention time, we solve the optimization problem based the 99%-confidence region with the SL.QP algorithm (Kraft) 1988). In Appendix C.6, we provide an ablation with different confidence levels.
matrix (E(x)). For the amortized version,
5 Results
Test performance In T we report the re- sults of the concept and target accuracy prior to interventions. Overall, SCBM performs on par with the baseline methods, with no clear outper- forming or underperforming technique through- the datasets. In Appendix C.7, show that other metrics lead to the same interpreta- tion. This shows that the additional overhead of learning the concept dependencies does not neg- atively affect the predictive performance. We note that the amortized covariance variant con- sistently surpasses the globally learned matrix due to its ability adjust the predicted concept dependency structure and uncertainty an instance level. On the other hand, the global variant offers a unified understanding of the concept correlations, an example of which is presented in Figure (c). Notably, in CIFAR-10, even though the concept performance of CEM is the worst of all methods, it has the best target performance. This might suggest the presence of leakage in CEM's embeddings, in CIFAR-10 10, the concept set alone is not sufficient to predict the target, and learning
Table 2: Relative time it takes for one epoch in the CUB dataset when training on the training set, or evaluating on the test set, respectively.
Method
Training
Inference
Hard CBM
5x
CEM
5x
lx
Autoregressive CBM
5x
15x
Global SCBM
5x
1x
Amortized SCBM
5x
1x
7

---

additional information might be useful. In Table we show the time it takes for training and testing of the methods. It is evident that the autoregressive CBM Havasi al. (2022) suffers from a slow sampling process due to its autoregressive structure, while SCBMs retain the efficiency of CBMs.
100
100
100
Concept accuracy (S)
99
90
80
97
90-
70
60
95
85
Number Concepts Intervened
Number Concapts intervened
Number of Concepts Intervened
80-
85
accuracy
75-
70
80
65
60-
70
70-
20
40
60
80
100
25
50
75
100
0
100 Number of Concepts Intervened
50
Number Concepts Intervened
Number Concepts intervened
(a) Synthetic
(b) CUB
(c) CIFAR-10
Hard CBM
CEM
Autoregressive CBM
Global SCBM
Amortized SCBM
Figure 2: Performance after intervening concepts in the order of highest predicted uncertainty. Concept and target accuracy (%) are shown in the first and second rows, respectively. Results are reported as averages and standard deviations of model performance across ten seeds.
Interventions In this paragraph, we analyze the intervention performance of SCBMs and their base- line models, focusing on their effectiveness in modeling concept dependencies and improving target accuracy. Figure[[show the intervention curves across ten seeds, where the performance measured based on the concept and target accuracy. The order of concepts intervene is determined by uncertainty-bac -bssd policy that makes use of the predicted probabilities. In Appendix C.3] we present the intervention performance if concepts were selected randomly. The intervention curves in the first row show that SCBMs are superior in modeling the concept dependencies, as evidenced by their significantly steeper intervention curves compared to the baseline methods. Furthermore, the second row of Figure 2 indicates that the strong concept modeling translates to a significant improvement in downstream performance, partly thanks the intervention strategy introduced in Section 3.3 We note that especially for the most practical scenario of only small number of interventions, SCBMs outperform their counterparts. Comparing the SCBM variants, the natural image datasets show an overall better intervention performance with the amortized covariance matrix, following the trend of Table D as it can capture the instance-wise correlation structure of the data. Only the synthetic dataset, where the data-generating covariance matrix is fixed, does the global SCBM slightly outper- form the amortized one. Thus, we advocate for the usage of the global variant only if the underlying assumption of fixed covariance is reasonable. Lastly, the success of SCBMs on CIFAR- 10, with CLIP-based based concepts, shows our proposed method can work without human-annotatee annotated concepts. To strengthen this point and also showcase the scalability of our method, in Appendix C we provide results on CIFAR- 100 with 892 concepts, where our SCBMs also strongly outperform baselines.
Analyzing the performance of the autoregressive CBM, which also captures concept dependencies, we observe that they expectedly have a better intervention performance than the hard vanilla CBM, which does not take correlations into account. However, it becomes evident that, compared to the concept performance of SCBMs, their autoregressive structure does not capture the dependencies to the full extent. This shows in the target accuracy, where they only match or outperform SCBMs towards the full of intervened concepts. We attribute the better performance on the full intervention set the independent training procedure utilized by autoregressive CBMs, which comes at the cost of lower test performance CIFAR-I0. 10. Arguably, in realistic use-case, such high number of instance-level interventions is not sensible, and if it were, SCBMs could also be trained independently. Finally, the CEM shows reduced intervention performance as the expressive concept embeddings, which are prone to information leakage, seem to suboptimally adapt to the injected concept information.
8

---

Table 3: Test-set calibration (%) of concept predictions. Re- sults are reported as averages and standard deviations of model performance across ten seeds. For each dataset and metric, the best-performing method bolded and the runner-up is underlined. Lower better.
Figure 3: Intervention perfor- mance of SCBMs measured in concept and target accuracy (%) on CUB for random and uncertainty thaaiitt---beedd policy.
Dataset
Method
Brier
ECE
100
Concept scauray 183)
99
Hard CBM
28.79 + 0.09
22.38
+ 0.15
98
CEM
29.32 + 0.08
23.55
+ 0.09
Synthetic
Autoregressive CBM
24.84 + 0.32
13.54
0.49
97
Global SCBM
27.73 = 0.09
20.10 0.14
96
Amortized SCBM
25.58 = 0.20
15.57 = 0.55
25
75
100
Hard CBM
3,93 + 0.05
2.44 + 0.06
Number Concapts Intervened
CEM
4.04
3.25+0.0
100
Targes Target accuracy (8)
CUB
Autoregressive CBM
3.75 0.05
2.73
Global SCBM
3.87
2.33
Amortized SCBM
3.64
0.08
Hard CBM
10.42 + 0.05
4.93
CEM
11.06 0.16
7
70
CIFAR-10
Autoregressive CBM
10.70 + 0.05
6.07 0.10
Number Concapts Intervened
25
Global SCBM
9.95 1 0.02
2.88 1 0.11
Giobal
Amortized SCBM
9.84 1 0.02
2.22 1 0.12
Dundan)
(Uncetainey)
Modeling the concept distribution A cornerstone of SCBMs is the explicit, distributional parame- terization of concepts. This helps in understanding the data correlations and allows for visualization, the example seen in Figure (c). The explicit probabilistic modeling results in improved concept uncertainty estimates compared to the baseline CBM counterparts, as shown in Table 3 where lower metrics imply better estimates. This proves useful for interventions, where the uncertainty estimates can be leveraged for the choice concepts to intervene on, improving the target prediction more effectively and reducing the need for manual user inspection. In Figure 3 compare the performance randomly intervening versus intervening based on the predicted uncertainty. We observe that there big gap between the two policies, indicating the usefulness of the estimated probabilities. Nevertheless, note that intervening at random remains successful and supports the observations made in the previous paragraph, shown in Appendix C.3
Conclusion
In this paper, we introduced SCBMs, new concept-based method that models concept dependencies with a multivariate normal distribution. We proposed novel, effective intervention strategy that takes concept correlations into account and is based on the confidence region inferred from the distributional parameterization. We showed that our modeling approach retains CBMs' training and inference speed, thus, being able to harness the benefits of end-to-end concept and target training. Additionally, the explicit parameterization offers the user a clearer understanding of the learned concept dependencies, providing deeper insights into how predictions and interventions made. Empirically, we demonstrated that by modeling the concept dependencies, SCBMs offer substantial improvement in intervention effectiveness, in concept as well target accuracy, compared to related work. We showed that our method excels when iteratively intervening on the most uncertain concept predictions, sparing users from having to manually search through the concept set to identify necessary interventions. Additionally, our results indicate that learning the concept correlations does not decrease performance prior to interventions, in many the baselines. Finally, the versatility SCBMs highlighted through their superior performance on CIFAR-10 10 and CIFAR-100, where concept values CLIP-based rather than human-ann annootaedd..
improving the performance

---

Limitations & Future Work This work opens multiple new research avenues. A natural extension go beyond binary concepts, such as continuous domains with their corresponding adaptations of modeling the concept distribution. Additionally, addressing the quadratic memory complexity of the covariance matrix essential for scaling to larger concept sets. Our proposed intervention strategy accounts for model uncertainty, but further research is needed to accommodate user uncertainty, as human interventions are not always the ground truth. This work allows the editing the learned dependency structure by adjusting the entries of the predicted covariance matrix, which could be explored. Lastly, to model additional information and reduce leakage, Koh et al. (2020); [Havasi et al. (2022) propose the adoption of a side channel. The complementary effectiveness of incorporating the side channel the covariance structure could be explored in the context SCBMs.
Acknowledgments and Disclosure of Funding
We thank Alexander Marx for the insightful discussions. MV and SL supported by the Swiss State Secretariat for Education, Research, and Innovation (SERI) under contract number MB22.00047. RM is supported by the SNSF grant #320038 0880038189096..
10

---

References
Ansel, J., Yang, E., He, H., Gimelshein, N., Jain, A., Voznesensky, M.,... others (2024). Pytorch Faster machine learning through dynamic python bytecode transformation and graph compilation. In Proceedings of the 29th acm international conference on architectural support for programming languages and operating systems, volume 2 (pp. 929-947). [Referenced on page 14]
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1), 1-3. Retrieved from https:
10.
VOFEIT>2.00 O. 20;;2 [Referenced on pagelo
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P..... others (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901.
[Referenced on page 3
Chauhan, K., Tiwari, R., Freyberg, J., Shenoy, P., & Dvijotham, K. (2023). Interactive concept bottleneck models. In Proceedings of the uaai conference on artificial intelligence (Vol. 37, pp. 5948-5955). [Referenced on page 3
Collins, K. M., Barker, M., Zarlenga, M. E., Raman, N., Bhatt, U., Jamnik, M.,.. ... Dvijotham, K. (2023). Human uncertainty concept-based AI systems. In F. Rossi, Das, J. Davis, K. Firth- Butterfield, & A. John (Eds.), Proceedings of the 2023 AAAT/ACM conference on ai, ethics, and sociery, AIES 2023, montréal, qc, canada, august 8-10, 2023 (pp. 869-889) ACM. [Referenced on page 3
Doshi-Velezz F., & Kim, B. (2017, March). Towards A Rigorous Science of Interpretable Machine Learning (No. arXiv:1702.0866 arXiv. doi: 1485500/rXii.1702.000660 [Referenced on page
Espinosa Zarlenga, M., Barbiero, P., Ciravegna, G., Marra, G., Giannini, F., Diligenti, M.,.. others (2022). Concept embedding models: Beyond the accuracy-epllainabiity trade-off. In Advances in neural information processing systems (Vol. 35, pp. 21400-21413). [Referenced on page 6
Espinosa Zarlenga, M., Collins, K., Dvijotham, K., Weller, A., Shams, Z., & Jamnik, M. (2024). Learning to receive help: Intervention-aware concept embedding models. Advances in Neural Information Processing Systems, 36. [Referenced on page 3
Friedman, J., Hastie, T., & Tibshirani. R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatisties, 9(3,, 432 441. [Referenced page
Havasi, M., Parbhoo, S., & Doshi-Velez, F. (2022), Addressing leakage in concept bottleneck models. In H. Oh, A. Agarwal, D. Belgrave, & K. Cho (Eds.), Advances neural information processing systems. Retrieved from httpp:://ppenrviieeee wttp:/ntno//oootttteeeoeroo
[Referenced on fn9
page 8 10
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the ieee conference computer vision and pattern recognition (pp. 770-778).
[Referenced on page [4]
Heidemann, L., Monnet, M., Roscher, K. (2023). Concept correlation and its effects on concept-
based models. In Proceedings of the ieee/cvf winter conference on applications of computer vision [Referenced on page
(pp.
Jaccard, P. (1901). Étude comparative de la distribution florale dans une portion des alpes et des jura. Bull Soc Vaudoise Sei Nat, 37, 547-579. [Referenced on page 17]
Jang, E., Gu, S., & Poole, B. (2017). Categorical reparameterization with gumbel-softmax. In 5th international conference on learning representationss ICLR 2017, toulon, france, april 24-26, 2017, conference track proceedings. OpenReview.net. Retrieved from
net/
Korumiid-rxE3y85ee [Referenced on page 4 6
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F,, & Sayres, R. (2018). Interpretabil- ity beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In
J. Dy A. Krause (Eds.), Proceedings of the 35th international conference on machine learning (Vol. 80, pp. (668-2677), PMLR. Retrieved from httts: https://proceedings.ntt mlr wtt.:mmer....eeeeeeeeeeeeeeeeeee /
kim18d. html [Referenced on page 3
Kim, F., Jung, D., Park, S., Kim, S., Yoon, S. (2023). Prohabilistic concept bottleneck models. In
A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, & J. Scarlett (Eds.), Proceedings of the

---

Marcinkevičs, R., Reis Wolfertstetter, P., Klimiene, U., Chin-Cheong. K., Paschke, A., Zerres, J., Vogt, J. E. (2024). Interpretable and intervenable ultrasonography- machine learning models for pediatric appendicitis. Medical Image Analysis, 91, 103042. Retrieved from h/t [Referenced on 40th international conference on machine learning (Vol. 202, pp. 16521-16540). PMLR. Retrieved from htmpttts://prrccdddngg. mlr. press//202/kim22gg html [Referenced on page
Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Y. Bengio & Y. LeCun (Eds.), 3rd international conference on learning representations, ICLR 2015, san diego. ca, usa, may 7-9, 2015, conference track proceedings. Retrieved from http://arxiv org/ org/abs/ abs/
1412.6980 [Referenced on page [4]
Kingma, D. P.& & Welling, M. (2014). Auto-encoding variational bayes. In Y. Bengio & Y. LeCun (Eds.), 2nd international conference learning representations. ICLR 2014, banff. ab, canada, april 14-16, 2014, conference track proceedings. Retrieved from http: // //xxiv org/ abs/
1312. 6114
[Referenced on page 3]
Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., & Liang, P. (2020).
Concept bottleneck models. In H. D. III & A. Singh (Eds.). Proceedings of the 37th international conference on machine learning (Vol. 119, pp. (338).348) Virtual: PMLR. Retrieved from [Referenced on page
https: htttps://proceednngs.o mlr www.ppressvviii prress//111//oh20aaa html
10
14
17
Kraft, D. (1988). A software package for sequential quadratic programming. Forschungsbericht-
Deutsche Forschungs- und Versuchsanstalt fur Luft- und Raunfahrt. [Referenced on page
Krizhevsky, A., Hinton, G., et al. (2009). Learning multiple layers of features from tiny images.
[Referenced on page 6
Kumar, A., Liang. P. S., & Ma, (2019). Verified uncertainty calibration. Advances in Neural Information Processing Systems, 32. [Referenced on 6
Kumar, N., Berg, A. C., Belhumeur, P.N., & Nayar, S. K. (2009). Attribute and simile classifiers for face verification. In 2009 ieee 12th international conference on computer vision (pp. 365- 372). Kyoto, Japan: IEEE. Retrieved from https: [Referenced on page 2
http///CCV.......... 5459250
Lampert, C. H., Nickisch, H., & Harmeling, S. (2009). Learning to detect unseen object classes by between-class attribute transfer. In 2009 IEEE conference on computer vision and pattern recognition. Miami, FL, USA: IEEE. Retrieved from tromhhtps::. https:://ot.ooooo wtt.:////...../////........111.t /wtp//CVR.20 2009
.5206594 [Referenced on page 2
Leino, K., Sen, S., Datta, A., Fredrikson, M., & Li, L. (2018). Influence-directed explanations for deep convolutional networks. 2018 IEEE international test conference (ITC). IEEE. Retrieved
hhom/ttps:://aat http///ooooo///////// 1109/test, 2018. 8624792 [Referenced on page 2
Lipton, Z. C. (2016, June). The Mythos of Model Interpretability. Communications of the ACM, [Referenced on page
61(10), 35-43. doi: 10.48550/arxiv.. wttp:/aaxxxx...6....0 1606.03490
Maddison, C. Mnih, A., & Teh, Y. W. (2017). The concrete distribution: A continuous relaxation of discrete random variables. In 5th international conference on learning representations, ICLR 2017, toulon, france, april 24-26, 2017, conference track proceedings. OpenReview.net. net. Retrieved from /tttps://openrevlit
[Referenced on page
Mahinpei, A., Clark, J., Lage, I., Doshi- F., & Pan, W. (2021). Promises and pitfalls of blo concept learning models. Retrieved from https: htt///oo..oo//00// 38550//aXivv... 2106. 13314 (arXiv:2106.1))) 13314) [Referenced on page
Marcinkevičs, R., Laguna, S., Vandenhirtz, M., & Vogt, E. (2024). Beyond concept bottleneck models: How to make black boxes intervenable? In Advances in neural information processing systems (Vol. 37). [Referenced on page 3
www. www.scieenceiirect.cc com/ scienceiee wwtpeiinneeeeeeteeeeeeeeeeeeeeee
page 6
Margeloiu, A., Ashman, M., Bhatt, U., Chen, Y., Jamnik, M., & Weller, A. (2021). Do concept bottleneck models learn as intended? Retrieved from https://doi. http::////o..o//1.... org/10.48500/ arXiv .2105. .04289 (40X0v:2005.0228)) [Referenced on page 3
12

---

Monteiro, M., Le Folgoc, L.. Coelho de Castro, D., Pawlowski, N., Marques, B., Kamnitsas, K.. Glocker, B. (2020). Stochastic segmentation networks: Modelling spatially correlated alcatoric uncertainty. In Advances in neural information processing systems (Vol. 33, pp. 12756-12767).
[Referenced on 3
Naeini, M. P., Cooper, G., & Hauskrecht, M. (2015). Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the aaai conference on artificial intelligence (Vol. 29).
[Referenced on page 6
Neal, R. M. (1995). Bayesian learning for neural networks (Doctoral dissertation, Univer- sity of Toronto, Canada). Retrieved homm:/cp:://bbraayyerd......b .utooonto.ooo.. cariontttccao.. www.o//k/KOUUTOOOOOOOOO INB__NNSTSASS
[Referenced page
Oikarinen, T., Das, S., Nguyen, L. M., & Weng, T.-W. (2023). Label-free concept bottleneck models. In The 11th international conference learning representations. Retrieved from wttp///eeeeeeeeeee///////ee/eeet /
:
htth/forummmdd dii.mmiidddd1CAAAA [Referenced on page 3
15
Panousis, lenco, D., & Marcos, (2023). Sparse linear concept discovery models. Proceed- ings of the ieeelcvf international conference computer vision (pp. 2767-2771). [Referenced on page
Panousis, K. Ienco, D., & Marcos, D. (2024). Coarse-to-fine concept bottleneck models. In Neurips 2024-38th annual conference on neural information processing systems. [Referenced on page 17
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S.,... others (2021). Learning transferable visual models from natural language supervision. In International conference on machine learning (pp. 8748-8763), [Referenced page 6
Sheth, I., Rahman, A. A., Sevyeri. L. R., Havaei, M., & Kahou, S. E. (2022). Learning from uncertain concepts via test time interventions. Workshop on trustworthy and socially responsible machine learning, neurips 2022, Retrieved from httmm/:t////oereeeeeeeee http:://ppaaaeeeeee..eeeooo//a//
[Referenced on page 3
Shin. S., Jo. Y., Ahn, S., & Lee, N. (2023). A closer look the intervention procedure of concept bottleneck models. In A. Krause, E. Brunskill, K. Cho. B. Engelhardt, S. Sabato, & J. Scarlett (Eds.), Proceedings of the 40th international conference on machine learning (Vol. 202, pp. 31504- 31520). PMLR. Retrieved from https://proceediie [Referenced on page
mlr, wwwesspes////. peress/v202// v202/ shin/hhinnaa..m html
T6
Silvey, S. (1975). Statistical inference. Taylor & Francis. Retrieved from hattt:tttt//okkee google
[Referenced on 6
Singhi, N., Kim, J. M., Roth, K., & Akata, Z. (2024). Improving intervention efficacy via concept
realignment in concept bottleneck models. arXiv preprint arXiv:2405.01531. [Referenced on
page
Steinmann, D., Stammer, W., Friedrich, F., & Kersting, K. (2023). Learning intervene
on concept bottlenecks. Retrieved from https://doi. org/10 005.44555/ax11 2308 13453 (arXiv:2308. 13453) [Referenced on page
Wah, C., Branson, S., Welinder, P., Perona, Belongie, S, (2011). The caltech-ucsd birds-200-
2011 dataset. [Referenced on page
Yuksekgonul, M., Wang, M., & Zou, J. (2023). Post-hoc concept bottleneck models. The 1/th international conference learning representations. Retrieved from https://openrevier: net/fooummid htw.:oruummrddddAAAA2EEEEEEYY [Referenced on p3ge
13

---

A Dataset Details
In this section, we provide additional details on the datasets that are being used in the experiments.
A.1 Synthetic Data-Gengrating Mechanism
Here, we describe the data- generating mechanism of the synthetic dataset in more detail. Let N, and C denote the number of independent data points {(r.. Ca. yr. } === covariates, and concepts, respectively. We set = 50,000, = 1,500, and C = 100, with 60%-20%- 20% train-validation- test split. The generative process follows:
1. Randomly sample W E RCX s.t. way ~ (O for sig and 1 S 10.
2. Generate positive definite matrix E E BCx s.t. E WWT + D. Let D E RCXC s.t.
D = 8I, where 5; ~ M1.11 for SiSC.
3. Randomly sample logits H NxC s.t. Th (O Σ) for Sn
4. Let Cn,i (0) for 1 M S N and sis
5. Let h RC + R be randomly initialised multilayer perceptron with ReLU nonlinearities.
6. Let æn = h (n,) + en S.I. En ~ I) for S n S N.
7. Let g: RC + R be randomly initialized linear perceptron.
8. Let Yn = 1 (2mma))))mma)) for S n I N, where Umed denotes the median of (c,))
A.2 Natural Image Datasets
Caltech-UCSD Birds-200-2011 We evaluate on the Caltech-UCST Birds-200-2011 (CUB) dataset (Wah et al. 201T). It comprises 11,788 photographs from 200 distinct bird species annotated with 312 concepts, such belly color and pattern. In this manuscript, split and revised the proposed dataset in the initial CBM work (Koh et al. 2020). Here, only the 112 most widespread binary attributes are included in the final dataset, and concepts are shared across samples in identical classes. The images were resized to a resolution of 224 x 224 pixels. Finally, following the original proposed augmentations, we applied random horizontal flips, modified the brightness and saturation, and applied normalization during training.
follow the original train-test
CIFAR-10 CIFAR- 10 Krizhevsky et al. 2009) is natural image benchmark with 60,000 32x32 colour images and 10 classes. We kept the original train-test split, with 50,000 samples in the train set and a balanced total of 6,000 images per class. We generated 143 concept labels as described in Section 4 using large language and vision models. At training time, as for CUB, we applied augmentations including modifications to brightness and saturation, random horizontal flips and normalisation. Images were rescaled to size of 224 x 224 pixels.
B Implementation Details
This section provides further implementation details of SCBM and the evaluated baselines. All methods were implemented using PyTorch (v 2.1.1) (Ansel et al. 2024). All models are trained for 150 epochs for the synthetic and 300 epochs for the natural image datasets with the Adam optimizer (Kingma & Ba 2015) with a learning rate of 10-4 and a batch size of 64. For the independently trained autoregressive model, split the training epochs into 2/3 for the concept predictor and 1/3 for the target predictor. For the methods requiring sampling, the number of Monte Carlo samples is set to M 100. We provide ablation for M 10 in Appendix C.2 Note that since the predictor head is very simple, the MC sampling of SCBMs is extremely fast and does not influence computational complexity by more than 0.1%. For the synthetic tabular data, connected neural network as backbone, with 3 non-linear layers, batch normalization, and dropout. For the CUB dataset,
a fully
pretrained ResNet- (He et al. 2016), and for the lower-nesolution
httpa://ww/ http:///wwwvviiionoo.o http://www..viiioo..caaalleeee.a https://www. cs. toronto. www.oto..eoooeoeeeoeo cadduu/kriz/cifaar html no license available
200_2011/ no license available
14

---

CIFAR-10 10 a simple convolutional neural network with 2 convolutional layers followed by ReLU, Dropout, and fully connected layer. For fairness in the comparisons, all baselines have the same model architecture choices and all experiments are performed over 10 random seeds.
Resource Usage For the experiments of the main paper, we used a cluster of mostly GeForce RTX 2080s with 2 CPU workers. Over all methods, we estimate an average runtime of 8h per experiment, each running on single GPU. This amounts to 5 methods 3 datasets 10 seeds x 8 hours 1200 hours. Adding to that, the Ablation Figures required another 40 runs, amounting to full total of 1520 hours of compute. Please note that we only report the numbers generate the final results but not the development time, which we roughly estimate to be around 10 times bigger.
C Further Experiments
In this section, we show additional experiments provide more in-depth understanding SCBM's effectiveness. We ablate multiple hyperparameters to provide an understanding of how they influence the model performance, as well as show the performance of our model in other settings.
94
100
Concopt accuracy (96)
92
Concept accuracy
90
98
88
97
86
95
84
0
20
40
60
80
100
25
50
75
100
Number Concepts Intervened
Number of Concepts Intervened
55
100
Target accuracy (96)
50
Target accuracy
90
45
40
35
70
20
40
60
80
100
25
50
75
100
Number Concepts
Number of Concepts Intervened
Autoregressive CBM
Global: 10
Global: 100
Global SCBM
Amortized SCBM
Amortized: 10
Amortized: 100
Figure 4: Performance after intervening
Figure Intervention performance in the or- der of highest predicted uncertainty in CUB, Concept and target accuracy (%) are shown the first and second rows, respectively. Results are reported as averages and standard devia- tions of model performance across 3 seeds.
concepts in the order of highest predicted un- certainty in CIFAR-100 with 892 concepts. Concept and target accuracy (%) are shown in the first and second rows, respectively. Re- sults are reported as averages and standard deviations of model performance across seeds.
C.1 Intervention Performance CIFAR-100
We present the result on the CIFAR-100 dataset with 892 concepts obtained from Oikarinen al.) (2023) in Figure 4 to showcase the scalability of SCBMs. The results underline the efficiency of our method. Notably, the Autoregressive baseline has a negative dip, which likely due to independently trained target predictor not being aligned with the concept predictors in this noisy CL.P-annotated scenario. Note that they need to train independently to avoid the sequential MC sampling during training, which would otherwise increase training time significantly. Our jointly
15

---

trained SCBMs do not have this issue and surpass the baselines. We use the same configuration as for CIFAR-10. 10, with the exception that we set M 10 reduce the memory requirement.
C.2 Number of Monte Carlo Samples
To showcase that SCBMs do not rely huge number of Monte Carlo samples, provide ablation of M Figure S It shows that even for M = 10, SCBMs thrive. Note, however, that since M is not driving factor of SCBMs computational cost, one can leave it at high number.
C.3 Random Intervention Policy
100
100
99
Concapt arcuracy
95
80
97
70
96
60
85
100
Concepts
Concepts
Number Concepts Intervened
80
accuracc
75
90
80
7.0
80
65
70
55
20
40
60
80
100
25
50
75
100
0
100 Number of Concepts Intervened
50
Number of Concepts Intervened
Number of Concepts intervened
(a) Synthetic
(b) CUB
(c) CIFAR-10
Hard CBM
CEM
Autoregressive CBM
Global SCBM
Amortized SCBM
Figure 6: Performance after intervening on concepts in random order. Concept and target accuracy (%) are shown in the first and second rows, respectively. Results are reported averages and standard deviations of model performance across ten seeds.
In Figure 6, we present the intervention performance of SCBM and baseline methods. Compared the mucertainty-based intervention policy of Figure 2 the intervention curves of all methods less steep, confirming the usefulness of Shin et al. (2023) 's proposed policy. Following the previous statements, SCBMs still outperform baseline methods with the amortized beating the global variant for real- world datasets. We observe that in CIFAR-10 for the first interventions, an improvement in concept accuracy not directly reflected improved target prediction for SCBMs, which is likely due to the low signal-to-noiss to-noise ratio of the CLIP-inferred concepts.
C.4 Regularization Strength
In Figure we analyze the impact of the strength of A2 from Equation Due to environmental considerations, we conducted experiments using only 5 seeds and limited the number of interventions to 20. Our findings indicate that SCBMs are not sensitive to the choice of X2, except that unregularized amortized variant exhibits slight patterns of overfitting.
16

---

.9
Targer accuracy (h)
Concept accuracy
86
85
70
10
15
20
Number Cancepts Intervened
Number of Concapts intervened
Global: 0
Global: 0.01
Global:
Global:
Amortized:
Amortized: 0.01
Amortized:
Amortized: 10
Figure 7: Performance on CUB after intervening on concepts in the order of highest predicted uncertainty with differing regularization strengths. Concept and target accuracy (%) shown in the first and second columns, respectively. Results are reported as averages and standard deviations of model performance across five seeds. For each SCBM variant, choose darker color, the higher the regularization strength of A2.
C.5 Intervention Strategy
In Figure 8 we analyze the effect of the intervention strategy. Our findings indicate that while SCBMs are still effective with the proposed strategy from Koh et al. (2020), that sets the logits the 5th (if = 0) or 95th (if = 1) percentile of the training distribution, our proposed strategy based on the confidence region results stronger intervenability.
100
100
Target accuracy (95)
90
98
97
Concept
96
Number Concepts Interveneed
Number of Concepts Intervered
Global SCBM (Empirical Percentile)
Global SCBM (Confidence Region)
Amortized (Empirical Percentile)
Amortized SCBM (Confidence Region)
Figure 8: Performance CUB after intervening concepts in the order of highest predicted uncertainty, comparing the proposed intervention strategy to Koh et al. (2020)'s intervention of setting the logits to the 5th 95th empirical percentile of the training distribution. Concept and target accuracy (%) are shown the first and second columns, respectively. Results are reported as averages and standard deviations of model performance across five seeds.
C.6 Confidence Region Level
In Figure 9 analyze the effect of the level of the likelihood- based confidence region. Our findings indicate that the SCBMs are not sensitive to the choice a, with higher levels being slightly better in performance.
C.7 Jaccard Index
Panousis et (2024) propose to interpret the interpretation capacity of concepts with the Jaceard Index (Jaccard 1901). As such, in Table 4 extend Table I with
this metric. It is evident that the interpretation does not change, indicating that the performance robust to the choice of evaluation metric.
17

---

Target accuracy (N6)
98
Concapt
80
96
70
95
75
100
25
50
100
Number Intervened
Number of Concopts Intervened
Global: 0.8
Global: 0.9
Global: 0.95
Global: 0.99
Amortized: 0.8
Amortized: 0.9
Amortized: 0.95
Amortized: 0.99
Figure 9: Performance on CUB after intervening on concepts in the order of highest predicted uncertainty with differing levels of the confidence region. Concept and target accuracy (%) shown in the first and second columns, respectively. Results are reported as averages and standard deviations of model performance across three seeds.
Table 4: Test-set performance before interventions. Results are averaged across ten seeds.
Dataset
Method
Concept Accuracy
Concept Jaccard
Target Accuracy
Hard CBM
61.42 + 0.07
43,80 + 1.32
58.38 t 0.39
CEM
61.42 0.12
44.84 + 1.36
58.01 + 0.49
Synthetic
Autoregressive CBM
62.17 1 0.11
45.30 1: 1.29
59.60 1 0.62
Global SCBM
61.57 0.05
44.53
58.39
Amortized SCBM
62.41 + 0.20
45,85 1: 1.45
58.96 1 0.38
Hard CBM
94.97 = 0.07
77.22 + 0.33
67.72 + 0.57
CEM
95.12 1 0.07
78.20 + 0.28
69.60 + 0,30
CUB
Autoregressive CBM
95.33 0.07
79.21 + 0.21
69.24 + 0.44
Global SCBM
94.99 + 0.09
76.83 + 0.47
68.19 + 0.63
Amortized SCBM
95.22 0.09
78.29 + 0.28
69.87 + 0.56
Hard CBM
85.51 0.04
81.54 1
69.73
CEM
85.12 1 0.14
81.06 1 0.21
72.24 1: 0.33
CIFAR-10
Autoregressive CBM
85.31 + 0.06
81.31 + 0.10
68.88 + 0.47
Global SCBM
85.86 + 0.04
81.81 + 0.19
70.74 + 0.29
Amortized SCBM
86.00 + 0.03
81.97 + 0.20
71.66 + 0.25
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Claims are supported by evidence in the Results section and Appendix. Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
18

---

Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Yes, we have Limitations Future Work paragraph at the end of the conclusion.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We provide derivations of the method's theoretical foundations (detailed up to an acceptable degree of expected math knowledge) in the Method section.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
19

---

Justification: We disclose hyperparameters in the main text and Appendix. We also offer the code for reproducibility case any information missing.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We have released an anonymized version of the repository.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.ce)
wwwpibmmsssiooooooooooco for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new spen-s benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:// nips. /ttccc//pl//. htcc/publcc//ccc / Cooe/eooeeeeeeeeeeee ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
20

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: See Question
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide error bars in all experiments importance to reproducible research. For the Appendix, we have reduced the number of seeds and/or experiment size to save computational resources for the environment's sake.
believe this to be of utmost
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
21

---

Justification:
See Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: httprrsssss//ppllet///ccccccccee
Answer: [Yes]
Justification: The code of ethics was followed.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Given the more foundational work of this paper, there not a direct negative influence that the authors think of that might arise from this work specifically.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairness considerations (c.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
22

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
To the best of our knowledge, our work does not have high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Licenses for all used datasets were clearly stated.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. cow com/datasets: has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: In the Appendix, the data generating mechanism is clearly stated for the introduced synthetic dataset. Additionally, the new method is described in detail.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
23

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "I4wRa5GYvz", "reviewer": "Reviewer_rLqE", "summary": "This paper presents a method of performing interventions on Concept Bottleneck Models. The method parametrizes the space of concepts with a generative model of Bernoulli distribution and concept logits with a normal distribution, whose mean and variance depends on the input data distribution. The method also compares amortized covariance, which is instance-based, and a global covariance, marginalized over all samples. Rather than doing the interventions one-by-one as in previous works, this method also  modifies a single concept by hand, which in turns modifies other related concepts using the covariance learned. Experiments demonstrate that the method performs competitively against related CBM works that also allow for intervention, and is much faster during inference. During intervention, the model performs competitively against other baselines.", "questions": "Often times we don’t have many labeled samples of fine-grain annotations (like CUB) or the annotation can be noisy (like CLIP annotations). To what extend do these two factors affect the learning of the distributions in your method? Is it true that if the global dependency structure is of poor quality (correlation matrix), then this can severely impact the downstream performance and interpretability?", "limitations": "A key part of this method is learning a good dependency matrix for each sample or globally. I would expect this requires a lot of samples and can severely impact the performance if the matrix is not representative of the relationships between concepts. This is clearly stated in the work, hence the limitations are sufficiently stated.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The problem presented in this work is well-motivated and is of great importance to the current field of interpretable machine learning. While there are existing works that also formulate the concepts as a known distribution, this method has demonstrated well its advantages from the lens of intervention. Hence, this work has met the bar in terms of novelty and contributions in this area of research. The experiments of this work are also solid and adequate in addressing the claims made. While the performance is sometimes worse than related methods, they are still comparable and is not the main focal point of this work.", "weaknesses": "While the overall writing of the paper is clear, Section 3.3 was a bit challenging to follow, especially the part about the confidence region. It is not entirely clear to me what the confidence region is of. As an example, L183-184 “the likelihood-based\nconfidence region provides a natural way of capturing the region of possible $\\mathbf{\\eta}’_{\\mathcal{S}}$ that fulfil our desiderata”. The confidence of what? What desiderata? I suggesting either adding a diagram, a bit more background, a reminder, or an example, on what the intervention is trying to achieve so it more clearly demonstrates the process.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, \nThank you for your comments and positive feedback. Please find below our answers to the open points.\n\n\n> While the overall writing of the paper is clear, Section 3.3 was a bit challenging to follow, especially the part about the confidence region. It is not entirely clear to me what the confidence region is of. As an example, L183-184 “the likelihood-based confidence region provides a natural way of capturing the region of possible $\\boldsymbol{\\eta}_{\\mathcal{S}}$ that fulfil our desiderata”. The confidence of what? What desiderata? I suggesting either adding a diagram, a bit more background, a reminder, or an example, on what the intervention is trying to achieve so it more clearly demonstrates the process.\n\nThank you for raising this point. We will make sure to improve the clarity of this subsection in the manuscript.\nBelow, we provide the rewritten version of lines 174-188.\n\n*In normal CBMs, an intervention affects only the concepts on which the user intervenes. As such, Koh et al. (2020) set $\\eta_i'$ to the 5th percentile of the training distribution if $c_i = 0$ and the 95th percentile if $c_i = 1$. While this strategy is effective for SCBMs too, see Appendix C.3, the modeling of the concept dependencies warrants a more thorough analysis of the *intervention strategy*. We present two desiderata, which our intervention strategy should fulfill.*\n\n*i)  $p(c_i | \\eta_i') \\geq p(c_i | \\mu_i)$*\n\n*That is, the likelihood of the intervened-on concept $c_i$ should always increase after the intervention. If SCBMs used the same strategy as CBMs, it could happen that the initially predicted $\\mu_i$ was more extreme than the selected training percentile. Then, the interventional shift $\\eta'_i - \\mu_i$ in Equation 7 would point in the wrong direction. This, in turn, would cause $\\boldsymbol{\\eta}\\_{\\setminus \\mathcal{S}}$ to shift incorrectly.* \n\n*ii) $|\\eta_i' - \\mu_i|$ should not be ``too large''*\n\n*We posit that the interventional shift should stay within a reasonable range of values. Otherwise, the effect on $\\eta_{\\setminus \\mathcal{S}}$ would be unreasonably large such that the predicted $\\boldsymbol{\\mu}_{\\setminus \\mathcal{S}}$ would be completely disregarded.*\n\n\n*To fulfill these desiderata, we take advantage of the explicit distributional representation: the likelihood-based confidence region of $\\mu_i$ provides a natural way of specifying the region of possible $\\boldsymbol{\\eta}\\_{\\mathcal{S}}'$ that fulfil our desiderata. Informally, a confidence region captures the region of plausible values for a parameter of a distribution.\nNote that the confidence region takes concept dependencies into account when describing the area of possible $\\boldsymbol{\\eta}\\_{\\mathcal{S}}'$. To determine the specific point within this region, we search for the values $\\boldsymbol{\\eta}\\_{\\mathcal{S}}'$, which maximize the log-likelihood of the known, intervened-on concepts $\\mathbf{c}_{\\mathcal{S}}$, implicitly focusing on concepts that the model predicts poorly.*\n\n> A key part of this method is learning a good dependency matrix for each sample or globally. Is it true that if the global dependency structure is of poor quality (correlation matrix), then this can severely impact the downstream performance and interpretability?\n\nYes, modeling the second central moment naturally induces more variability. As the reviewer rightly states, it is important that the learned dependency structure is somewhat accurate. That is, the benefits of learning the dependencies have to outweigh the difficulty of learning them. This is why we introduced the LASSO-like regularizer, which helps in avoiding overfitting. For the regularizer weight $\\lambda_2 \\rightarrow \\infty$, we would recover the hard CBM (apart from diagonal variance), thus, with a validation set at hand, one can find a suitable value. Note that we also show in App. C.2, that SCBMs are not very sensitive to this $\\lambda_2$. The importance of not learning a poor-quality structure is also what led us to not model higher-order central moments, thereby reducing the overfitting potential.\n\n\n\n > Often times we don’t have many labeled samples of fine-grain annotations (like CUB) or the annotation can be noisy (like CLIP annotations). To what extend do these two factors affect the learning of the distributions in your method? \n\nLearning a covariance structure adds a layer of complexity, therefore, it is natural to wonder how our method can deal with non-optimal scenarios. We believe with the regularization, discussed in the previous answer, users have good control over the method’s behavior and, therefore, good control over the potential issues you point out. Of course, noisier annotations will require a higher $\\lambda_2$ regularization, thus, potentially omitting some signal. However, this tradeoff is unavoidable, and regularizing the parameters is standard practice [1]. Additionally, the proposed intervention strategy aids in preventing wrongful interventions by adhering to the predicted uncertainty. \n\nWe have deliberately chosen the datasets in the manuscript to showcase the versatility of SCBMs in such settings. That is, the CUB dataset contains only around 6000 training samples, thus being a rather small dataset, in which SCBMs prevail. To cover the case of noisy concepts, as well as a dataset without fine-grained human annotations, we have chosen the CLIP-annotated CIFAR-10 dataset, in which SCBMs also outperform the baselines. These findings also hold out-of-the-box on the CLIP-annotated CIFAR-100 dataset reported in Figure 1 of the supplementary PDF file, showcasing the scalability of SCBMs to larger concept sets. Thus, we can confidently conclude that SCBMs are effective in capturing the underlying dependencies.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QRQ0zSeEtf", "reviewer": "Reviewer_sfUo", "summary": "This paper introduces a novel concept dependency modelling scheme via an explicit distributional parameterization based on multivariate Gaussian distributions. This allows for capturing the dependencies between different concepts, while giving rise to an effective intervention strategy. \n\nThe experimental results vouch for the efficacy of the proposed approach, which allows for end-to-end training, and exhibits on-par or improved performance before and after interventions.", "questions": "Please see the Weaknesses section.", "limitations": "The authors mention the limitations in the dedicated section. The main limitation of the proposed approach is complexity.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "This constitutes a well-motivated approach based on some probabilistic arguments for capturing and examining  concept dependency. The overall idea is simple and easy to follow.", "weaknesses": "Even though the proposed approach uses either an amortized or global  formulation, the complexity of learning the covariance matrix either way surely introduces a lot of complexity. This is especially true in cases where the number of concepts are large, a common occurrence in complex datasets with many classes such as CIFAR-100 and ImageNet (not explored in this work). This complexity is further burdened by the Monte Carlo sampling scheme; commonly, 1-10 samples are enough, but the authors report the usage of 100 MC samples which would greatly slow down the training process. \n\nIn this context:\n1) What is the complexity compared to a standard diagonal approach? \n2) It would be important to assess the complexity and performance in the case of larger datasets like ImageNet. The authors use the CLIP generated data of [1] for CIFAR-10, so applying the approach to CIFAR-100 and ImageNet should be quite easy. \n3) In my experience with MC-based methods, I find Table 2 a bit hard to believe, especially with 100 MC samples that the authors report. Can the authors provide wall time measurements for the per-epoch time for each method? Even for just Hard CBM and Amortized SCBM. 4) Having access to the code, so the experiments could be validated would also be important. \n5) How does the number of MC samples affect the complexity and performance? I find an ablation study to be necessary here.\n6) How many GPUs did the utilized cluster have?\n\nThe considered Bernoulli formulation is very similar to the work in [2], which they call concept discovery. It constitutes a data-driven Bernoulli posterior for concept selection and the formulation is very similar. I find this method more appropriate compared to other approaches mentioned in the related work and the experimental evaluation, and some discussion/results should be included in the main text. \n\nIn Table 1, the authors report the concept accuracy which I assume is based on a binary accuracy. Recent works have suggested that maybe this is not a good metric in the context of interpretability, since most concepts are sparse and this can lead to misleading results [3]. Can the authors report the Jaccard similarity between the ground truth and the obtained concepts for all the methods?\n\n[1] Oikarinen, T., et al., Label-free concept bottleneck models, ICLR 2023\n\n[2] Panousis, K. P., et al., Sparse Linear Concept Discovery Models, ICCV CLVL 2023 \n\n[3] Panousis, K. P., et al., Coarse-to-Fine Concept Bottleneck Models, Arxiv 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments. In our response below, we address the remaining open points.\n\n> What is the complexity compared to a standard diagonal approach?\n\nAs the reviewer rightly points out, modeling dependencies comes at a complexity overhead cost. In terms of memory complexity SCBMs scale quadratically with the number of concepts, while a diagonal approach would scale linearly. However, the sampling itself is very fast. We use the Cholesky decomposition $\\Sigma = L L^T$; thus, sampling from the multivariate distribution is done via $Lx$, where x is a vector of $C$ samples of a univariate standard normal distribution. This is in stark contrast to the autoregressive baseline, which requires $C$ sequential passes through MLPs. \n\n> It would be important to assess the complexity and performance in larger datasets like ImageNet. The authors use the CLIP-generated data of [1] for CIFAR-10, so applying the approach to CIFAR-100 and ImageNet should be quite easy.\n\nWe present the result on the CIFAR-100 dataset with 892 concepts obtained from [9] in Figure 1 of the rebuttal PDF to showcase the scalability of SCBMs. Additionally, we present the wall time per method in Table 2. As can be seen, the CEM and AR baselines take a long time to compute when increasing the concept set size. After 4 days, we are still waiting for the Autoregressive CBM to finish and can, unfortunately, only present their intervention performance up to this point. Therefore, we refrain from computing results on ImageNet with 4751 concepts, as we would be unable to get any results on time.\n\nThe additional results underline the efficiency of our method in terms of computational complexity and performance. Notably, the Autoregressive baseline has a negative dip, which is likely due to the independently trained target predictor not being aligned with the concept predictors in this noisy CLIP-annotated scenario. Note that they need to train independently to avoid the sequential MC sampling during training, which would otherwise increase training time significantly, as seen in the test wall times in Table 2. Our jointly trained SCBMs do not have this issue and surpass the baselines.\n\n> … Can the authors provide wall time measurements for the per-epoch time for each method? Having access to the code, so the experiments could be validated would also be important.\n\nWe chose 100 MC samples to not disadvantage the Autoregressive CBM, which used 200 MC samples. We provide an ablation for a smaller number of samples in Figure 2 of the supplementary PDF, showing SCBMs remain effective.\n\nIn Table 2 of the supplementary PDF, we present the wall time for each method in two datasets with varying concept set sizes. Please note that the sampling itself is extremely fast for the multivariate Gaussian distribution used in SCBMs, and, contrary to e.g., VAEs, the MC samples are only passed through a lightweight target prediction head. This is a significant benefit over the autoregressive baseline, as their procedure consists of sequentially sampling each concept via MLPs, while SCBMs can obtain all samples directly from the specified multivariate Gaussian.\n\nLet us denote by $C$ the number of concepts and $M$ the number of MC samples.\nIf we analyze the computational complexity by the amount of MLP forward passes, then CEMs scale $O(C)$ (2 embeddings per concept), Autoregressive CBMs scale $O(C \\times M)$ (1 pass through $C$ MLPs per MC sample), and SCBMs scale $O(M)$ (1 pass through classifier head per MC sample).\n\nFor code, we refer to the footnote on page 3 of the originally submitted manuscript for our anonymized repository.\n\n> How does the number of MC samples affect the complexity and performance?\n\nIn Figure 2 of the supplementary PDF we provide an ablation for the number of MC samples on CUB. The runtimes differ minimally by ~0.01s per epoch. Results show that SCBMs still perform well with fewer MC samples. Still, a larger amount (e.g. 100) of samples is completely feasible in SCBMs, supported by our fast sampling and forward pass.\n\n> How many GPUs did the utilized cluster have?\n\nThe experiments were run on an ordinary HPC platform, mostly consisting of GeForce RTX 2080ti’s. We would like to emphasize that each run uses only a single GPU.\nWe will adjust the “Resource Usage” paragraph to avoid ambiguity.\n\n> The considered Bernoulli formulation is very similar to the work in [2] … I find this method more appropriate compared to other approaches mentioned in the related work and the experimental evaluation, and some discussion/results should be included in the main text.\n\nWe agree that Panousis et al. (2023) and SCBMs share the Bernoulli relaxation for concept modeling. It is important to note that CDMs focus on discovering concepts, while SCBMs assume the presence of a concept-labeled dataset and focus on modeling their dependencies. As such, we believe the two works to be complementary, not substitutional. This is observable by noticing that both cited works by Panousis do not study interventions. We have reported successful results of SCBMs in CIFAR-10 and CIFAR-100, where concept annotations are “discovered” from CLIP. This shows that the concept discovery methods like CDMs can synergize with SCBMs. However, as the focus of this work lies on modeling the concept dependencies and interventions, we believe that autoregressive CBM is more closely related to our task and focus on it. \n\nWe will make sure to discuss the shared similarities with CDMs in the main text, as well as outline the combination of both methods as an exciting avenue for future work.\n\n> In Table 1, the authors report the concept accuracy…. Can the authors report the Jaccard similarity?\n\nIn Table 1 of the supplementary PDF, we include the Jaccard similarity between ground truth and predicted concepts. The resulting interpretations are in line with the prior reported accuracies. We will add this metric proposed by Panousis et al. (2024) to the manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OKhLlGVlUG", "reviewer": "Reviewer_9xQm", "summary": "Focusing on concept bottleneck models, this paper extracts concept dependencies with a multivariate normal distribution and derives an intervention strategy based on the confidence region of the normal distribution that incorporates concept correlations for better interventional effectiveness.", "questions": "1. Intervention by setting values can be difficult in some case, the example given by the paper: \"intervene on the CBM by setting its value to 1\" is an extreme case. Humans are not good at estimating probabilities.\n2. \"do not use the intervened-on concepts to update their remaining concept predictions\" may not necessarily be a bad thing. As in many real-world cases, there are exceptions to rules and patterns. Nothing is absolute. Universally \"extend the concept predictions with the modeling of their dependencies\" may be problematic. Are these exceptions handled by confidence region?\n3. What is the novelty comparing with E. Kim et al. (2023)? just relax the assumption of diagonal covariance matrix?\n4. Will the proposed method suffer leakage?", "limitations": "Limitations are adequately addressed.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is organized and written well.\n2. Concept bottleneck model is an important direction for xAI and even generalizability.", "weaknesses": "1. Experiment results show marginal improvement over existing methods.\n2. The novelty and contribution of this paper is limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and questions! Below is our point-by-point response.\n\n> Intervention by setting values can be difficult in some case, the example given by the paper: \"intervene on the CBM by setting its value to 1\" is an extreme case. Humans are not good at estimating probabilities.\n\nIndeed, specifying an exact value for interventions is a challenging task and an active research problem [8]. Still, a major benefit of CBMs is their ability for interventions via human-model interactions [2,3], as it enables domain experts to correct mistakes, so that more accurate target predictions can be made at the end. This is important in domains like healthcare, where human-model interactions have a significant impact on trust and patient outcomes.\nThus, we believe our research in this direction is important while acknowledging that the complementary problem of the human factor within interventions is not solved. \n\nNotably, while most CBM methods require users to intervene with exact values, SCBMs solve an optimization problem to arrive at a “reasonable” probability estimate. That is, not only do we not ask humans to specify probabilities, but we even solve an optimization problem to determine probabilities given hard concept values. Naturally, if desired by the user, this optimization routine can be omitted. Nevertheless, we will mention this problem in the limitation section.\n\n> \"do not use the intervened-on concepts to update their remaining concept predictions\" may not necessarily be a bad thing. As in many real-world cases, there are exceptions to rules and patterns. Nothing is absolute. Universally \"extend the concept predictions with the modeling of their dependencies\" may be problematic. Are these exceptions handled by confidence region?\n\nThe goal of this work is to enhance the effect of individual interventions and make the human-model interactions more scalable. This removes the burden of intervening on multiple correlated concepts, leading to improved performance. SCBMs achieve this by exploiting the concept dependencies observed during training. As such, we do make the assumption that the concept structure in the training set holds at inference time. If this assumption is violated, one could always compute the marginal instead of the conditional distribution at intervention time, thereby ignoring the concept dependencies.\n\n\n> What is the novelty comparing with E. Kim et al. (2023)? just relax the assumption of diagonal covariance matrix?\n\nWhile both works use a normal distribution, we believe there are substantial differences between our manuscript and the ProbCBM [4], as we outline below. We will make them more clear in the revised version of this manuscript.\n\n* ProbCBMs build upon CEMs. That is, the mean and variance are learned over the concept *embeddings* rather than modeling the concepts themselves. Even if they relaxed the assumption of a diagonal covariance matrix, in their current form, they would only capture the covariances of the embedding dimensions *within* a given concept but not the dependencies *across* concepts. Thus, we would not expect that such a relaxation would improve ProbCBMs’ intervention performance, and as such, in the context of interventions, we do not consider them to be very different from CEMs.\n\n* In a similar vein, please note that the loss of ProbCBMs contains a KL term, which “prevents the variances from collapsing to zero” [4]. On the other hand, SCBMs do not require such a term, as we’re modeling actual non-zero concept dependencies. To draw an analogy, we consider ProbCBMs to work similarly to VAEs [5], where the normal distribution acts as a regularizing prior (via the KL term) on the concept latent space. On the other hand, SCBMs leverage the normal distribution to learn the structure of the data itself, which in our case, are the concepts.\n\n* As pointed out by the reviewer, a major difference between the works is that SCBMs do not assume a diagonal covariance matrix. However, this generalization is non-trivial and poses multiple technical challenges. As such, we parameterize the covariance via its Cholesky decomposition, derive a maximum-likelihood-based loss different from [4], apply the Gumbel-Softmax reparameterization to perform joint training, and introduce the Lasso regularization. Lastly, a significant contribution is the introduction of the novel intervention strategy based on the confidence region (see Eq. 8) that leverages the learned concept dependencies while fulfilling the posed desiderata.\n\n* While ProbCBMs mainly focus on capturing concept and target uncertainty, SCBMs focus on interventions. As we have shown in Fig. 2 of the manuscript, we achieve this gap by a margin of up to 10 percentage points in some cases. Especially when comparing to CEMs, whose performance we believe to be a good proxy for ProbCBMs with respect to interventions, there is a major difference in intervention effectiveness.\n\n* A seemingly minor but, in fact, very important difference is that ProbCBMs work with concept embeddings as introduced by CEMs. On the other hand, SCBMs use hard, binary concepts as bottleneck. We elaborate on the importance of this difference below.\n\n> Will the proposed method suffer leakage?\n\nLeakage is an important problem in the CBM literature, as it damages the interpretability. While we do not explicitly optimize against leakage, we have made the deliberate design choice to characterize the concept bottleneck with hard, binary concepts to avoid leakage as much as possible [6]. We believe that leakage is the main driver of why CEMs perform suboptimal during interventions. Note that [7] even propose to approximate leakage by performance during interventions. Since ProbCBMs build upon CEMs, we believe that they are more susceptible to leakage than SCBMs. \nBut of course, our design choices do not fully prevent the occurrence of leakage, which is why we mention it in the limitations section.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear reviewers,

We would like to thank all of you for your thorough reviews and constructive feedback! Below, we summarise our responses to your main concerns, additional results, and changes to be implemented upon acceptance in the revised manuscript.

* We have included experiments on a new large-scale dataset, CIFAR-100 [10], with annotations for 892 concepts coming from VLMs. The results, displayed in Figure 1 of the additional supplementary PDF file, are consistent with the interpretation of SCBMs in previous datasets, showcasing the scalability of our method to a larger number of concepts and classes. For a more detailed discussion, we refer to our response to reviewer sfUo.

* To understand the computational complexity of SCBM, we report the wall times for one training and inference epoch for each of the studied methods in Table 2 of the supplementary PDF. In view of the large-scale dataset introduced in the rebuttal, we report both CUB and CIFAR-100 to provide insights for two different datasets. These results show that SCBMs are significantly more efficient and hence more scalable than prior work for modeling concept dependencies, i.e., Autoregressive (AR), which required longer computational times, especially when scaling the number of concepts. Particularly, AR is slower at inference time, where speed is a key factor as a user will interact with the deployed model. For a more detailed discussion, we refer to our response to reviewer sfUo.

* The experiments in the proposed SCBM were carried out with 100 MCMC samples. We performed an ablation on this amount and show in Figure 2 of the supplementary PDF that SCBMs remain effective with 10 MCMC samples. 

* We include the Jaccard Index as a new metric for measuring concept prediction prior to interventions for all models and datasets in Table 1 of the supplementary PDF. The resulting interpretation is consistent with the previously reported accuracies. 

* We further clarified the method with a comprehensive explanation of the proposed intervention strategy via confidence regions, which we will include in the revised version of the manuscript. For more details, we refer to our response to reviewer rLqE.


[1] Tibshirani, Robert. "Regression shrinkage and selection via the lasso." Journal of the Royal Statistical Society Series B: Statistical Methodology 58.1 (1996): 267-288.

[2] Koh, Pang Wei, et al. "Concept bottleneck models." International conference on machine learning. PMLR, 2020.

[3] Shin, Sungbin, et al. "A closer look at the intervention procedure of concept bottleneck models." International Conference on Machine Learning. PMLR, 2023.

[4] Kim, Eunji, et al. "Probabilistic Concept Bottleneck Models." International Conference on Machine Learning. PMLR, 2023.

[5] Kingma, D. P., & Welling, M. “Auto-encoding Variational Bayes.” International Conference on Learning Representations, ICLR, 2014.

[6] Havasi, Marton, Sonali Parbhoo, and Finale Doshi-Velez. "Addressing leakage in concept bottleneck models." Advances in Neural Information Processing Systems 35 (2022): 23386-23397.

[7] Zabounidis, Renos, et al. "Benchmarking and Enhancing Disentanglement in Concept-Residual Models." arXiv preprint arXiv:2312.00192 (2023).

[8] Collins, Katherine Maeve, et al. "Human uncertainty in concept-based ai systems." Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. 2023.

[9] Oikarinen, T., et al., "Label-free concept bottleneck models", ICLR 2023

[10] Krizhevsky, A., & Hinton, G. (2009). “Learning multiple layers of features from tiny images.” Toronto, Ontario: University of Toronto.

Meta-review: The paper introduces a novel Stochastic Concept Bottleneck Model (SCBM) that significantly enhances the functionality of Concept Bottleneck Models (CBMs) by modeling concept dependencies using a multivariate Gaussian distribution. This approach allows for more effective interventions where a change in one concept automatically adjusts related concepts, improving the model's overall interpretability and usability. Despite the complexity of learning the covariance matrix among concepts being a challenge, the authors effectively address this through detailed computational analyses and comparisons with baseline methods. They also commit to further clarifications in the final manuscript for sections that were difficult to understand, leading to a recommendation for acceptance.
Decision: Accept (poster)