Stochastic Concept Bottleneck Models

Authors: moritz  vandenhirtz
sonia  laguna
ričards  marcinkevičs
julia e vogt
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 17:13:18

Abstract: Concept Bottleneck Models (CBMs) have emerged as a promising interpretable method whose final prediction is based on intermediate, human-understandable concepts rather than the raw input. Through time-consuming manual interventions, a user can correct wrongly predicted concept values to enhance the model's downstream performance. We propose *Stochastic Concept Bottleneck Models* (SCBMs), a novel approach that models concept dependencies. In SCBMs, a single-concept intervention affects all correlated concepts, thereby improving intervention effectiveness. Unlike previous approaches that model the concept relations via an autoregressive structure, we introduce an explicit, distributional parameterization that allows SCBMs to retain the CBMs' efficient training and inference procedure. 
Additionally, we leverage the parameterization to derive an effective intervention strategy based on the confidence region. We show empirically on synthetic tabular and natural image datasets that our approach improves intervention effectiveness significantly. Notably, we showcase the versatility and usability of SCBMs by examining a setting with CLIP-inferred concepts, alleviating the need for manual concept annotations.

Main paper: 
 introduction: 
 In today's world, machine learning plays a crucial role in making important decisions, from healthcare to finance and law. However, as these algorithms become more complex, understanding how they arrive at their decisions becomes increasingly challenging. This lack of interpretability is a significant concern, especially in situations where trustworthiness, transparency, and accountability are paramount (Lipton, 2016; Doshi-Velez & Kim, 2017) . Recent studies have focused on Concept Bottleneck Models (CBMs) (Koh et al., 2020; Havasi et al., 2022; Shin et al., 2023) , a class of models that predict human-understandable concepts upon which the final target prediction is based. CBMs offer interpretability since a user can inspect the predicted concept values to understand how the model arrives at its final target prediction. Moreover, if they disagree with a concept prediction, they can intervene by adjusting it to the right value, which in turn affects the target prediction.For example, consider the yellow warbler in Figure 1 (a) , where a user might notice that the binary concept 'yellow primary color' is mispredicted. Upon this realization, they can intervene on the CBM by setting its value to 1, which increases the probability of the class yellow warbler. This way of interacting allows any untrained user to engage with the model to increase its predictive performance.However, if the user input is that the primary color is yellow, should not the likelihood of a yellow crown increase too? This adaptation would increase the predicted likelihood of the correct class even more, as yellow warblers are characterized by their fully yellow body. Currently, vanilla CBMs do not exhibit this behavior as they do not use the intervened-on concepts to update their remaining concept predictions. This indicates that they suboptimally adapt to the additional knowledge gained. S is used to infer the logits η \S of the remaining concepts. (c) Visualization of the learned global dependency structure as a correlation matrix for the 112 concepts of CUB (Wah et al., 2011) . Characterization of concepts on the left.To this end, we propose to extend the concept predictions with the modeling of their dependencies, as depicted in Figure 1 .The proposed approach captures the concept dependencies by modeling the concept logits with a learnable non-diagonal normal distribution, which enables efficient, scalable computing of the effect of interventions on other concepts. By integrating concept correlations, we reduce the time and effort of having to laboriously intervene on many correlated variables and increase the efficacy of interventions on the downstream prediction. Thanks to the explicit distributional assumptions, the model is trained end-to-end, retaining the training and inference speed of classic CBMs as well as the benefits of training the concept and target predictor jointly. Moreover, we show that our method excels when querying user interventions based on predicted concept uncertainty (Shin et al., 2023) , further highlighting the practical utility of our approach as such policies spare users from manually sifting through the concepts to identify necessary interventions. Lastly, based on the distributional concept parameterization, we propose a novel approach for computing dependency-aware interventions through the likelihood-based confidence region.Contributions This work contributes to the line of research on concept bottleneck models in several ways. (i) We propose to capture and model concept dependencies with a multivariate normal distribution. (ii) We derive a novel intervention strategy based on the confidence region of the normal distribution that incorporates concept correlations. Using the learned concept dependencies during the intervention procedure allows for stronger interventional effectiveness. (iii) We provide a thorough empirical assessment of the proposed method on synthetic tabular and natural image data. Additionally, we combine our method with concept discovery where we alleviate the need for annotations by using CLIP-inferred concepts. In particular, we show the proposed method (a) discovers meaningful, interpretable patterns in the form of concept dependencies, (b) allows for fast, scalable inference, and (c) outperforms related work with respect to intervention effectiveness thanks to the proposed concept modeling and intervention strategy. 
background & related work: 
 Concept bottleneck models (Koh et al., 2020; Lampert et al., 2009; N. Kumar et al., 2009) are typically trained on data points (x, c, y), comprising the covariates x ∈ X , target y ∈ Y, and C annotated binary concepts c ∈ C. Consider a neural network f θ parameterized by θ and a slice ⟨g ψ , h ϕ ⟩ (Leino et al., 2018) s.t. ŷ • • = f θ (x) = g ψ (h ϕ (x)). CBMs enforce a concept bottleneck ĉ • • = h ϕ (x)such that the model's final output depends on the covariates x solely through the predicted concepts ĉ.While Koh et al. (2020) propose the soft CBM, where the concept logits parameterize the bottleneck, Havasi et al. (2022) argue that such a representation leads to leakage, where additional unwanted information in the concept representation is used to predict the target (Margeloiu et al., 2021; Mahinpei et al., 2021) . Thus, they parameterize the bottleneck by binarized concept predictions and call it the hard CBM. Then, Havasi et al. (2022) equip the hard CBM with an autoregressive structure of the form c i |x, c <i , which is supposed to learn the concept dependencies. As such, the implicit autoregressive modeling of concept dependencies by Havasi et al. (2022) is the most related to the current work. Complementary to our work, Heidemann et al. (2023) analyze how a CBM's performance is affected by concept correlations. Unlike approaches that restrict the bottleneck to prevent leakage, Concept Embedding Models (CEM) (Espinosa Zarlenga et al., 2022) represent each concept with an embedding vector from which the concept probabilities can be inferred. E. Kim et al. (2023) model the embedding with a normal distribution, assuming a diagonal covariance matrix, which prevents them from capturing concept dependencies. Therefore, their intervention performance is not expected to differ from that of CEMs. Recent works explored how a CBM-like structure can be enforced even without a concept-annotated training set. Yuksekgonul et al. (2023) transform a pre-trained model into a CBM via a concept bank from concept activation vectors and multimodal models (B. Kim et al., 2018) , while Oikarinen et al. (2023) query GPT-3 (Brown et al., 2020) for the concept set C and assign the values of the concept activations to each datapoint x with CLIP (Radford et al., 2021) similarities. Similarly, Panousis et al. (2023) uses CLIP to probabilistically discover a sparse set of concepts for each input, which could be used in our model for a fully probabilistic pipeline. Lastly, Marcinkevičs et al. (2024) instead relax the need for a concept labeled training set to a smaller validation set by fine-tuning a pre-trained model.Intervenability (Marcinkevičs et al., 2024) is a crucial element of CBMs as it allows the user to correct wrongly predicted concepts ĉ to c ′ , which in turn affects the target prediction of the model ŷ′ . If multiple concepts are intervened on sequentially, the order of interventions is important. To this end, Sheth et al. (2022) and Shin et al. (2023) explore multiple policies according to which the order of concepts is determined. Chauhan et al. (2023) propose to combine predefined policies with learnable weighting parameters, while Espinosa Zarlenga et al. ( 2024) learn the policy itself. Concurrently, Singhi et al. (2024) learn a realignment module to align concept predictions. Steinmann et al. (2023) argue that instance-specific interventions are costly and store previous interventions in a memory to automatically reapply them for similar data points. Lastly, Collins et al. (2023) explore the advantages of including uncertainty rather than treating humans as oracles.Our work models concept dependencies by parameterizing the bottleneck with a distribution. In a similar vein, Variational Autoencoders (Kingma & Welling, 2014) parameterize the bottleneck with a normal distribution to model and generate new data. Stochastic Segmentation Networks (Monteiro et al., 2020) parameterize the logits of a segmentation map with a non-diagonal normal distribution to capture the spatial correlations of pixels and model the aleatoric uncertainty. The modeling of uncertainty with a distribution is also explored by Bayesian Neural Networks (Neal, 1995) that learn a probability distribution over the neurons of a neural network.Σ ∈ R C×C s.t. Σ = W W T + D. Let D ∈ R C×C s.t. D = δI, where δ i ∼ U [0,1] for 1 ≤ i ≤ C.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Yes, we have a Limitations & Future Work paragraph at the end of the conclusion. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methods: 
 We propose Stochastic Concept Bottleneck Models1 (SCBM), a novel concept-based method that relaxes the implicit CBM assumption of independent concepts. SCBM captures the concept dependencies by learning their multivariate distribution. As a result, interventions become more effective and scalable, as a single intervention can influence multiple correlated concepts. A schematic overview of the proposed method is depicted in Figure 1 (b).To capture the concept dependencies, we model the concept logits η with a learned multivariate normal distribution. Modeling logits with a normal distribution has proven to be effective in the context of segmentation (Monteiro et al., 2020) . While Monteiro et al. (2020) use it to capture the spatial dependencies of pixels, we, instead, model the relations between concepts, where the properties of the normal distribution will prove useful. A neural network is trained to predict the distribution's parameters η | x ∼ N (µ(x)), Σ(x)), where µ(x) ∈ R C , and Σ(x) ∈ R C×C . Thus, the traditional assumption of independent concepts c i ⊥ ⊥ c j | x, ∀i ̸ = j is relaxed to c i ⊥ ⊥ c j | η, ∀i ̸ = j, where the assumed normal distribution induces linear concept dependencies. The inductive bias of linearity is useful in practice as it is more robust to overfitting and computationally more scalable with respect to C compared to its nonlinear alternative (Havasi et al., 2022) , as we will show in Section 5.To learn the distribution, we minimize the negative log-likelihoodEQUATIONwhere ϕ are the parameters of a neural network that predicts the distribution η | x ∼ N (µ(x)), Σ(x)). This integral is intractable due to the softmax operation applied in p(c | η). Thus, the integral is approximated by M Monte Carlo samples-log p(c | η)p ϕ (η | x)dη ≈ -log 1 M M m=1 p(c | η (m) ), η (m) | x ∼ N (µ(x)), Σ(x)) . (2)In order to learn ϕ, we make use of the parameterization as normal distribution and employ the reparameterization trickη (m) | x = µ(x) + L(x)ϵ (m) , L(x)L(x) T = Σ(x), ϵ (m) ∼ N (0, I)such that gradients can be computed with respect to the parameters. Lastly, we incorporate the new relaxed conditional independence assumptionEQUATIONwhere p(c i | η i ) describes a Bernoulli distribution parameterized by the sigmoid-transformed logits σ(η i ). Combining the above considerations results in the following reformulation of the negative log-likelihood:EQUATIONwhere BCE stands for Binary Cross Entropy, and the logsumexp trick is used for numerical stability.The distribution-based modeling procedure allows for efficient sampling, thus, enabling SCBM to train concept and target predictors jointly, sequentially, or independently. In contrast, the autoregressive alternative (Havasi et al., 2022) requires independent training due to the computational complexity. We adopt a joint training scheme to obtain the benefits of end-to-end learning where concept and target predictors can adjust to each other. To prevent leakage, we follow Havasi et al. (2022) and train the model with the hard {0, 1} concept values as bottleneck rather than the logits used in the original CBM (Koh et al., 2020) . To this end, we employ the straight-through Gumbel-Softmax trick (Jang et al., 2017; Maddison et al., 2017) that approximates Bernoulli samples while being differentiable. The target predictor g ψ is then learned by minimizing the negative log-likelihood-log p(y | x) = -log c∈C p ψ (y | c)p(c | x) ≈ -log 1 M M m=1 p ψ (y | c (m) ), c (m) ∼ p(c | x).(5)Lastly, the learned dependencies are regularized by following Occam's razor and to prevent overfitting. We take inspiration from the Graphical Lasso (Friedman et al., 2008) and penalize the off-diagonal elements of the precision matrix Σ -1 .By combining concept, target, and precision loss with weighting factors λ 1 and λ 2 , we arrive at the final loss function-log M m=1 exp C i=1 -BCE c i , σ(η (m) i ) + λ 1 CE y, 1 M M m=1 g ψ (c (m) ) + λ 2 i̸ =j Σ(x) -1 i,j . (6)The introduced amortized covariance matrix Σ(x) provides the flexibility to tailor its predicted concept dependencies to each data point, making it adaptable to many data-generating mechanisms. For example, in the commonly used CUB (Wah et al., 2011; Koh et al., 2020) , it can learn the class-wise concept structure present in the dataset. The explicit dependency representation inferred by the learned covariance matrix is useful as it provides insights into the learned correlations among the concepts, which is important for understanding and interpreting the model behavior.However, an amortized covariance matrix comes at the price of not being able to visualize and interpret a unified concept structure on a dataset level. Depending on the need of the application, such a global structure might be preferable. Thus, we propose a variation of SCBM, where the covariance matrix is not amortized (Σ(x)), but learned globally (Σ). An example of the global concept structure learned on CUB is shown in Figure 1 (c) . This variation has the inductive bias of assuming a constant covariance matrix, whose utility depends on the underlying data-generating mechanism. We recommend using the more flexible, amortized version by default and only utilizing a global covariance if the strong assumption of fixed dependencies is reasonable. We will explore this empirically in more detail in Section 5.A distinguishing property of CBM-like methods is the user's capacity to correct wrongly predicted concepts, which in turn affects the target prediction (Marcinkevičs et al., 2024) . For a big concept set, this intervention procedure can become quite laborious as a user has to inspect and manually intervene on each concept separately. SCBMs are designed to alleviate this need by utilizing the learned concept dependencies such that a single intervention affects all related concepts as modeled by the multivariate normal distribution.The parameterization as a multivariate normal distribution allows for a quick, scalable intervention procedure. Given a set S ⊂ {1, . . . , C} of concept interventions, the effect on the remaining concepts c \S is computed via their logits η \S by conditioning on the intervention logits η ′ S , utilizing the known properties of the normal distributionEQUATIONIn standard CBMs, an intervention affects only the concepts on which the user intervenes. As such, Koh et al. (2020) set η ′ i to the 5th percentile of the training distribution if c i = 0 and the 95th percentile if c i = 1. While this strategy is effective for SCBMs too, see Appendix C.5, the modeling of the concept dependencies warrants a more thorough analysis of the intervention strategy. We present two desiderata, which our intervention strategy should fulfill.i) p(c i | η ′ i ) ≥ p(c i | µ i )The likelihood of the intervened-on concept c i should always increase after the intervention. If SCBMs used the same strategy as CBMs, it could happen that the initially predicted µ i was more extreme than the selected training percentile. Then, the interventional shift η ′ i -µ i in Eq. 7 would point in the wrong direction. This would cause η \S to shift incorrectly.i -µ i | should not be "too large". We posit that the interventional shift should stay within a reasonable range of values. Otherwise, the effect on η \S would be unreasonably large such that the predicted µ \S would be completely disregarded.To fulfill these desiderata, we take advantage of the explicit distributional representation: the likelihood-based confidence region of µ i provides a natural way of specifying the region of possible η ′ S that fulfill our desiderata. Informally, a confidence region captures the region of plausible values for a parameter of a distribution. Note that the confidence region takes concept dependencies into account when describing the area of possible η ′ S . To determine the specific point within this region, we search for the values η ′ S , which maximize the log-likelihood of the known, intervened-on concepts c S , implicitly focusing on concepts that the model predicts poorly:EQUATIONwhere d = |S|. The first inequality describes the confidence region. It is based on the logarithm of the likelihood ratio, which, after multiplying with -2, asymptotically follows a χ 2 distribution (Silvey, 1975) . The last two inequalities restrict the region to the desired direction. Note that η ′ S is computed to determine the conditional effect of the interventions on η \S using Equation 7. When predicting ŷ′ under interventions, the logits η \S are then used for sampling the binary concept values c \S while the intervened-on concepts c ′ S are directly set to their known, binary value.H ∈ R N ×C s.t. η n ∼ N (0, Σ) for 1 ≤ n ≤ N . 4. Let c n,i = 1 {ηn,i≥0} for 1 ≤ n ≤ N and 1 ≤ i ≤ C.5. Let h : R C → R p be a randomly initialised multilayer perceptron with ReLU nonlinearities.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Justification: We provide derivations of the method's theoretical foundations (detailed up to an acceptable degree of expected math knowledge) in the Method section. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experimental setup: 
 Datasets and Evaluation We perform experiments on a variety of datasets to showcase the validity of our method. Inspired by Marcinkevičs et al. (2024) , we introduce a synthetic tabular dataset with a data-generating mechanism that contains fixed concept dependencies we can regulate. In particular, the concept logits η are sampled from a randomly initialized positive definite covariance matrix and generate x. Binary concept values c are inferred from η and generate the target y. We refer to Appendix A.1 for a more detailed description.As a natural image classification benchmark, we evaluate on the Caltech-UCSD Birds-200-2011 dataset (Wah et al., 2011) , comprised of bird photographs from 200 distinct classes. It includes 112 concepts, such as wing color and beak shape, shared across the same class instances as revised in the original CBM work (Koh et al., 2020) . Additionally, we explore another natural image classification task on CIFAR-10 ( Krizhevsky et al., 2009) with 10 classes. To mitigate the concept annotations requirement, the concepts are synthetically acquired in a similar fashion to the concept discovery literature. We adopt the 143 concept classes generated via GPT-3 (Brown et al., 2020) in prior work (Oikarinen et al., 2023) . To obtain the binary concept values, we use the CLIP model (Radford et al., 2021) to compute the similarity between each instance of an image with the text embedding of a specific concept and compare it to the similarity of its negative counterpart, i.e. not the concept. Appendix A.2 contains further details about the natural image datasets.To compare methods, we evaluate the model performance based on the concept and target accuracy. We compute test performance before and after intervening on an increasing number of concepts. The order of concepts in the intervention is determined by an uncertainty-based policy (Shin et al., 2023) that selects the concept whose predicted probability is closest to 0.5. We also show results for a random policy in Appendix C.3. Additionally, we evaluate the calibration of the predicted concept uncertainties that are being used for the uncertainty-based policy, with the Brier score (Brier, 1950) and the Expected Calibration Error (Naeini et al., 2015; A. Kumar et al., 2019) .Baselines We evaluate the performance of our method in comparison with state-of-the-art models. Namely, we focus on the vanilla concept bottleneck model (CBM) by Koh et al. (2020) in its hard version (Havasi et al., 2022) , trained jointly using the straight-through Gumbel-Softmax trick (Jang et al., 2017; Maddison et al., 2017) , as a sensical baseline to our binary modeling of concepts. Additionally, we explore the concept embedding model (CEM) by Espinosa Zarlenga et al. ( 2022) that learns two concept embeddings, ĉ+ i and ĉi . These representations are used to predict the final concept probability with a learnable scoring function pi = s(ĉ+ i , ĉ- i ) = σ(W s [ĉ + i , ĉ- i ] T + b s ) and are then combined into a final concept embedding ĉi = (p i ĉ+ i + (1 -pi )ĉ - i )that is passed to the target predictor. Interventions are modeled by altering the concept probabilities pi . Note that Espinosa Zarlenga et al. ( 2022) optimize for intervention performance during training, which we omit, to ensure a fair comparison where no method was explicitly trained for intervention performance. Finally, we evaluate the autoregressive CBM structure proposed by Havasi et al. (2022) , where concept dependencies are learned with an autoregressive structure. Here, each concept c i is predicted with a separate MLP that takes as input a latent representation of the input f θ (x) and all previous concepts c 1 , ..., c i-1 . To obtain a good initialization of the autoregressive structure, it is pretrained for 50 epochs. As the Monte Carlo sampling from the autoregressive structure is time-consuming, the target predictor g ψ is trained independently using the ground-truth concepts as input. At intervention time, a normalized importance sampling algorithm is used to estimate the concept distribution.The model architectures comprise a backbone for concept prediction followed by a linear layer as head for an interpretable target prediction. More details can be found in Appendix B. To ensure the positive definiteness of the concept covariance matrix Σ, we parameterize it via its Cholesky decomposition Σ = LL ⊤ . Thus, we directly predict the lower triangular Cholesky matrix L. We will evaluate two options for SCBMs: using a global (Σ) or an amortized covariance matrix (Σ(x)). For the amortized version, we set the weighting terms λ 1 and λ 2 of Equation 6 to 1.For the global version, we initialize it with the estimated empirical covariance matrix and set λ 2 = 0, as we did not observe big differences when varying λ 2 . In Appendix C.4, we provide an ablation study, demonstrating that SCBMs are not very sensitive to the choice of λ 2 . At intervention time, we solve the optimization problem based on the 99%-confidence region with the SLSQP algorithm (Kraft, 1988) . In Appendix C.6, we provide an ablation with different confidence levels. Test performance In Table 1 , we report the results of the concept and target accuracy prior to interventions. Overall, SCBM performs on par with the baseline methods, with no clear outperforming or underperforming technique throughout the datasets. In Appendix C.7, we show that other metrics lead to the same interpretation. This shows that the additional overhead of learning the concept dependencies does not negatively affect the predictive performance. We note that the amortized covariance variant consistently surpasses the globally learned matrix due to its ability to adjust the predicted concept dependency structure and uncertainty on an instance level. On the other hand, the global variant offers a unified understanding of the concept correlations, an example of which is presented in Figure 1 (c). Notably, in CIFAR-10, even though the concept performance of CEM is the worst of all methods, it has the best target performance. This might suggest the presence of leakage in CEM's embeddings, as in CIFAR-10, the concept set alone is not sufficient to predict the target, and learning additional information might be useful. In Table 2 , we show the time it takes for training and testing of the methods. It is evident that the autoregressive CBM of Havasi et al. (2022) suffers from a slow sampling process due to its autoregressive structure, while SCBMs retain the efficiency of CBMs. In this paragraph, we analyze the intervention performance of SCBMs and their baseline models, focusing on their effectiveness in modeling concept dependencies and improving target accuracy. Figure 2 shows the intervention curves across ten seeds, where the performance is measured based on the concept and target accuracy. The order of concepts to intervene on is determined by an uncertainty-based policy that makes use of the predicted probabilities. In Appendix C.3, we present the intervention performance if concepts were selected randomly. The intervention curves in the first row show that SCBMs are superior in modeling the concept dependencies, as evidenced by their significantly steeper intervention curves compared to the baseline methods. Furthermore, the second row of Figure 2 indicates that the strong concept modeling translates to a significant improvement in downstream performance, partly thanks to the intervention strategy introduced in Section 3.3. We note that especially for the most practical scenario of only a small number of interventions, SCBMs outperform their counterparts. Comparing the SCBM variants, the natural image datasets show an overall better intervention performance with the amortized covariance matrix, following the trend of Table 1 , as it can capture the instance-wise correlation structure of the data. Only in the synthetic dataset, where the data-generating covariance matrix is fixed, does the global SCBM slightly outperform the amortized one. Thus, we advocate for the usage of the global variant only if the underlying assumption of a fixed covariance is reasonable. Lastly, the success of SCBMs on CIFAR-10, with CLIP-based concepts, shows our proposed method can work without human-annotated concepts. To strengthen this point and also showcase the scalability of our method, in Appendix C.1, we provide results on CIFAR-100 with 892 concepts, where our SCBMs also strongly outperform baselines.Analyzing the performance of the autoregressive CBM, which also captures concept dependencies, we observe that they expectedly have a better intervention performance than the hard vanilla CBM, which does not take correlations into account. However, it becomes evident that, compared to the concept performance of SCBMs, their autoregressive structure does not capture the dependencies to the full extent. This shows in the target accuracy, where they only match or outperform SCBMs towards the full set of intervened concepts. We attribute the better performance on the full intervention set to the independent training procedure utilized by autoregressive CBMs, which comes at the cost of lower test performance in CIFAR-10. Arguably, in a realistic use-case, such a high number of instance-level interventions is not sensible, and if it were, SCBMs could also be trained independently. Finally, the CEM shows reduced intervention performance as the expressive concept embeddings, which are prone to information leakage, seem to suboptimally adapt to the injected concept information. Modeling the concept distribution A cornerstone of SCBMs is the explicit, distributional parameterization of concepts. This helps in understanding the data correlations and allows for visualization, as the example seen in Figure 1 (c). The explicit probabilistic modeling results in improved concept uncertainty estimates compared to the baseline CBM counterparts, as shown in Table 3 , where lower metrics imply better estimates. This proves useful for interventions, where the uncertainty estimates can be leveraged for the choice of concepts to intervene on, improving the target prediction more effectively and reducing the need for manual user inspection. In Figure 3 , we compare the performance of randomly intervening versus intervening based on the predicted uncertainty. We observe that there is a big gap between the two policies, indicating the usefulness of the estimated probabilities. Nevertheless, note that intervening at random remains successful and supports the observations made in the previous paragraph, as shown in Appendix C.3.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: We disclose hyperparameters in the main text and Appendix. We also offer the code for reproducibility in case any information is missing.Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
open access to data and code: 
 Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes]Justification: We have released an anonymized version of the repository.Guidelines:• The answer NA means that paper does not include experiments requiring code. Justification: See Appendix Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
conclusion: 
 In this paper, we introduced SCBMs, a new concept-based method that models concept dependencies with a multivariate normal distribution. We proposed a novel, effective intervention strategy that takes concept correlations into account and is based on the confidence region inferred from the distributional parameterization. We showed that our modeling approach retains CBMs' training and inference speed, thus, being able to harness the benefits of end-to-end concept and target training. Additionally, the explicit parameterization offers the user a clearer understanding of the learned concept dependencies, providing deeper insights into how predictions and interventions are made. Empirically, we demonstrated that by modeling the concept dependencies, SCBMs offer a substantial improvement in intervention effectiveness, in concept as well as target accuracy, compared to related work. We showed that our method excels when iteratively intervening on the most uncertain concept predictions, sparing users from having to manually search through the concept set to identify necessary interventions. Additionally, our results indicate that learning the concept correlations does not decrease performance prior to interventions, in many cases even improving the performance over the baselines. Finally, the versatility of SCBMs is highlighted through their superior performance on CIFAR-10 and CIFAR-100, where concept values are CLIP-based rather than human-annotated.Limitations & Future Work This work opens multiple new research avenues. A natural extension is to go beyond binary concepts, such as continuous domains with their corresponding adaptations of modeling the concept distribution. Additionally, addressing the quadratic memory complexity of the covariance matrix is essential for scaling to larger concept sets. Our proposed intervention strategy accounts for model uncertainty, but further research is needed to accommodate user uncertainty, as human interventions are not always the ground truth. This work allows the editing of the learned dependency structure by adjusting the entries of the predicted covariance matrix, which could be explored. Lastly, to model additional information and reduce leakage, Koh et al. (2020) ; Havasi et al. (2022) propose the adoption of a side channel. The complementary effectiveness of incorporating the side channel in the covariance structure could be explored in the context of SCBMs.n = h (η n ) + ϵ n s.t. ϵ n ∼ N (0, I) for 1 ≤ n ≤ N .7. Let g : R C → R be a randomly initialized linear perceptron.8. Let y n = 1 {(g(cn)≥y med )} for 1 ≤ n ≤ N , where y med denotes the median of g (c n ). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: The code of ethics was followed.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: Given the more foundational work of this paper, there is not a direct negative influence that the authors can think of that might arise from this work specifically.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).11. Safeguards • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In this section, we provide additional details on the datasets that are being used in the experiments.Here, we describe the data-generating mechanism of the synthetic dataset in more detail. Let N , p, and C denote the number of independent data points {(x n , c n , y n )} N n=1 , covariates, and concepts, respectively. We set N = 50,000, p = 1,500, and C = 100, with a 60%-20%-20% train-validationtest split. The generative process is as follows:1. Randomly sample W ∈ R C×10 s.t. w i,j ∼ N (0, 1) for 1 ≤ i ≤ C and 1 ≤ j ≤ 10.Caltech-UCSD Birds-200-2011 We evaluate on the Caltech-UCSD Birds-200-2011 (CUB)2 dataset (Wah et al., 2011) . It comprises 11,788 photographs from 200 distinct bird species annotated with 312 concepts, such as belly color and pattern. In this manuscript, we follow the original train-test split and revised the proposed dataset in the initial CBM work (Koh et al., 2020) . Here, only the 112 most widespread binary attributes are included in the final dataset, and concepts are shared across samples in identical classes. The images were resized to a resolution of 224 × 224 pixels. Finally, following the original proposed augmentations, we applied random horizontal flips, modified the brightness and saturation, and applied normalization during training.CIFAR-10 CIFAR-103 (Krizhevsky et al., 2009 ) is a natural image benchmark with 60,000 32x32 colour images and 10 classes. We kept the original train-test split, with 50,000 samples in the train set and a balanced total of 6,000 images per class. We generated 143 concept labels as described in Section 4 using large language and vision models. At training time, as for CUB, we applied augmentations including modifications to brightness and saturation, random horizontal flips and normalisation. Images were rescaled to a size of 224 × 224 pixels.This section provides further implementation details of SCBM and the evaluated baselines. All methods were implemented using PyTorch (v 2.1.1) (Ansel et al., 2024) . All models are trained for 150 epochs for the synthetic and 300 epochs for the natural image datasets with the Adam optimizer (Kingma & Ba, 2015) with a learning rate of 10 -4 and a batch size of 64. For the independently trained autoregressive model, we split the training epochs into 2/3 for the concept predictor and 1/3 for the target predictor. For the methods requiring sampling, the number of Monte Carlo samples is set to M = 100. We provide an ablation for M = 10 in Appendix C.2. Note that since the predictor head is very simple, the MC sampling of SCBMs is extremely fast and does not influence computational complexity by more than 0.1%. For the synthetic tabular data, we use a fully connected neural network as backbone, with 3 non-linear layers, batch normalization, and dropout.For the CUB dataset, we use a pretrained ResNet-18 (He et al., 2016) , and for the lower-resolution CIFAR-10 a simple convolutional neural network with 2 convolutional layers followed by ReLU, Dropout, and a fully connected layer. For fairness in the comparisons, all baselines have the same model architecture choices and all experiments are performed over 10 random seeds.Resource Usage For the experiments of the main paper, we used a cluster of mostly GeForce RTX 2080s with 2 CPU workers. Over all methods, we estimate an average runtime of 8h per experiment, each running on a single GPU. This amounts to 5 methods × 3 datasets × 10 seeds × 8 hours = 1200 hours. Adding to that, the Ablation Figures required another 40 runs, amounting to a full total of 1520 hours of compute. Please note that we only report the numbers to generate the final results but not the development time, which we roughly estimate to be around 10 times bigger.In this section, we show additional experiments to provide a more in-depth understanding of SCBM's effectiveness. We ablate multiple hyperparameters to provide an understanding of how they influence the model performance, as well as show the performance of our model in other settings. We present the result on the CIFAR-100 dataset with 892 concepts obtained from Oikarinen et al. (2023) in Figure 4 to showcase the scalability of SCBMs. The results underline the efficiency of our method. Notably, the Autoregressive baseline has a negative dip, which is likely due to the independently trained target predictor not being aligned with the concept predictors in this noisy CLIP-annotated scenario. Note that they need to train independently to avoid the sequential MC sampling during training, which would otherwise increase training time significantly. Our jointly trained SCBMs do not have this issue and surpass the baselines. We use the same configuration as for CIFAR-10, with the exception that we set M = 10 to reduce the memory requirement.To showcase that SCBMs do not rely on a huge number of Monte Carlo samples, we provide an ablation of M in Figure 5 . It shows that even for M = 10, SCBMs thrive. Note, however, that since M is not a driving factor of SCBMs computational cost, one can leave it at a high number. In Figure 6 , we present the intervention performance of SCBM and baseline methods. Compared to the uncertainty-based intervention policy of Figure 2 , the intervention curves of all methods are less steep, confirming the usefulness of Shin et al. (2023) 's proposed policy. Following the previous statements, SCBMs still outperform baseline methods with the amortized beating the global variant for real-world datasets. We observe that in CIFAR-10 for the first interventions, an improvement in concept accuracy is not directly reflected in improved target prediction for SCBMs, which is likely due to the low signal-to-noise ratio of the CLIP-inferred concepts.In Figure 7 , we analyze the impact of the strength of λ 2 from Equation 6. Due to environmental considerations, we conducted experiments using only 5 seeds and limited the number of interventions to 20. Our findings indicate that SCBMs are not sensitive to the choice of λ 2 , except that the unregularized amortized variant exhibits slight patterns of overfitting. Figure 7 : Performance on CUB after intervening on concepts in the order of highest predicted uncertainty with differing regularization strengths. Concept and target accuracy (%) are shown in the first and second columns, respectively. Results are reported as averages and standard deviations of model performance across five seeds. For each SCBM variant, we choose a darker color, the higher the regularization strength of λ 2 .In Figure 8 , we analyze the effect of the intervention strategy. Our findings indicate that while SCBMs are still effective with the proposed strategy from Koh et al. (2020) , that sets the logits to the 5th (if c i = 0) or 95th (if c i = 1) percentile of the training distribution, our proposed strategy based on the confidence region results in stronger intervenability. In Figure 9 , we analyze the effect of the level 1 -α of the likelihood-based confidence region. Our findings indicate that the SCBMs are not sensitive to the choice of 1 -α, with higher levels being slightly better in performance. et al. (2024) propose to interpret the interpretation capacity of concepts with the Jaccard Index (Jaccard, 1901) . As such, in Table 4 , we extend Table 1 with this metric. It is evident that the interpretation does not change, indicating that the performance is robust to the choice of evaluation metric. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.The code is available here: https://github.com/mvandenhi/SCBM.https://www.vision.caltech.edu/datasets/cub_200_2011/, no license availablehttps://www.cs.toronto.edu/~kriz/cifar.html, no license available 


Decision: The paper introduces a novel Stochastic Concept Bottleneck Model (SCBM) that significantly enhances the functionality of Concept Bottleneck Models (CBMs) by modeling concept dependencies using a multivariate Gaussian distribution. This approach allows for more effective interventions where a change in one concept automatically adjusts related concepts, improving the model's overall interpretability and usability. Despite the complexity of learning the covariance matrix among concepts being a challenge, the authors effectively address this through detailed computational analyses and comparisons with baseline methods. They also commit to further clarifications in the final manuscript for sections that were difficult to understand, leading to a recommendation for acceptance.

Reviews: review: paper_summary: paper_summary: This paper presents a method of performing interventions on Concept Bottleneck Models. The method parametrizes the space of concepts with a generative model of Bernoulli distribution and concept logits with a normal distribution, whose mean and variance depends on the input data distribution. The method also compares amortized covariance, which is instance-based, and a global covariance, marginalized over all samples. Rather than doing the interventions one-by-one as in previous works, this method also  modifies a single concept by hand, which in turns modifies other related concepts using the covariance learned. Experiments demonstrate that the method performs competitively against related CBM works that also allow for intervention, and is much faster during inference. During intervention, the model performs competitively against other baselines.
strength_weakness: strength_weakness: While the overall writing of the paper is clear, Section 3.3 was a bit challenging to follow, especially the part about the confidence region. It is not entirely clear to me what the confidence region is of. As an example, L183-184 “the likelihood-based
confidence region provides a natural way of capturing the region of possible $\mathbf{\eta}’_{\mathcal{S}}$ that fulfil our desiderata”. The confidence of what? What desiderata? I suggesting either adding a diagram, a bit more background, a reminder, or an example, on what the intervention is trying to achieve so it more clearly demonstrates the process.
questions: questions: Often times we don’t have many labeled samples of fine-grain annotations (like CUB) or the annotation can be noisy (like CLIP annotations). To what extend do these two factors affect the learning of the distributions in your method? Is it true that if the global dependency structure is of poor quality (correlation matrix), then this can severely impact the downstream performance and interpretability?
limitations: limitations: A key part of this method is learning a good dependency matrix for each sample or globally. I would expect this requires a lot of samples and can severely impact the performance if the matrix is not representative of the relationships between concepts. This is clearly stated in the work, hence the limitations are sufficiently stated.
paper_summary: paper_summary: This paper introduces a novel concept dependency modelling scheme via an explicit distributional parameterization based on multivariate Gaussian distributions. This allows for capturing the dependencies between different concepts, while giving rise to an effective intervention strategy. 

The experimental results vouch for the efficacy of the proposed approach, which allows for end-to-end training, and exhibits on-par or improved performance before and after interventions.
strength_weakness: strength_weakness: Even though the proposed approach uses either an amortized or global  formulation, the complexity of learning the covariance matrix either way surely introduces a lot of complexity. This is especially true in cases where the number of concepts are large, a common occurrence in complex datasets with many classes such as CIFAR-100 and ImageNet (not explored in this work). This complexity is further burdened by the Monte Carlo sampling scheme; commonly, 1-10 samples are enough, but the authors report the usage of 100 MC samples which would greatly slow down the training process. 

In this context:
1) What is the complexity compared to a standard diagonal approach? 
2) It would be important to assess the complexity and performance in the case of larger datasets like ImageNet. The authors use the CLIP generated data of [1] for CIFAR-10, so applying the approach to CIFAR-100 and ImageNet should be quite easy. 
3) In my experience with MC-based methods, I find Table 2 a bit hard to believe, especially with 100 MC samples that the authors report. Can the authors provide wall time measurements for the per-epoch time for each method? Even for just Hard CBM and Amortized SCBM. 4) Having access to the code, so the experiments could be validated would also be important. 
5) How does the number of MC samples affect the complexity and performance? I find an ablation study to be necessary here.
6) How many GPUs did the utilized cluster have?

The considered Bernoulli formulation is very similar to the work in [2], which they call concept discovery. It constitutes a data-driven Bernoulli posterior for concept selection and the formulation is very similar. I find this method more appropriate compared to other approaches mentioned in the related work and the experimental evaluation, and some discussion/results should be included in the main text. 

In Table 1, the authors report the concept accuracy which I assume is based on a binary accuracy. Recent works have suggested that maybe this is not a good metric in the context of interpretability, since most concepts are sparse and this can lead to misleading results [3]. Can the authors report the Jaccard similarity between the ground truth and the obtained concepts for all the methods?

[1] Oikarinen, T., et al., Label-free concept bottleneck models, ICLR 2023

[2] Panousis, K. P., et al., Sparse Linear Concept Discovery Models, ICCV CLVL 2023 

[3] Panousis, K. P., et al., Coarse-to-Fine Concept Bottleneck Models, Arxiv 2024
questions: questions: Please see the Weaknesses section.
limitations: limitations: The authors mention the limitations in the dedicated section. The main limitation of the proposed approach is complexity.
paper_summary: paper_summary: Focusing on concept bottleneck models, this paper extracts concept dependencies with a multivariate normal distribution and derives an intervention strategy based on the confidence region of the normal distribution that incorporates concept correlations for better interventional effectiveness.
strength_weakness: strength_weakness: 1. Experiment results show marginal improvement over existing methods.
2. The novelty and contribution of this paper is limited.
questions: questions: 1. Intervention by setting values can be difficult in some case, the example given by the paper: "intervene on the CBM by setting its value to 1" is an extreme case. Humans are not good at estimating probabilities.
2. "do not use the intervened-on concepts to update their remaining concept predictions" may not necessarily be a bad thing. As in many real-world cases, there are exceptions to rules and patterns. Nothing is absolute. Universally "extend the concept predictions with the modeling of their dependencies" may be problematic. Are these exceptions handled by confidence region?
3. What is the novelty comparing with E. Kim et al. (2023)? just relax the assumption of diagonal covariance matrix?
4. Will the proposed method suffer leakage?
limitations: limitations: Limitations are adequately addressed.

Comments: comment: I would like to thank the authors for their thorough rebuttal. I really appreciate the additional experiments using CIFAR-100 and the consideration of the proposed metric. ImageNet would also be important to evaluate at some point since it commonly considers a very large amount of concepts and it would be interesting to observe the complexity and the behavior of the proposed method. 

After the provided clarifications, I also agree that the proposed approach is complementary to Panousis et al. 2023 and I believe it would be a good probabilistic combination. To this end, and after reading the comments and the discussions with the other reviewers I decided to increase my score.
Thanks for the authors' response, which clarified some points.
Thank you for the author's detailed response. The improved writing improved my understanding of the work much better.
Dear reviewer 9xQm,

please let us know if we could address your concerns with our rebuttal or if there are other open questions remaining. We would be grateful if the reviewer could acknowledge the rebuttal. Thank you in advance!
Dear reviewer sfUo,

please let us know if we could address your concerns with our rebuttal or if there are other open questions remaining. We would be grateful if the reviewer could acknowledge the rebuttal. Thank you in advance!
Dear reviewer rLqE,

please let us know if we could address your concerns with our rebuttal or if there are other open questions remaining. We would be grateful if the reviewer could acknowledge the rebuttal. Thank you in advance!