Concepts' Information Bottleneck Models
Authors: Karim Galliamov
Syed M Ahsan Kazmi
Adil Khan
Adín Ramírez Rivera
Venue: ICLR 2026 Poster (2026)
Keywords: Concept bottleneck models
Information bottleneck
Variational Inference
Primary area: interpretability and explainable AI
TL;DR: Enhances Concept Bottleneck Models by integrating the Information Bottleneck principle to reduce concept leakage and improve performance

Abstract: Concept Bottleneck Models (CBMs) aim to deliver interpretable predictions by routing decisions through a human-understandable concept layer, yet they often suffer reduced accuracy and concept leakage that undermines faithfulness. We introduce an explicit Information Bottleneck regularizer on the concept layer that penalizes $I(X;C)$ while preserving task-relevant information in $I(C;Y)$, encouraging minimal-sufficient concept representations. We derive two practical variants (a variational objective and an entropy-based surrogate) and integrate them into standard CBM training without architectural changes or additional supervision. Evaluated across six CBM families and three benchmarks, the IB-regularized models consistently outperform their vanilla counterparts. Information-plane analyses further corroborate the intended behavior. These results indicate that enforcing a minimal-sufficient concept bottleneck improves both predictive performance and the reliability of concept-level interventions. The proposed regularizer offers a theoretic-grounded, architecture-agnostic path to more faithful and intervenable CBMs, resolving prior evaluation inconsistencies by aligning training protocols and demonstrating robust gains across model families and datasets.

Published as a conference paper at ICLR 2026
CONCEPTS' INFORMATION BOTTLENECK MODELS
Karim Galliamov
Syed M Ahsan Kazmi²
Adil Khan
Adín Ramírez Rivera?
University of Amsterdam University of West of England
University of Hull
*University of Oslo
"eeiim wew.gggllleiettteeommmm uva.n1
"ahsan.kazmii kazmi 9 uwe.
a.m. whanfhullll hu21 aan.l..c....
"adinrBuio..
ABSTRACT
Concept Bottleneck Models (CBMs) aim deliver interpretable predictions by routing decisions through nunan-understandable concept layer, yet they often suffer reduced accuracy and concept leakage that undermines faithfulness. We introduce explicit Information Bottleneck regularizer the concept layer that penalizes (X; C*) while preserving task-relevant information in / (C;; Y), encour- aging minimal-sufficien concept representations. We derive two practical variants (a variational objective and an sutropy-baseds surrogate) and integrate them into standard CBM training without architectural changes or additional supervision. Evaluated across six CBM families and three benchmarks, the IB-regularized mod- els consistently outperform their vanilla counterparts. Information-plane analyses further corroborate the intended behavior. These results indicate that enforcing a concept bottleneck improves both predictive performance and the reliability of concept-level interventions. The proposed regularizer offers a theoretic- theoretic-roounded, architecture-agnoss path to more faithful and intervenable CBMs, resolving prior evaluation inconsistencies by aligning training protocols and demonstrating robust gains across model families and datasets.
INTRODUCTION
In many real-world settings, models must not only be accurate but also provide explanations that humans can scrutinize and act upon. Among explainability approaches, self-explainable models are particularly compelling because they produce structured, intrinsic rationales and facilitate targeted debugging. Concept bottleneck models (CBMs) (Koh et al., 2020) embody this paradigm by predicting through human -intrannitterretabll concepts and enabling test-time interventions those concepts.
Despite their appeal, CBMs often underperform black-box models and suffer from concept leakage (the encoding of extraneous input information into concept activations) which undermines inter- pretability and weakens intervention reliability (Mahinpei et al., 2021; Margeloiu et al., 2021). Prior efforts to alleviate these issues typically modify architectures enrich concept embeddings (Havasi et al., 2022; Kim et al., 2023), which introduces design complexity, can reduce intervenability, and may not generalize across CBM variants.
We propose a simple, architecture-agnot Information Bottleneck (IB) regularizer directly on the concept layer (Alemi et al., 2017: Tishby et al., 2000). Our objective enforces minimal-sufficient concepts by compressing ((X; C) while preserving I(CYY), thereby discouraging spurious signal flow into concepts without sacrificing task-relevant information as shown in Fig. 1. We derive two practical training objectives, including an entropy-baad based surrogate that replaces the explicit I(X:C) term with an equivalent formulation in terms of entropies. We empirically justify its simplifying assumption on the concept entropy and show it performs on par with a variational bound. The regularizer and-aay--lay: it requires no architectural changes, side channels, additional supervision, and applies broadly across CBM families.
solution grounded in information theory: impose an
For our evaluation, first, we follow intra-method protocol: for each baseline, compare it against its proposed regularized variant under identical backbones, data, and training recipes. Second, we sub- stantiate interpretability improvements beyond accuracy: IB IB-regularized CBMs exhibit substantially lower leakage, stronger and more predictable test-time interventions, and clearer information plane dynamics consistent with the minimal-sufficiency goal. Third, we show stable performance across

---

Published as a conference paper at ICLR 2026
(2(20)
p(z [x)
g(c =)
<iy c)
1. ((()
Figure 1: Our proposed CIBMs pipeline. The image is encoded through p(2 z). which in turn encodes the concepts with g(c| 2), and the labels are predicted through 7(V c). These modules are implemented as neural regularization as mutual information opti-
Figure 2: Our generative model ply x)p(c] p))p2 (=)p(z) (solid lines), and its variational approxi- mation g(v) () (=)))(()(=(()))) ())))))))
networks. We introduced mizations over the variables as shown in dashed lines.
(dashed lines).
a range of IB strengths. These experiments indicate that IB at the concept layer improves both predictive performance and the reliability of concept-level reasoning.
Our contributions are: (i) We establish, to our knowledge, the first direct integration of the Information Bottleneck principle at the concept layer in CBMs, reframing interpretability from heuristic add- on into a structurally enforced, theoretically grounded property of the model. We provide two concrete, tractable training objectives with empirical support for their assumptions. (ii) We present a comprehensive, reproducible assessment across diverse CBM variants and datasets showing that IB-regularized CBMs consistently improve target accuracy, reduce concept leakage, and enhance intervention effectiveness.
Rather than redesigning CBM architectures accumulating concept embeddings (Havasi et al., 2022; Kim et al., 2023), our information-theorrtic regularization offers a simple, general mechanism to obtain concepts that are both maximally predictive of the target and minimally contam- inated by the input-yielding more faithful, intervenable, and performant concept- models.
2 RELATED WORK
2.1 CONCEPT BOTTLENECK MODELS
CBMs. A CBM (Koh et al., 2020) predicts via 0 fg(()), where x E g: IR - IR maps inputs to K human- human-interpree interpreetaeeeeee concepts, and f: IR K - RR maps concepts the target. Training uses triplets (1,, Ct, 31) J with ground-truth concepts c, and can proceed independently, sequentially, or jointly (Koh et al., 2020). While CBMs enable reasoning and interventions through concepts, they often trail black-box models in accuracy. Concept Embedding Models (CEM) (Espinosa Zarlenga et al., 2022) improve accuracy by learning "active" and "inactive" embeddings per concept, but require extra regularization for effective test-time interventions and exhibit increasing T(X; C) without compression. In contrast, we retain the native concept space and explicitly regularize it with concept information bottleneck, incorporating infurl--nformaaiio constraint into the loss; this plug-in regularizer is architecture-agnosicc and applicable across CBM variants.
Probabilistic CBMs. ProbCBMs (Kim al., 2023) and energy-based CBMs (Xu et al., 2024) model concept uncertainty by predicting distributions over concepts and often rely on anchor points for class mapping. Other works extract concepts without annotations by introducing inductive biases, e.g., language or label-free supervision (Oikarinen et al., 2023; Yang et al., 2023). We avoid anchor points (since they increase inference cost and introduce additional tuning hyperparameters) and instead learn concepts using a variational approximation to our concept-level information bottleneck.
Post-hoc CBMs. Another line of work converts pre-trained models into CBMs. Side-channel and recurrent CBMs (Havasi et al., 2022) introduce, respectively, parallel concept bottleneck and sequential concept prediction; the former lowers intervenability, while the latter compromises concept disentanglement. Post-hoc CBMs (Yuksekgonul al., 2023) train concept and class predictors frozen penultimate-layer embeddings and typically require residual connections for competitive accuracy, leaving residual information paths that concept supervision cannot shape undermining both interpretability and intervention reliability.
Concept Leakage. CBMs are prone to concept leakage (i.e., encoding input information in concept activations beyond intended semantics) across both soft and hard concepts (Mahinpei al., 2021; Margeloiu et al., 2021). Margeloiu al. (2021) report that CBM desiderata are best met
2

---

Published as a conference paper at ICLR 2026
under independent training, while joint and sequential training tend to encode extra input information; saliency analyses further suggest that, under all three regimes, concepts are often not grounded in meaningful input features. To quantify leakage, Oracle and Niche Impurity Scores were proposed by Espinosa Zarlenga et al. (2023a). Motivated by these findings, we posit that compressing T(x; C) while preserving T(C; Y) yields more faithful concepts and representations; our experiments across diverse tasks support this hypothesis.
2.2 INFORMATION BOTTLENECK
Tishby et (2000) introduced the information bottleneck (IB) as the minimization of the functional LI T(X: Z) BT(Z; where ((::)) is the mutual information, / is the Lagrange multiplier, X, Y and Z are random variables that represents the data, labels, and latent representations, respec- tively. This functional seeks representation Z that is maximally predictive of Y while díscarding irrelevant information about X. This "minimal-sufficii the memorization compreesioonooeiiininooooooooon dynamics, wherein (Z; Y) grows throughout training while [(X; Z) first increases (memorization) and later decreases (compression).
principle connects to generalization via
Alemi et al. (2017) extended the IB framework deep neural networks by doing variational approximation of latent representation. And, Kawaguchi et al. (2023) analyzed the role of IB in estimation of generalization gaps for classification task. Their result implies that by incorporating the IB into learning objective one may get more generalized and robust network. Unlike prior work that applies IB generic latent features, we target the concept layer in CBMs. To our knowledge, we are the first to formalize and integrate IB at the concept level in CBMs: we introduce the concept variable C and explicitly regularize I (X; C`) while preserving I(C;Y), yielding concept representations that are both compact and task-relevant. We derive tractable upper bound that links predicted concepts and ground-truth signals into training regularizer that enforces the desired behavior at the concept layer. Practically, this yields objectives realizable via a variational approximation and, alternatively, a mutual-information estimator, enabling seamless integration across common CBM formulations without architectural changes.
3 CONCEPTS' INFORMATION BOTTLENECK
Concept Bottleneck Models (CBMs) aim for high interpretability by introducing human- understandable concepts, C, as an intermediary between latent representations, Z, and the labels Y. so that the overall pipeline is X + Z - C + Y-cf. Fig. 2. To keep the concepts truly interpretable we must prevent them from encoding irrelevant details of the raw input X. This is captured by minimizing the mutual information between the inputs and the concepts, I(X; C), which directly limits the amount of data that can "leak" into C. At the same time we want the concepts to be predictive of the label and be well-supported by the latent representation. Accordingly, we maximize (C;Y) and [(2; C*.. Our initial objective is max / (Z; C) + +(C; s.t. [(x; 2) S. Ic. where le is an information constraint constant, that equivalently is the maximization of the functional of the concepts' information bottleneck (CIB)
CCB = + -I;;
+
(1)
where / is Lagrangian multiplier. This formulation ensures strong connection between latents, Z, and the concepts, C. This means that one wants Z to be maximally useful in shaping the concepts C, while also ensuring that the concepts are informative about the target.
While the data processing inequality, (X;C) S I(X; suggests that compressing the latent space Z implicitly limits the information in C, we propose enforcing this constraint directly on the concept space. We treat this not merely as bound, but as a design choice. By explicitly minimizing [[X; C.., we strictly control the amount of source information retained in the concepts, regardless of the capacity of Z. This prioritizes the cleanliness of the interpretable layer, leading to the objective
LCIM = I(Z;
C).
(2)
Note that one can obtain the same loss if the optimization problem is constrained over the concepts instead, i.e., max T(Z: C)= I(C:Y) s.t. T(X:C) < Tc. Nevertheless, we present the relation with the traditional compression for completeness.
3

---

Published as a conference paper at ICLR 2026
We posit that by compressing the information between the data, X, and the concepts, C, instead of the latent representations, Z. we can control the redundant information of the data within the concepts. By limiting [X; C"), we explicitly control the amount of redundant spurious information that can enter the concepts. In contrast, compressing leave pathway for such leakage to survive the Z - C' mapping. Moreover, the bound LCIMM (2) can be interpreted as compressing the information between X and C' directly. This provides stronger interpretability constraint than limiting / (X; Z). yielding a more robust compression and, consequently, more faithful concepts.
+ Z first and then deriving C' from Z may
By shifting the compression from the latent space the concept space we obtain (i) concepts that are intrinsically concise and, thus, easier for a human to interpret, and (ii) a principled safeguard against data leakage that would otherwise contaminate the bottleneck. The CIBM formulation (2) provides a clean cooomationntthoreee framework for building such interpretable CBMss We propose two implementations of our framework by exploring different ways of solving the mutual information based on variational approximation of the data distribution.
Our CIBMs provide a tighter generalization bound than traditional CBMs, established through a PAC-Bayes framework (McAllester, 1999; 2003). The derived generalization gap (A) reveals the theoretical advantage of the CIBM over the traditional CBM. The result (Theorem 2) demonstrates that the guaranteed True Risk of the CIBM is strictly lower than that of the CBM, provided the regularization strength /3 is sufficiently small to ensure the gap is positive (A > 0). This occurs because the CIBM's complexity reduction significantly outweighs the resulting slight increase in empirical training eтror (i.e., the 3 penalty), leading to a tighter generalization bound. We detail the derivation and analysis of the conditions for this gap in Appendix B.
3.1 BOUNDED CIB
We consider the bound the concept bottleneck loss (2) in terms of the entropy- based definitions of the mutual information. By using variational approximation of the data distribution, we bound it as
LCIB 2(1 - 3) [M(ee =)) - (w(cl 2), g( 2))] p(z)
E H (p(y (), gly c)), (3)
p(e)
LS-CIBM.
We detail this derivation in Appendix A. We maximize the concepts' information bottleneck by minimizing the cross entropies of the predictive variables, and and their corresponding ground truths and by adjusting the entropy of the concepts-ef ef. Fig. 2. We denote the models that were trained using this loss Ls-CIBM (4) by lBg. To implement it, we need to estimate the entropy of the concepts distribution p(c). We give details of this estimator in Appendix D.4.
3.2 ESTIMATOR HHAMAD CIB
Another way obtain bound over the concept information bottleneck (2) is to only expand the conditional entropies that are not marginalized (A. I) to avoid widening the gap in the bound, i.e.,
LCIM (YY + (CC) - p(e)
(p(y c), g(y c)) - p())
H (p(c) 2), g(c| =)) - -I(X (5)
we treat the entropies of the concepts and the labels as constants, we obtain
CE-CIB = H (p(y c). 9(9 c)) + EHH H (p(c g(c 2)) - (p I(X:C))
(6)
p(c)
p(=)
where is constant. Moreover, we found empirical evidence that also supports our assumption which we discuss in Appendix C. We denote the models that use this loss as IB g since it relies on the estimator of the mutual information. We detail the estimator we used in our implementation in Appendix D.4.
It is interesting to highlight that other optimization approaches emerge out of our bound. For instance, Kawaguchi et al. (2023) proposed Lx Ey(s) H (p(v] 2), q(v =)) + 3(p I(Z; X)). Our mutual information estimated loss (6) resembles that proposal with the corresponding conditioning changes in the labels and the concepts. In contrast, our proposal is a generalized framework that encompass wide range of possible implementations.
4

---

Published as a conference paper at ICLR 2026
Comparison between our estimators. Unlike Ly-CISM (4), which simplifies the mutual information terms into cross-entropy losses, Lr. cm retains explicit control over C();C). This allows for more granular control over the information flow from inputs to concepts, leading to tighter constraint on concept leakage. As we show in the results (Table 1), this additional control translates to im- proved performance in both concept and class prediction aceuracy, cf. Section 4.
We report accuracy results (over runs) of our proposed regularizers, IBg and IBg, applied to different CBMs. Black-box is gold standard for class prediction that offers no explainability over the concepts.
Table 1:
Method
Concept
Class
CUB
Black-box
0.919+00.002
CBM (HJ)
0.956:10.001
0.650:10.002
CBM (HJ) + IBg
0.951=00000
0.654:0.000
CBM (HJ) + IBg
0.951=00000
0.655=00.000
CBM (HI)
0.956:20.011
0.644=0.001
4 EXPERIMENTS
CBM (HI) + 1B
0.954:20.000
0.689:10.000
CBM (HI) + IBs
0,952=00.000
0.691 :000
CBM (SJ)
0.956.10.001
0,708 0,708.10.006
We extend several CBM variants with our IB-regularizers, yielding CIBMs. The CIBMs are slight variations of the original models as they require a variational approximation in order to study and apply the proposed IB-regularizers. We train each model from scratch and compare CIBMs to their vanilla counterparts of equal capacity, measuring both dlassppreiittin accuracy and concept leakage. Our goal is to close the accuracy gap to black-box models without sac- rificing interpretability or test-time intervenability. Finally, we analyze information flows via timates and benchmark intervention performance.
CBM (SJ) + IB
0.961=000005
0.000
CBM (5J) + IBg
0.958=00.005
0,726++0.000
CBM (SI) CBM (SD) + IBar
0.956:10.001
0.639=0.000
0.00==0.002
0.643++0.000
CBM (SD) + IBg
0,958=00000
0.646=#0.000 0.00
ProbCBM
0.956=00.001
0.718::0.005
ProbCBM + 1B
0.960=20.000
0,741=10.006
ProbCBM + 1Bg
0,962:00.000
0,734 002
InCEM
0,954.10.001
0.759.10.002
InCCEM +Bg
0.957=000000
0.778::0.000
es-
InCEM + 1Bg
0.959=00.000
0.000
AR- AR-CBM
0.956:20.002
0.761=10.010
AR-CBM + 1B
0.961=0.000
0,777 0.08
We benchmark our approach on three datasets: CUB (Wah
AR-CBM + 1Bg
0.950=000000
0.774=000006
et al., 201 1), Aw (Xian et al., 2019), and aPY (Farhadi et al., 2009). We present all implementation details in Appendix D. For our regularizers, we evaluate their setups and select the best hyperparameters (ef. Section 4.6). In the following experiments, we use the same hyperparameters and setup for our regularizers for fair comparisons.
AwA2
Black-hos hox
0,893-40.000
CBM (HT)
0.979=000000
0.853-+0.002
CBM (HT) + IB
0.968=00000
0,845 0.0
CBM (HI) + IB,
0.970.10.000
0.85610.000
CBM (HI)
0.979=0.000
0.836=0.001
CBM (HI) + IB
0.966=+0.000
0,823-+00000 0.0000
CBM (HT) IB
0.976=0.000
0,834++0.000 000
CBM (SJ)
0.979-t0.001
0.876=0.001
4.1 PERFORMANCE ACROSS ALL DATASETS
CBM (SJ) IB
0.978=0.003
0,878:+0.000
CBM (SJ) IB
0.980=00.000
0.874 0-0.02
We present the evaluation results on three datasets in Ta- ble 1. Our "black-box model" serves as hold-standard: it represents the highest possible class accuracy that can be achieved by CBM when it is trained only to predict class labels (i.e., without providing any explanations). Because these models are seminal, we compare against hard (H)
CBM (SI)
0.979=0.000
0.852=+0.004
CBM (SD) + IB
0.967=000000
0,847-+0 005
CBM (SD) +0 IB
0.969-10.000
0.853.10.000
ProbCBM
0.979=0.000
0 003
ProhCBM + 1Bg
0.971+10.001
0.883-100000 +-0.00
ProhCBM + 18
0.973++0.000
0,883-+0.00 0.000
InCCEM
0.979=000000
0.884++0.002 002
and soft (S) CBMs trained jointly (J) or independently (I)
InCEM IBg
0.977=0.000
0,8844-0.00 000
InCEM IB2
0.982:10.000
0.886110.003
(Havasi et al., 2022). We discuss their reproducibility and training stability in Appendices D.2 and D.3, respectively. We also compare against more recent CBM variants such as ProbCBMs (Kim et al., 2023), intervention-awarc CEM (InCCEM) (Espinosa Zarlenga et al., 2023b), and AR-CBM (Havasi et al., 2022).
AR-CBM
0.979=000001
0,884::0.000
AR- AR-CBM + 18
0.977=40.004
0.876-+-000m
AR AR-CBM- + 1B
0.978:20.004
0.879110.000
aPY
Black box.
0.866110.003
CBM (SJ)
0.967:10.00
0.707
CBM (SJ) IB
0.966=0.000
0.850=10.000 0.00
Our main objective is to show that the proposed regulariz- ers (IB and (Bx) maintain or improve parget-prediction accuracy relative to their original counterparts, while si- multancously enhancing concept-predee prediction accuracy and reducing concept leakage. The latter is crucial for guaran- teeing the explainability of the results.
CBM (SJ)+ IB
0.968=0.000
0,860:10.000 006
CBM (SD)
0.96700.000
0.702
CBM (SI) IB
0.966:10.000
0.845.10.002
CBM (SD) IB
0.957=0.000
0.848:1:0.00 000
ProhCBM
0.967=+0.000
0.863++0.007
ProbCBM BBg
0.960=20.000
0.873:10.005
ProbCBM + 1Bg
0.958:10.002
0.868:10.000
InCEMM
0.967 000000
0.86910.004
On the CUB dataset, both BBg and BB improve class- prediction accuracy over all baselines and always yield better class performance. These gains are accompanied by higher (or, in the worst case, comparable) concept
InCEM lB
0.966:10.000
0.868-10.000
InCEM 1Bg
0.971=0.000
0.877=00.002
AR-CBM
0.967:00000
0.873++0.000 004
AR- AR-CBM + lB
0.963-10.003
0.871 0.006
AR-CBM + IB
0.968=0.000
0.879=000077
5

---

Published as a conference paper at ICLR 2026
accuracy, thereby, achieving the fundamental goal of approach: to boost performance and interpretability at the same time. The CBM families (Havasi et al., 2022) were trained for 100 epochs, in line with current training protocols (Espinosa Zarlenga et al., 2022; Kim et al., 2023); reproducibility details are provided in Appendix D.2.
For the AwA2 dataset, the increase in class accuracy is modest but still comparable that of the original methods, and concept prediction remains on par with the unregularized models. We attribute this to the relative simplicity of the dataset, which leaves little room for substantial improvement.
In the more diverse, real-world aPY dataset, our regularizers significantly outperform the baseline CBMs in class accuracy. Remarkably, they even surpass the black-box model while delivering interpretability comparable to the original CBMs-an an essential property for real-world applications where explanations are required.
The overall rise in both class and concept accuracy relative to existing methods highlights the benefits of our -nntuallifformation regularization. By curbing concept leakage, the regularizers ensure that concepts are both informative and tightly coupled to the final prediction (see Section 4.2 for details). This empirical finding aligns with our theoretical framework, which argues that controlling the information flow between inputs and concepts via the Information Bottleneck produces more interpretable and meaningfully useful concepts without sacrificing performance (see Section 4.5 for deeper discussion).
4.2 CONCEPT LEAKAGE
Concept leakage occurs when spurious or task-irrelevant information contaminates concept activa- tions, thereby eroding both interpretability and the effectiveness of test-time interventions (Mahinpei et al., 2021; Margeloiu et al., 2021). Espinosa Zarlenga al. (2023a) introduced two complementary metrics for quantifying this phenomenon: the Oracle Impurity Score (OIS), which measures impurity localized within individual concepts, and the Niche Impurity Score (NIS), which captures impurity distributed across the whole set of learned concepts.
We evaluate OIS and NIS under three different settings: (i) Complete concept set: all concepts retained. (ii) Selective dropout: we remove the most predictive half of the concepts (i.e., the half with the highest class-prediction importance). This scenario has unique configuration, report a standard deviation. (iii) Random dropout: we randomly discard half of the concepts, serving as control condition.
do not
We chose dropout experiments because omitting relevant concepts can dramatically increase concept leakage (Havasi et al., 2022). Tables 2, G.I, and G.2 present the results. In every scenario our IB-regularizers (IB g and IBg) reduce leakage substantially, achieving the lowest OIS and NIS scores even when a large fraction of concepts is removed. These findings confirm that imposing an information bottleneck on the concepts effectively curtails spurious encoding and mitigates concept leakage.
4.3 INTERVENTIONS
key advantage of Concept Bottleneck Models (CBMs) is the possibility of test-time interventions: a user can overwrite predicted concepts with their ground-truth values and, thereby, improve the final decision. To evaluate this capability we simulate interventions by replacing the model's predicted concepts with the corresponding ground-truth concepts at test time.
Following prior work (Kim al., 2023; Koh et al., 2020), we intervene on groups of concepts rather than individual concepts. Grouping lets us measure how cumulative corrections affect dlassppredittion performance and mirrors the practical scenario where user corrects several related concepts at once. To intervene, concept groups sampled uniformly at random; the procedure is repeated five times and the results are averaged to reduce variance. For each number of intervened groups we compute the classification accuracy and plot it against the number of groups. The curve visualizes how quickly performance improves as more concept groups are corrected.
Figure 3 shows that CBMs regularized with our information information-bottleeedd bottleneck penalties (IB and IBg) exhibit a monotonic rise in accuracy as each additional concept group is intervened. This smooth ascent demonstrates that the models truly leverage accurate concept information and suffer minimal

---

Published as a conference paper at ICLR 2026
Table 2: Concept leakage evaluation (lower better) CUB.
Complete CS
Selective Drop-out CS
Random Drop-out CS
Model
OIS
NIS
OIS
NIS
OIS
NIS
CBM (HJ)
4.31 + 0.57
65.03 + 1.84
15.71
77.02
13.10 + 1.20
73.09 + 2.12
CBM (HJ) +IB
= 0.65
60.02 + 1.28
12.67
70.86
10.32 + 0.57
70.38 2.12
CBM (HJ) +IBg
2.56 = 0.47
59.52 = 1.91
12.47
71.09
10.36 - 0.80
69.95 = 1.35
CBM (HI)
3.77 = 0.58
05.15 + 2.03
14.27
74.16
12.31 1.04
71.78 = 1.86
CBM (HI) +IBB
3.58 + 0.76
64.95 + 1.59
14.01
73.16
12.63 = 1.31
70.89 - 2.04
CBM (HI) +IB
3.69 + 0.71
64.89 = 1.51
14.00
73.13
12.60 + 1.29
70.93 2.09
CBM (SJ)
4.69 = 0.43
66.25 2.31
16.29
78.39
12.97 = 0.78
74.19 = 1.04
CBM (SJ) +
2.01 = 0.07
62.32 + 1.95
13.26
72.80
10.95 + 1.49
71.94 = 0.98
CBM (SJ) F
2.00 = 0.09
62.31 + 2.08
13.23
72.85
11.03 + 1.38
72.03 + 1.07
IntCHM
8.74 + 0.30
75.41 + 3.83
20.85
80.19
18.31 + 0.09
76.56 + 2.00
InICEM +IBB
6.15 = 0.20
69.96 + 2.22
17.22
76.67
14.07 + 0.43
72.60 = 2.40
InCCEM +IBg
6.07 + 0.23
71.28 + 1.87
18.04
75.75
13.98 + 0.29
73.89 = 1.99
AR-CBM
3.90 + 0.27
62.30 + 1.52
14.16
63.40
12.58 + 0.86
60.86 + 1.32
AR-CBM + IBg
2.24
59.84 + 1.48
10.96
59.66
10.17 = 0.48
56.31 = 1.33
AR-CBM lBg
2.72 0.16
59.96 + 0.98
11.09
60.77
11.01 + 1.05
57.23 = 0.79
BM
4.30 0.10
64.22 + 1.04
16.01
76.92
13.81 = 0.21
75.01 = 0.86
ProbCBM 1B
2.43 = 0.38
60.29
2.02
13.04
72.76
10.29 + 0.50
70.86 1.87
ProbCBM + IB
2.57 + 0.52
59.81 + 1.74
13.05
73.08
11.15 + 0.62
71.13 = 2.09
CUB
AwA2
1.0
1.0
NR
IB
18g
111 Performance
0.9
0.9
CBM (II))
CBM (SJ)
0.8
0.8
InCEMM
AR-CRM
0.7
0.7
0.6
0.6
8
12
16
20
24
0
2
8
10
12
14
Number of intervened groups
Number of intervened groups
Figure 3: Change target prediction accuracy after intervening on concept groups following the random strategy as described in Section 4.3. (TTI stands for Test-Time Interventions, and NR for non-regularized.) We show expanded plots, with less clutter, in Fig. F.I.
leakage: every intervention yields consistent performance boost. In contrast, soft-joint CBMs display pronounced dips in the middle of the sequence, which we attribute to their more leaky representations that become unstable under random group corrections. Hard CBMs, which use binary concept slots, eventually reach high accuracy when large fraction of concepts is intervened (their low leakage helps), but they start far below the CIBMs and improve much more slowly when only a few concepts are correctedespecially on coarser datasets such as AwA2.
Overall, our IB-regularised models combine low-leakage encodings with adaptive flexibility, pro- ducing steady gains and outperforming every CBM variant both in the intervention curves and in the overall accuracy reported in Table 1, Full experimental details are provided Appendix I. Interestingly, the recent methods IntCEM and AR-CBM benefit the most from our regularization, showing a marked improvement on both datasets.
4.4 CONCEPT SET GOODNESS MEASURE
In CBMs, the quality of the concept set is crucial for accurate downstream task predictions. However, there is lack of effective metrics to reliably assess concept set goodness. Existing metrics, such as the Concept Alignment Score, proposed by Espinosa Zarlenga et al. (2022), evaluate whether the model has captured meaningful concept representations but do not explicitly measure how well these concepts improve downstream task performance during interventions. Moreover, this metric is tuned for CEM and do not extend beyond it.
7

---

Published as a conference paper at ICLR 2026
Table 3: Change in interventions performance with concept set corruption for CBM (SJ) and its regularized versions with our proposed methods. We show the disaggregated plots Fig. F.2.
CUB
AUC
NAUC
Corrupt
CBM
+
+IB
CBM
+IBg
+IBg
0
54.374
65,5886
64.6338
0.001260
0.0691
0.0035
4
53,135
64,4933
63.4168
0.001 198
0.0967
0.0269
8
51.291
53.1240
60.1120
0.001166
0.0758
0.0608
16
50.694
60.2241
59.4141
0.001068
0.0399
0.0451
32
46.101
$2.8855
51.1966
0.000863
0.0045
0.0432
64
32,069
30.5203
29.2557
0.000339
0.0985
0.0513
AwA2
AUC
NAUC
Corrupt
CBM
+IB
+IBg
CBM
+IB
+IBg
No
84.753
91.5144
92.1625
0.002808
0.0030
0.0115
Yes
83,985
90.5738
90.8121
0.004484
0.0023
0,0218
Similar to previous methods that rely on area under the curve for the interventions (Espinosa Zarlenga et al., 2023b; Singhi et al., 2024), we measure and compare the concept quality in CIBMs using the following metrics: area under interventions curve, and the area under curve of relative improvements. Denote by I(x) the model's performance for concept groups used in the intervention. Then the Test-Time Time Interventions (TTI) accuracy is
AUC = I(t),
1=1
and the normalized version of the TTI accuracy is
NAUCTTI Σ (I(I) 1))-
The idea behind these measures simple: if a concept set is of high quality, the task accuracy will steadily approach 100% more concept groups are intervened upon, resulting large area under the curve. Conversely, if the concept set is incomplete or noisy, performance gains will be limited, even with multiple interventions, which can indicate concept leakage.
The latter expression (8) could be simplified to just scaled difference between a model with full concept set used for interventions and performance of a model with no interventions, however, the meaning it has is how much does the performance change per one group added to the interventions pool. To test this, we generate corrupted concept sets by replacing selected concepts with noisy ones. Importantly, we maintain the original groupings of concepts.
Table 3 shows the results of our metrics. We also show the commonly reported disaggregated plots in Fig. F.2 The number in the "corrupt" column denotes the number of concepts replaced with random ones for CUB, and for AwA2 "No" denotes clear concept set and "Yes" denotes a concept set with one concept changed to corrupt. As expected, performance drops with corrupt concepts, since they contain no useful information for the target task. One consequence of our training is that if one has two concept annotations for some dataset, then is possible which concept set is better.
CIBMs performance determine
Our results demonstrate that regularizing with IB is more sensitive to concept quality compared to vanilla CBM, making it a better indicator of concept set reliability. Negative values in normalized intervention AUC indicate possible concept leakage.
4.5 INFORMATION PLANE DYNAMICS IN CBMS AND CIBMS
To further evaluate the proposed regularizers, we examined the information plane dynamics of CBM, IntCEM, and AR-CBM, shown in Fig. H.1. In general, we expected to observe higher mutual information between the concepts and the labels, (C;Y), and between the latents and the concepts, T(7; C), while expecting lower mutual information between the data and the concepts, T(X;
8

---

Published as a conference paper at ICLR 2026
and between the data and the latents, I(X; 2). We clearly observed this behavior when applying our 1 to IntCEM, and to a lesser degree with lBg. This pattern was also evident in AR-CBMs, although with more noise. However, in certain cases, this pattern deviated. More specifically, found that CBMs exhibit greater compression with respect to the data compared to their regularized counterparts. Nevertheless, our CIBMs demonstrate greater expressiveness due their higher mutual information with respect to the labels, Y.
We think that vanilla CBMs "over-compress" their internal representations-
((;;C)
and [x; 2) so aggressively that they discard useful, task-relevant features. This indiscriminate bottleneck explains their lower end-to-end accuracy (Table 1) and higher concept leakage (Table 2). By contrast, our CIBMs apply a structured Information Bottleneck: they retain all the signal that drives Y-gheher T(C; - shedding only the noise-lower T(x;C)= which both boosts predictive performance and cuts leakage. In other words, achieving expressiveness first (then selective compression) yields representations that are both robust and interpretable. Appendix H presents detailed information-plant trajectories, and our findings echo recent theory on IB in deep nets, which warns against blind compression in favor of task-guided pruning (Kawaguchi et al., 2023).
Overall, we have found that pursuing compression alone is not the solution for obtaining more robust representations. Instead, we see that achieving more expressive representations (i.e., higher mutual information with respect to the labels) followed by compression (i.e., lower mutual information with respect to the data) helps reduce the gaps in predictive tasks (see Table 1) as well as in leakage (see Table 2). However, due to the requirements for expressiveness, the CIBMs do not compress as much, since they must retain some useful information. Our findings align with recent theoretical insights on the Information Bottleneck principle in deep learning (Kawaguchi et al., 2023), which emphasize that indiscriminately minimizing the mutual information between the data and the latent representations, I(X; Z), does not guarantee expressive or generalizable representations. Effective models must selectively compress iirelenaele information while retaining essential features for decision-making.
4.6 EVALUATION OF OUR REGULARIZERS' HYPERPARAMETERS
We evaluated the hyperparameters of proposed regularizers on CBM (SJ) to select the values that we used for all other experiments. We evaluated our regularizers in single model to find the best setup due to computational constraints. We compare the performance of IB n and IB y on concept and class prediction accuracy for the CUB dataset (using 3 = 0.5) and report the results in Table D.3. As shown, g g. which retains an explicit mutual information term I(X: C). outperforms B when trained in a fair setup (vanilla) in both metrics. We found that the lack of performance of the vanilla BBg regularizer comes from instabilities during training the latent representations encoder p( 2 x.). We hypothesize that the gradient from the (C) in the loss (4) damages the feature encoder p(z / z) since the entropy is computed w.r.t. the generative concepts p(c) instead of the variational approximated ones (()). To alleviate this problem, we experimented gradient clipping as well as stopping the gradient from H (C) into the encoder. We found that the latter performs on par with iBg. In the experiments, we use BBg with stop gradient on it. Overall, IB 's more granular control over information flow limits concept leakage, results in better accuracies for concepts and labels in comparison the baselines (cf. Table 1) without changes its training framework. We also evaluate six different values for the 3 constant that controls the mutual information between the data and the concepts. We show these results in Table D.4. Since we obtained inconclusive results, we selected 3 = 0.5 for our experiments.
These results supports our earlier discussion that the direct estimation of I I(X; C) leads more effec- tive use of concepts in downstream tasks without further changes to the training regime. Nevertheless, with a correctly regularized feature encoder p(2 z)., simple estimation IBg g can achieve similar levels of information gain and accuracy.
5 CONCLUSION
We present Concepts' Information Bottleneck Models (CIBMs). first-principled fusion of Infor- mation Bottleneck theory and Concept Bottleneck Models that both explains CBMs' failure modes and prescribes their cure. By penalizing T(X; C') while preserving 7(C; Y), Concept Information
9

---

Published as a conference paper at ICLR 2026
Bottleneck reveals why vanilla CBMs over-compress and leak spurious signals and how surgical, task task-guided compression can retain exactly what matters. We validate CIBMs across six CBM fami- lies (hard/soft. jointindependent, ProbCBM, ImtCEM, and AR-CBM) three benchmarks (CUB, AwA2, and aPY), employing concept accuracy, class accuracy. Oracle and Niche Impurity (OIS and NIS), and intervention metrics (AUCTTI, NAUCrTI). The result uniformly higher class accuracy, dramatically reduced concept leakage, and equal or better concept-preed preepptctreiiioon performance- much of the CBM-black-box gap. Crucially, our findings show that: (a) simple, selective compres- sion can unlock robust, interpretable concept representations; and (b) that leakage undermines the use of concepts far more than their detection, explaining why near-pen perfepefect concept predictors can still yield subpar end-to-end performance.
ACKNOWLEDGMENTS
This work was funded, in part, by RCN (the Research Council of Norway) through Visual Intelligence, Centre for aesearch-basse Innovation (grant no. 309439), and FRIPRO (grant no. 359216). The computations were performed, in part, on resources provided by Sigma2- the National Infrastructure for High-Performance Computing and Data Storage in Norway (Project NN8104K).
REFERENCES
Alessandro Achille and Stefano Soatto. Emergence of invariance and disentanglement in deep representations. Mach. Learn. Res., 19(50): 334, 2018. URL http:://mm1: watpel..rrpppppppp
s/v19/17-646 htt.s/116646.mmm.. html.
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy. Deep variational information bottleneck. In Inter. Conf. Learn. Represent. (ICLR), 2017. URL http:::s////peneeieee....
et/forum? weetimiid--ixxoe Hy
Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, Pietro Lio, and Mateja Jamnik. Concept embedding models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Adv. Neural Inf. Process. Sys. (NeurIPS), 2022. URL https: /pppr///eeeeeeeeeeeeeeeee
www.areeieeeeeeeeeeeeeemmm .nt/ www.mriidioooooeooo.......oiiiir forummii--XXEEAAGE
Mateo Espinosa Zarlenga, Pietro Barbiero, Zohreh Shams. Dmitry Kazhdan, Umang Bhatt, Adrian Weller, and Mateja Jamnik. Towards robust metries for concept representation evaluation. In AAAI Conf. Artif. Intell. (AAAI), pp. 11791-11799, June 2023a. doi: 10.16099aaai.v37iill v37:10.26392 URL httpiiiwwwppppp.//aaaeteette AAAT/arti wwwevieeeeeeeeeeeeeeeeeeeeeeeeee 26392..
https:/ /ajs./j..////// aaai org/ / ndex. php/
Mateo Espinosa Zarlenga, Katherine M. Collins, Krishnamurthy Dj Dvijotham, Adrian Weller, Zohreh Shams, and Mateja Jamnik. Learning receive help: Intervention-aware concept embedding models. In Adv. Neural Inf. Process. Sys. (NeurIPS), 2023b. URL https: http:////pp//eeeee/////eeeeeeeee et/forum? d= ImžxqmT1k,
Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. Describing objects by their attributes. In JEEE/CVF Inter. Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 1778-1785, 2009. doi: 10.1 (09/CVVR.2009952266
Marton Havasi, Sonali Parbhoo, and Finale Doshi- Addressing leakage in concept bottleneck models. In Koyejo, Mohamed, A. Agarwal, D. Belgrave, Cho, and A. Oh (eds.), Adv. Neural Inf. Process. Sys. (NeurIPS), volume 35, pp. 23386-23397 Curran Associates, Inc., 2022. ct/ wc/pappr.r..... walles/ppeez/ee filee//
URL https: http::///pcceeeinnssssnnee hon.aeoeeeeppe---Coeeeeeee.e..e Conference
44ecf65a4 (5a.
Kenji Kawaguchi, Zhun Deng, Xu and Jiaoyang Huang. How does information bottleneck help deep learning? In Inter. Conf. Mach. Learn. (ICML), pp. 16049-16096. PMLR, 2023.
Eunji Kim, Dahuin Jung, Sangha Park, Siwon Kim, and Sungroh Yoon. Probabilistic concept bottleneck models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Inter. Conf. Mach. Learn. (ICML), volume 202 of
10

---

Published as a conference paper at ICLR 2026
Proceedings of Machine Learning Research, pp. 16521- PMLR, 23-29 Jul 2023. URL https:://proceedingg mir..ress / v2222/22//////////222 kim2 3.. html.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Inter. Conf. Learn. Represent. (ICLR), San Diego, CA, USA, 2015.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Hal Daumé III and Aarti Singh (eds.), Inter. Conf. Mach. Learn. (ICML), volume 119 of Proceedings of Machine Learning Research, pp. 5338-5348 PMLR, 13-18 Jul 2020. URL https: http:////ccccei/eeeeeeoo mlr wtt.ppppwsss..eeeeppppppp ml.
ht
A. Mahinpei, J. Clark, I. Lage, F. Doshi-Velers and P. WeiWei. Promises and pitfalls of black-box concept learning models. In Inter. Conf. Mach. Learn. Wksps. (ICMLW), volume 1, pp. 1-13. PMLR, 2021.
Andrei Margeloiu, Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller. Do concept bottleneck models learn as intended? In Inter. Conf. Learn. Represent. Wksps. (ICLRW), 2021.
David A McAllester. Some PAC- PAC-Bayesian theorems. Mach. Learn., 57:355-3633 1999.
David A McAllester. PAC-Bayesian stochastic model selection. Mach. Learn., 51(1): 2003.
Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, and Tsui-Wei Weng. Label-free concept bottleneck models. In Inter. Conf. Learn. Represent. (ICLR), 2023. URL BRLLtttss//pppezeeeee net/f orum?ide FCg4 7MNvBA
Nishad Singhi, Jae Myung Kim, Karsten Roth, and Zeynep Akata. Improving intervention efficacy via concept realignment in concept bottleneck models. In European Conf. Comput. Vis. (ECCV), 2024.
Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physsics/0004057, 2000.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. CUB-200-2011. Technical Report CNS-TR-201 1-001, California Institute of Technology, 2011.
Yongqin Xian, Christoph H. Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning- comprehensive evaluation of the good, the bad and the ugly. IEEE Trans. Pattern Anal. Mach. 4109):25111 2019. doi: 20000//....211......
Xinyue Xu, Yi Qin, Lu Mi, Hao Wang, and Xiaomeng Li. Energy-bascd based concept bottleneck models. In Inter. Conf. Learn. Represent. (ICLR), 2024. URL https: /oppnreview net/forum
?id=IIquo Txzux.
Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In /EEE/CVF Inter. Conf. Comput. Vis. Pattern Recog. (CVPR), pp. 19187-19197, 2023.
Mert Yuksekgonul, Maggic Wang, and James Zou. Post-hoc concept bottleneck models. In Inter. fort/formm?ico
Conf. Learn. Represent. (ICLR), 2023. URL 5AZ8CEyow.
A DETAILED DERIVATION OF CIB
In this section we present the detailed derivations to obtained the results described in Section 3.1.
We re-tt the lower bound of the concepts' information bottleneck as
LCUM = (Y) + (1 (C) H(Y [) H(C Z) (H (X)
(A.1)
11

---

Published as a conference paper at ICLR 2026
work with the entropies instead. We consider approximation of the predictors for the labels and the concepts, Q(y c) and g(c 2), based on two variational distributions that will be implemented through neural networks -f. Fig. 2. A conditional entropy, H (A] B), can be expressed in terms of a variational family g(a] b), as
(AA B) =
llgp(a b)],
(A.2a)
p(a,b)
log p(a b)
E
ạa b) + log a(a b)
(A.2b)
p(a b)
((b)
[- (p(a b) |l d(a b)) + H (galb)). 6))]
(A.2c)
P(b)
S
E p(b)
H (æ(a b). ( b]]],
(A.2d)
given that the KL (L(p(alb) b) ((aa(())))))))))))))){{)))))({ 6)) 2 Consequently, we can re-write our conditional entropies as
HY C) <
E M (p(y) e), 1 c))l
(A.3)
p(c)
((C) Z) S E
H (p(c) 2), g(e x))]
(A.4)
p(=)
The conditional entropy of the concepts w.r.t. the data is bounded by
X(C H(C X, X, I[C; X),
(A.5a)
2 H(C Z),
(A.5b)
since I(C; Z X)) 2 0.
Thus, the concept bottleneck loss (2) is lower bounded, given that remove the KLs constraints, due to their positivity, from the conditional entropies (A.3), (A.4) and (A.5), and the labels entropy. by
LCISM 2(1 211 ()
EH (ply c), aly 0)) (1 B) (p(c 2). [(cl p())
(A.6)
p(c)
Moreover, we can re-write the entropy of the concepts as
H(C) = (C 7) + T(Z:C),
(A.7a)
H(C Z).
(A.7b)
Thus, the CIBM loss is then lower bounded by
CCIBM 2 (1 - 8)
[ (p(c) 2)) H (plc ())c)2),00 |
E H (plylc),((y1o))))
(A.8)
p(z)
p(e)
-1 3) E (L l| [((l=))
E plc)
H (p(y c). aly c)).
(A.9)
p(=)
-SS-CIMM
(A.10)
We can then maximize this lower bound, Ls CIBM. to get closer to the concept bottleneck loss. In other words, we can maximize the concepts' information bottleneck by minimizing the cross entropies of the predictive variables, y and c, and their corresponding ground truths and by maximizing the entropy of the concepts.
B GENERALIZATION BOUND DETAILS
To analyze the generalization our concepts' information bottleneck models, we use the PAC-Bayes framework (McAllester, 1999; 2003).
Let D {Ix, Vs, N be dataset of N i..d. samples drawn from unknown population distribution P. such that D P. Let be the parameters of the neural networks that approximate the encoders (.).
12

---

Published as a conference paper at ICLR 2026
Proposition 1 (eAC-Bayes CBM), For any prior P(O), fixed before training. and posterior Q
learned by the optimization, with probability at least over the drawn data D ~ P, we have that for all 0
2VN 8
ReaM (Q()) < RCAM (Q())
KI(Q(0) P(O)) + log /2
(B.1)
Proof. For a tuple of data d = (x,, y/ c), we define the traditional CBM's loss as ( (0, d) =Id ( (p c), g c)) AH (p(c ge 22)), where À models different variants of CBMs. The true risk is then ReAM (Q(0))
(B.2)
E E e~ d'op
(B, d)];
(B.3)
while the empirical risk is computed over the sampled data
N
(Q())((())) = E 8~Q
Σ
(O de),
(B.4)
N
where d; = (Is, Mi, ci) E D.
Since D = {( Mi, c;) c:) are i..dd,, the terms (CBM d) are independent for fixed 0. Thus, by applying the standard PAC PaC-sayes theorem (McAllester, 1999; 2003), we obtain
2VN
RCBM RCBM(Q(8)) (Q(0)) < RCBM(Q(()) (Q(0)) V2N
KL(Q(() || P(0) log
(B.5)
Theorem (PAC-Bayes CIBM), Any prior P(O), fixed before training, and posterior Q(0) learned by the optimization, with probability at least 1 5 over the drawn data D P, we have that for all
1
2VN
ReIBM ReIns(((())) (Q(1)) < RemuM (Q())) +
KL(Q(0) || P(O)) + log
(B.6)
v 2N
8
Proof. For a tuple of data d (x, y. c), we define the CIBM's loss by expanding our concepts' information bottleneck (2)
fcIm( (0, d) = Hppyy c), go c)) + H(p(e =), (c 2)) + 3T(X;))
KL(p(v e) g c)) KL(p(c (a =)) - H(Y) H(C),
(B.7a)
= -H((y c), g c)) + H(p(c =), [c] =)) + +C
(B.7b)
For the purpose of the generalization bound, we consider the cross-entropies as the risk and the mutual information, while ignoring the reminder terms C, such that we can compare it with the standard CBM in the following steps, since
(Cuu (0, d) < (0, BT(X
(B.8)
since C negative, Thus, the true risk
REIBM (Q(0))
E E (0IBM d));
(B.9)
0~~ d-p
while the empirical risk computed over the sampled data
RCBEM (Q(#))
E 8-
1
Σ
EcuMM d:),
(B.10)
Q
N
where d; (xi, (is, D.
13

---

Published as a conference paper at ICLR 2026
Similarly to Proposition by applying the standard PAC-Bayes theorem (McAllester, 1999; 2003), we obtain
2v
RCIM (Q(M) RcIBm))( (Q(0))
KL(Q(0) || P(0)) + log
(B.11)
V2N
Theorem 2 (Generalization Advantage of CIBM), Let the encoders q(-) be parameterized by 0 e. Provided that 3 is sufficiently small, the True Risk of the CIBM satisfies the inequality
R(QCIBM) BeиM A,
(B.12)
where BCвM is the generalization bound of the traditional CBM. and the improvement gap A is defined as
2 11/2NKL(QQBB || P) + -
AKL
(B.13)
where AKI. KL(QCRM P) KL(QCIRM P), and K. = log((v/N/
Proof. First, let's find the optimal distributions that minimize the empirical risk of each model. For the CBM, the optimal posterior is
QCBM = arg min (Q),
(B.14)
while for CIBM
QCmM = arg min RCBM (Q);
(B.15a)
Q
arg min RCEM((Q) 3T(x:C))
(B.15b)
where the relation between the empirical risks (B. is given by the relation between the CIBM and CBM losses (B.8). Achille & Soatto (2018) showed that the information of the representations is bounded by the information the weights, that is,
((; < ((D:0) = (Q((0(0) D) P(0)).
(B.16)
D
Thus, our minimizer for the CIBM empirical risk is actually solving
[RCMM(Q + BEELLLICC (D) | P(O)
QCIM arg min Q
(B.17)
In other words, the posterior QCIS that minimizes the empirical risk for the CIBM already minimizing the posterior of the generalization bound, which is fundamentally different from the posterior minimizer of the CBM, QCAM, which only minimizes the risk.
Per the PAC-Bayes theorem, the True Risk R(Q) for any posterior is bounded by its Empirical Risk R(Q)) plus complexity term C(Q)= (KL(Q + K)/2N, where X. 1og(2VN/6).
Therefore, for the CIBM, we start with the inequality
1
RCIem(QCIBN) RCIBM(QCIBM) +
KL(QQBBM
PP+ K.
(B.
V2N
Our goal is to substitute the terms the right-hand side with those of the CBM reveal the gap. Substituting the Empirical Risk. Since QCIM is the global minimizer of the regularized objec- tive (B.17), its objective value must be lower than equal that of any other distribution, including QCRM. such that
(KUEQ P) < RRCEM(QQEEM + +.KLLQCUMM P).
(B.19)
14

---

Published as a conference paper at ICLR 2026
Let AKL KL(QQuBM P) KL(QCIBM || P), such that AKL due to the explicit penalization in CIBM. Rearranging the inequality provides bound on the increase in empirical risk as
RCwM(QCUMM) - RCBm(QCEM) + AAKL.
(B.20)
We substitute this into the true risk bound (B.18)
I V2N
R(QCIBM) < <R(QCBM) + 3AKL +
KKL(QCTMM P
(B.21)
Substituting the Complexity Term. Since ÅKL. > O due to the information penalty, we can rewrite the complexity term of CIBM in terms of the CBM's complexity
KL(QcmmM P) +
KL(QCRMM P) AKI. +
(B.22)
V2N
V2N
Let A = KL(QCBM P) + K. Using the first-order Taylor approximation concavity bound of the square root function, vz y vz 2/
we obtain
AKI.
AKL
VA
(B.23a)
2N
V2N
2v A
AKL.
C((CBM)
(B.23b)
2V2NA A
Deriving the Final Gap. Now we substitute this upper bound for the complexity back into the true risk bound (B.21)
AKL
R(QCIBM) S it(QcAM) + C((CAM)) + BAKII.
(B.24)
2/2NA
R(QCIBM) S BERM AKI. 2/2NA - 3)
(B.25)
We define the gap term as
A = AKL
2 (KLL(OMM P) + K.) -B
(B.26)
Provided that is sufficiently small such that 2V/NN then A > Thus, we conclude
R(Qcuum) BCBM A.
(B.27)
C STUDY OF EMPIRICAL ENTROPY H(C)
In deriving the E-CIB bound (6), we simplified the formulation by treating the entropy of the concepts as approximately constant. On one hand, we assume that the concepts entropy was constant since it depends the prior distribution of the concepts. On the other hand, this simplification is also empirically supported by our results (see Fig. C.1), which confirm that concept entropy indeed rapidly stabilizes early in training, exhibiting minimal fluctuation thereafter.
D IMPLEMENTATION DETAILS
D.1 DETAILS ON THE MODELS
In the following we describe the experimental setup for each model's encoder p( x).. The IB- regularized variants exactly the same configuration their unregularized counterparts, except for the addition of the IB-reuularizer (see the paragraph below for implementation details). The original CBM family (Havasi al., 2022) employed Inception backbone, and followed this choice.
15

---

Published as a conference paper at ICLR 2026
14
12
10
HIC
6
4
2
0
10
20
30
10
50
60
70
80
90
100
Epochs
Figure C.I: Calculation of the empirical entropy of the concepts, H(C), during training for CBM (SJ) on CUB.
Our code based on the ProbCBM implementation (Kim et al., 2023). We trained the models for 100 epochs while the authors of CBM (Havasi et al., 2022) used 1000 epochs, but we adopt the shorter schedule that standard in more recent works see the sensitivity analysis in Appendix D.3). ProbCBM (Kim et al., 2023) uses a ResNet-18 18 backbone. We adopted the exact configuration and code from the original repository and trained for 100 epochs. InCCEM (Espinosa Zarlenga al., 2023b) is implemented with ResNet-34 backbone. We reused the IntCEM code and trained the model for 300 epochs. AR-CBM (Havasi et al., 2022) also relies on an Inception backbone. We followed the AR-CBM repository and trained for 200 epochs. All remaining hyper-parameters (learning rate, optimizer, batch size, etc.) are kept identical to the original implementations. A detailed comparison of different backbone choices and their impact on performance provided in Appendix D.2.
To endow any existing concept-bottlenee model with our infformation-bottleenn regularizer we modify the layer that produces the latent representation Z. Specifically, we attach two lightweight heads to the model's embedding layer (the bottleneck): one head predicts a mean vector ju(2) and vector (z).. In nutshell for these heads, we add on top
the other predicts standard- of the model's embedding layer (the bottleneck of the model) two I I-layer MLP (i.e., our heads), for mean and standard deviation using the reparametrization trick in the variational approximation q(c 2), each of dimensionality 112-the number of concepts left after filtration identical to one done in Koh et al.'s (2020) work. For IntCEMs, we introduce variational approximation for every concept embedding projection. We obtain concept logits as C = pred,, (x) + pred, (x) e, where is a random standard Gaussian noise. On top of concepts logits, we stack label predictor g(y) c) (also I-layer MLP). All activations between the layers are ReLU.
D.2 DISCUSSION ON REPRODUCIBILITY OF PREVIOUS WORK
We emphasize that in our implementation of each baseline and their IB-regularized versions utilized the same setups (backbones and hyperparameters) as discussed in Appendix D.I. Additionally, we validated the execution of the base model and found that our results were within one standard deviation of the reported baselines in the original papers, confirming the accuracy of our baselines. Despite these sanity check values, we reported the original values, as is standard practice.
CBMs' results (Havasi et al., 2022) use two backbones: Inception and ResNet18. We retrained the CBM (SJ) and our regularizers for 100 epochs while varying the backbone and observed similar results, albeit slightly lower with ResNet18, as shown in Table D. For example, the CUB dataset trained for 100 epochs, we observed class accuracy of 0.708 with the Inception backbone compared to 0.683 with ResNet18. Considering our experiments with up to 1000 epochs (cf. Appendix D.3), we predict these results will scale similarly. Notably, the concept accuracies are comparable at 0.95, suggesting that the results reported by Kim et al. (2023) align more closely with the Inception backbone than with ResNet18.
Regarding ProbCBM (Kim et al., 2023), our results follow those using a ResNet18, aligning with the original results. In the case of IntCEM and CBM results, despite the claims in ProbCBM, believe there are inconsistencies between the original reported results (Kim et al., 2023) and the claims regarding the backbones, as explained below.
For IntCEM, the results are remarkably similar between the ResNet18 and ResNet34 backbones, as noted by Espinosa Zarlenga al. (2022) in Appendix A.4 (Fig. A.7) of their paper. This similarity
16

---

Published as a conference paper at ICLR 2026
Table D.I: Effect of the backbone the CBM (SJ) and regularizers.
Method
Backhone
Concept
Class
CBM
Inception
0.956
0.708
CBM 1B
Inception
0.957
0.726
CBM IBg
Inception
0,957
0.727
CBM
Resnet18
0.951
0.683
CBM IB 8
Resnet18
0.954
0.701
CBM IB
Resnet18
0.954
0.703
Table D.2: Comparison of concept and class prediction accuracies when changing the number of epochs in
CUB.
Method
Epochs
Concept
Class
CBM
100
0.956
0.708
CBM IB
100
0.957
0.726
CBM IBg
100
0.957
0.727
CBM
1000
0.957
0.795
CBM + IB
1000
0.958
0,806
CBM IBg
1000
0.958
0.810
ProbCBM
50
0.954
0.702
ProbCBM IBg
50
0.954
0.710
ProbCBM + IB
50
0.954
0.708
ProbCBM
0.956
0.718
ProbCBM + IBg
100
0.957
0.734
ProbCBM + IB
100
0.957
0.732
might explain the scores reported in the ProbCBM (Kim et al., 2023). Additionally, there is no evidence that Kim et al. (2023) implemented and IntCEMs; rely their assertion regarding ResNet18, 8, but there is no code-based justification. We executed InCMM with 300 epochs, as described in the original paper, using different backbones and found accuracies of 0. 0.757, 0.762, and 0.770 for ResNet18, ResNet34, and Inception, respectively. These results support the original InCEM paper's claim that there no sigmificant difference with respect to the backbone. Moreover, the results for the ResNet models fall within the standard deviation of the values reported in the ProbCBM paper. However, establishing significant results would require more runs and a proper hypothesis test, which are beyond the scope of our work.
D.3 CBMs SENSITIVITY TO TRAINING EPOCHS
In the main experiments, we trained the CBMs for 100 epochs following protocol similar to ProbCBMs (Kim et al., 2023). However, the original CBMs (Havasi et al., 2022) were trained for 1000 epochs. Table D.2, we reproduced results from Havasi al.'s (2022) work using their exact setups, including the number of training epochs. As observed, our proposed IB- IB-regularized models obtain better concept and class predicitons overall. Interestingly, the concept predictions saturate when training for 1000 epochs, while the class predictions improve. This may be due to overfitting for class prediction. These results validate that our main experiments are valid and comparable despite using fewer epochs for the CBMs. We highlight that the other methods follow the original setup that trained for hundreds of epochs as well.
D.4 ESTIMATORS DETAILS
Mutual Information Estimator. Before each gradient update, compute cross-entropies over the current batch B., and then randomly sample batch B' from the training dataset to estimate 1 (X;C) on this batch.
Our mutual information estimator follows Kawaguchi et al''s (2023) work. We rely on the fact that concepts logits have Gaussian distribution for estimation of p(c ()). And then, use the random samples B' to approximate the marginal of the concepts log p(c). The mutual information (C; is then Monte-Carlo estimate of log p(c ()) log,(c).
17

---

Published as a conference paper at ICLR 2026
Table D.3: Accuracies of CBM (SJ) with proposed regularizers, B and IB gs CUB dataset (avg. runs).
Method
Concept
Class
IB a (vanilla)
0.934
0.608
(clip_norm 1.0)
0,948
0.663
(clip_norm 0.1)
0.948
0.650
(stop grad. from HC)) into p( x))
0.958
0.726
BB
0.958
0.727
Table D.4: Evaluation of CBM (SJ) with the proposed regularizers three datasets with two different values of
$.
CUB
AwA2
aPY
Method
Concept
Class
Concept
Class
Concept
Class
0.10
IB
0,957
0.726
0.978
0.879
0.967
0.851
IB
0.958
0.728
0.979
0.881
0.967
0.854
0.20
IB
0.957
0.726
0.980
0.880
0.967
0.853
IB g
0.958
0.727
0.978
0.880
0.967
0.852
0.25
IB
0,958
0.725
0.980
0.886
0.967
0.850
0.958
0.727
0.980
0.885
0.967
0.858
0.50
IB
0.958
0.725
0.979
0.885
0.967
0.856
IB g
0.959
0.729
0.979
0.883
0.967
0.856
0.75
0.958
0.724
0.979
0.882
0.967
0.853
IBg
0.958
0.724
0.980
0.884
0.966
0.854
0.90
IBg
0.957
0.722
0.980
0.884
0.967
0.856
IB
0.958
0.726
0.979
0.883
0.967
0.855
Entropy Estimator: Since concepts C' are distributed normally, we use H(C) 21 + log(2x)) log El For simplicity (since the number of concepts D constant throughout the training and inference) use HCC) 2 log | Σ log(a;) since Σ is diagonal matrix in setup.
D.5 TRAINING PARAMETERS
We explained the hyperparameter selection in Section 4.6. We experimented with different setups to find the best configuration. We show these results Tables D.3 and D.4. The other training parameters for the models are as follows. We set batch size to 128 and number of samples for MI estimation 64. For all experiments we used Adam (Kingma & Ba, 2015) optimizer with Ir = 0.003 and wd 0.001. We experimented with gradient clipping. but it led either slow or divergent training, so we are not clipping the gradients in any of the experiments.
D.6 DATASETS
We benchmark our approach on 3 datasets: CUB (Wah et al., 201 AwA2 (Xian et al., 2019), and aPY (Farhadi et al., 2009). While CUB is a recognized dataset for comparing concept- based approaches (Espinosa Zarlenga et al., 2022; Kim al., 2023; Koh et al., 2020), we add the other two datasets for additional evaluations and analysis.
CUB. Mattech-UCSD Birds dataset (Wah al., 2011) dataset of birds images totaling in 11788 samples for 200 species. Following Koh et al.'s (2020) work, for reproducibility. we reduce instance- level concept annotations to class-level ones with majority voting. We then keep only the concept that are annotated as present in 10 classes at least after the described voting, resulting in 1 112 concepts instead of 312. We also employ train/val val/test splits provided by Koh et al. (2020), operating with 4796 train images, 1198 val images and 5794 test images. To diversify training data, augment the images with color jittering and horizontal flip, and resize the images to 299 x 299 pixels for the Inception V3 V3 backbone. Concept groups obtained by common prefix clustering.
AwA2. Animals with attributes dataset (Xian et al., 2019) is a dataset 37322 images 50 animal species. For the concepts set, follow Kim al.'s (2023) work and keep only the 45 concepts
18

---

Published as a conference paper at ICLR 2026
CBM (SJ)
CBM (SJ)+IBB
CBM (SJ)+IBg
Concept losses
Label losses
1.0
1.0
Normalized loss
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
10
20
30
40
0
10
20
30
40
Epochs
Epochs
Figure D.1: Losses the validation set of CUB for CBM (SJ) and its variants regularized with our proposed methods.
which could be observed on the image. We use ResNet18 embeddings provided by the dataset authors and train FCN on top of them. No additional augmentations are applied to those embeddings.
aPY. This is dataset (Farhadi et al., 2009) of 32 diverse real-worke classes used for proof of concept. We split the dataset into 7362 train, 3068 validation and 4909 test samples stratified on target labels. We train FCN on top of ResNet18 embeddings of input images provided by the dataset authors (Xian et al., 2019). No additional augmentations are applied to those embeddings.
D.7 DETAILS ON EXPERIMENTS
The image embedder backbone is only trained for CUB dataset (Wah et al., 2011), and for AwA2 (Xian et al., 2019) and aPY (Farhadi et al., 2009) we use pre-computed image embeddings. The ground truth concept labels are binary across all dataset, but concepts predictions passed to label classifier non-binary: concepts for class prediction.
training only (and comparing only against) models using soft
When training models with IB,, used the CBM (4) for better performance. We backpropagate the gradients from the cross-entropies over concepts and labels through the entire network-both backbone q(c =) and MLPs top of the encoder q(y c). For HCC), however, the situation is different: gradients from this part of the loss function are propagated only through the MLPs, g(c 2) and ( c). but not the image embedder backbone p(> .r). We found that such (partial) "freezing" of the encoder with respect to (CC)) constraint dramatically improves the quality of both concepts and labels prediction. While we do have access to the ground truth probability distribution for the concepts p(c] 2). we have access to the ground truth concept labels. Our implementation uses the supervised cross-entropy using the ground truth labels. The concepts' predictor can be seens as multi-label task classifier. In practice, we compute C' logits, then, we compute binary cross-entropy (BCE) for each of these logits with binary labels. Finally, we backpropagate them through the means of BCEs.
We show the normalized loss function values on the validation set of CUB Fig. D.1 to show the convergence of CIBMs in comparison CBM (SJ). Note that visually the concept losses between CBM (SJ) and its variant regularized with IBg and the label losses between CIBMs are similar, but they differ slightly.
E EXTENDED RESULTS ON PREDICTION
We trained the CBMs on CUB using the soft sequential (SS) versions our proposed method to complement the different versions of vanilla CBMs. We present the results in Table E. As demonstrated, our regularizers perform as expected and continue to outperform the corresponding baselines. While report these additional results for completeness, we highlight that, most CBM- based works report the soft joint version of CBM, with only the original paper presenting the soft sequential versions.
19

---

Published as a conference paper at ICLR 2026
Table E.1: Extended results for Table 1 on CUB. Accuracy results include and std. 5 runs. We report results of our proposed regularizer methods. 1B R and EBg, applied to different CBMs. Black-box is gold standard for class prediction that offers no explainability over the concepts.
Method
Concept
Class
Black-box
0.919-000022
CBM (HD)
0.956110.001
0.650.10.002
CBM (II) IB
0.946_00000
0,65310.0004
CBM (HJ) IBg
0.960=0.000
0.647=00000
CBM (HI)
0.956:+0.001
0.64411:0001
CBM (HI) + IB
0.955=0.000
0.683=0.000
CBM (HI) + IBg
0,948=0.002
0.679=0.000
CBM (SI)
0.956110.001
0.639110.001
CBM (SI) + IB g
0.955=0.000
0.642=00.000
CBM (SI) IBg
0.958=0.004
0.642=0.0099
CBM (SS)
0.956.10.001
0.640.10.001
CBM (SS) 1B
0.054=0.000
0.673-10.008
CBM (SS)
0.955=0.000
0.664=0.000
CBM (SJ)
0.956:10.001
0.006
CBM (SJ) + IB
0.050000000
0.724
CBM (SJ) + 1B
0.907=0.000
0.731
F EXTENDED RESULTS ON INTERVENTIONS
In Fig. F.I. we show the plots of Fig. 3 separated and grouped by the type of method and dataset in order to better visualize the trends. We highlight that the fewer points in the results for InCCEM follows the results from Espinosa Zarlenga et al. (2022).
In Fig. F.2, we show additional results about the aggregated interventions that we dicussed in Section 4.4 and that we showed in Table 3. We plot the interventions in the traditional way by showing the intervened groups and the TTI performance for six different corruption settings.
G EXTENDED RESULTS ON CONCEPT LEAKAGE
G.1 OIS AND NIS METRICS
The Oracle Impurity Score (OIS) (Espinosa Zarlenga et al., 2023a) quantifies impurities localized within individual concept representations. Given a concept encoder g: X - Ó' C Rd test samples Ixx and their concept annotations I, OIS is defined as:
OgS I'x, 21[ - (T,,))lF
(G.I)
k
where .(f; T) is purity matrix whose entries (f. () contain the AUC-ROC score when predicting the ground truth value of concept given the i-th concept representation. The normalization ensures OIS ranges in [0, 1], with 0 indicating perfect alignment between the predictive capacity of learned and ground truth concepts.
The Niche Impurity Score (NIS) (Espinosa Zarlenga al., 2023a) captures impurities distributed across multiple concept representations. Let f: RD - C' be a classifier, and 'D = (r((), c(0),_ be a dataset of labeled concept representations. For each concept a concept niche N, (v, ()) is defindd as the set of concept indices whose representations are highly entangled with concept according to concept niche function v and threshold 3. Let the impurity set, 1, = {l,...,d] (M, 3), as the set of all indices excluded from the niche.
The Niche Impurity (NI) for concept measures how predictable this concept is from representations
outside its niche:
NI e, AUC-ROC (fN.()"))
(G.2)
20

---

Published as a conference paper at ICLR 2026
CUB
AwA2
Performance
1.0
1.0
NR
IB
0.9
CBM (HJ)
0.8
0.8
CBM (SI)
0.7
0.7
[MCEMM AR-CBM
0.6
0.6
0
12
16
20
24
8
10
12
14
Performane
1.0
1.0
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
12
20
10
12
14
TTI Perforance
1.0
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0
1
8
12
16
20
24
0
8
10
12
14
T11 Performance
1.0
1.0
0.9
0.8
0.8
0.7
0.7
0.6
0
12
16
20
24
10
12
14
TTI Perforanne
1.0
1.0
0.9
0.9
0.8
0.7
0.7
0.6
0.6
12
16
20
21
10 12
14
Number intervened groups
Number intervened groups
Figure F1: Expanded results from Fig. 3. Change in target prediction accuracy after intervening concept groups following the random strategy as described in Section 4.3. (TTI stands for Test-Time Interventions and for non-regularized.)
where ft, denotes the classifier restricted to operate only on the dimensions I, and ( denotes the sub-vector formed by selecting only the indices found in II; (the irrelevant features). The overall NIS is then calculated by integrating NIs across all concepts and threshold values:
L'' Σ NI,(,
NIS v)
d3.
(G.3)
NIS of 0.5 indicates random performance (no impurity), while a NIS of suggests that concept information is dispersed across multiple representations. Together, these metrics effectively evaluate concept quality without making unrealistic assumptions about concept independence or representation dimensionality.
G.2 CONCEPT SETS REDUCTION
We employed two different algorithms to cut the concepts set to half the size: selective (information- based) and random dropout. In the former, computed E[T(Y;C;) for all concept groups
21

---

Published as a conference paper at ICLR 2026
CBM (SJ)
CBM (B])+IBB
CBM (SJ)+IBE
corrupted
4 corrupted
0.8
0.8
T11 Porforrmaec
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
8
20
24
8 corrupted
16 corrupted
0.8
0.8
TTI Performance
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
8
12
16
20
24
8
12
16
20
24
32 corrupted
64 corrupted
0.8
0.8
TT1 Performance
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
20
12
20
24
Number intervened groups
Number of intervened groups
Figure E2:: Change in target prediction accuracy for different number of corrupted concepts. These the expanded results of Table (TTI stands Test-Time Interventions.)
subsample of the training set. Then we dropped out the concepts groups with the highest mutual information-that is, we made the "fair" (leakage-free) learning as unprofitable and hard as possible. On the other hand, the random dropout selects half of the concepts at random and drops the rest.
G.3 ADDITIONAL RESULTS
To complement the results of Table 2, we also performed the concept Icakage evaluation on AwA2 (Table G.1 and aPY (Table G.2). These results demonstrate that our regularizers are highly effective at mitigating concept leakage, evidenced by the consistent reduction in OIS and NIS across all three datasets (CUB, AwA2, and aPY). This improvement robust across every tested model architecture (including various CBMs, IntCEM, and ProbCBM) and intervention strategy. indicating that these regularizers successfully force models to learn better concepts by preventing the encoding of extraneous information. Furthermore, the performance of our regularizers is similar, suggesting that both are equally reliable solutions for enforcing concept integrity without a significant difference in efficacy between them.
22

---

Published as a conference paper at ICLR 2026
Table G.1: Concept leakage evaluation (lower is better) AwA2.
Complete CS
Selective Drup-out CS
Random Drop-oul CS
Model
OIS
NIS
OIS
NIS
OIS
NIS
CBM (HJ)
2.61 + 0.27
57.98 + 1.15
10.09
67.58
9.49 = 1.36
70.74 + 1.89
CBM (HJ) p
2.29 0.23
56.89
1.91
9.71
64.37
9.28 + 0.88
69.79 1.37
CBM (Hl)
2.01 0.43
63.03 = 1.97
9.27
64.18
8.71 = 1.03
69.14 t 0.34
CBM (HI) + BB
1.78 1 0.34
60.05 0.91
8.40
67.23
8.31 1 1.94
67.25 1.52
CBM (SJ)
3.01 0.29
64.44 + 1.92
11.24
70.41
10.69 + 1.00
70.68 0.57
CBM (ST) + DBp
2.62 0.24
61.63 + 0.49
9.52
68.12
8.94 - 1.52
68.57 + 1.26
IntCEM
6.34 0.70
72.46 t 2.86
14.74
74.19
12.16 + 0.24
72.24 + 1.58
IntCEM+ 18g
5.79 1.03
68.57 + 2.06
9.95
68.02
10.93 1 0.45
68.94 + 2.04
AR-CBM
3,90 3.
62.30 1 1.52
14.16
63.40
12.58 1 0.86
60.86 1 1.32
AR-CBM+ BB
2.70 0.19 0.19
60.04 1 0.98
11.05
60.85
11.03 1.02
57.21 0.85
ProbCBM
4.30 0.10
64.22 - 1.04
16.01
76.92
13.81 = 0.21
75.01 + 0.86
ProbCBM+ B
2.61 0.53
59.86 + 1.83
13.07
73.06
11.20 = 0.68
71.20 2.07
Table G.2: Concept leakage evaluation (lower is better) on aPY.
Complete CS
Selective Drop-out CS
Random Drop-out CS
Model
OIS
NIS
OIS
NIS
OIS
NIS
CBM (HJ)
2.66 + 0.19
57.97 + 1.20
10.17
67.54
9.44 + 1.45
70.78 + 1.84
CBM (HD) +IB
2.35 1 0.16
56.91 + 1.91
9.78
64.38
9.35 1 0.81
69.70 +: 1.34
CBM (HI)
1.94 1 0.41
62.97 1 1.91
9.19
64.26
8.78 1 0.95
69.06 1 0.24
CBM (HI) +
1.80 = 0.43
60.03 = 0.86
8.31
67.18
8.25 + 2.02
67.30 + 1.59
CBM (SJ)
2.97 + 0.37
64.39 1.96
11.23
70.40
10.72 + 1.01
70.73 0.58
CBM (SJ) +1B
2.60 + 0.30
61.55 + 0.55
9.56
68.03
9.01 + 1.43
68.59 + 1.21
InCCEM
6.36 1 0.64
72.48 1 2.89
14.74
74.24
12.24 1 0.16
72.31 1 1.53
InICEM+IB
5.78 1 1.06
68.66 1 2.09
9.86
68.04
10.99 1 0.39
68.91 1 1.98
AR-CBM
3.89 0.18
62.25 1 1.55
14.17
63.42
12.62 1 0.87
60.85 1 1.40
AR-CBM+ lB
2.72 0.20
60.00 + 1.00
11.04
60.92
11.00 1.08
57.19 0.86
ProbCBM
4.35 0.06
64.19 + 1.13
16.06
76.98
13.78 + 0.22
74.94 + 0.87
ProbCBM+ B
2.59 0.46
59.86 + 1.84
13.02
73.15
11.17 + 0.77
71.23 2.00
H INFORMATION PLANE DYNAMICS
We analyze the flow of information between inputs, X, latents, Z, concepts, C, and labels, Y, and present them in Fig. H. The objective of the information plane is to show the mutual information on the model variables after training. In particular, we expect to see a model with high 7(Z; C) and I(C; Y) such that the corresponding variables are dependent on each other (maximally expressive), and simultaneously, low I(X; C) and I(X; Z) to show that the corresponding variables compressive. However, the compression of the variables alone, minimal T(X; C*) or I(X; 2), does not guarantee that the important parts of the variables are being compressed and retained. Thus, show the other experiments to complement this analysis.
maximally
InICEM has a lower mutual information between the inputs and the latent and concept representations, [(;; 2) and ()) than CBM (SJ). Interestingly, our regularizers reduce these mutual information while maintaining the mutual information w.r.t. the target, (C;Y) and [Z; C). However, for CBM (SJ), our methods increase the mutual information w.r.t. the data. This behavior may reflect the fact that CIBMs are optimized retain task-relevant information while removing irrelevant or redundant information but not necessarily compressing as much- reflectee in the higher T(X; ; C') and T(X; 2). Nevertheless, lower mutual information T T(X; C.) and T(X; Z) in CBMs does not necessarily indicate better compression given its lower predictive accuracy. Instead, it may reflect failure capture meaningful input features, resulting in noisier or less predictive concepts. Moreover, we note that the plots in Fig. (f) for IB and BBg look similar but they differ in hundredths.
For AR- RBMM the information flow is more noisy. Despite the noise, we can observe that CIBMs obtain higher mutual information w.n.t the labels than their vanilla counterpart. While the compression w.r.t. the data is not as evident, the final mutual information w.r.t. the data is closer between the original method and its regularized versions. Nevertheless, still observed better predictive
23

---

Published as a conference paper at ICLR 2026
performance (ef. Table 1). Thus, hypothesize that the regularizer is increasing the expressiveness of the representations with trade-off of the compression as observed with the CBMs but not as apparent. On the other hand, the CIBMs obtain better compression-exprssion patterns for the latent representations, see Table (d).
To demonstrate the effects of the compression patterns, evaluate the alignment between represen- tations and the target [(C; Y) and show that CIBMs consistently outperform CBMs, and, while noisy, they show improvements over InCCEM, indicating that the retained information is both relevant and predictive-cf. Section 4.1. Additionally, CIBMs achieve better interpretability and concept quality, reinforeing that the higher mutual information a reflection meaningful expressiveness rather than leakage cf. Section 4.3. This is further supported by the proposed intervention based metrics (AUCrTI and NAUCrn) which highlight the importance of retaining task-relevant information the concepts C. While CBMs exhibit lower mutual information between inputs and representations in contrast the regularized versions, [(x; ; C)) and [X; 2). their poorer performance on these metrics, particularly under concept corruption, suggests that this lower information content stems from a fail- ure to capture sufficient relevant features. By contrast, the higher I(X;0) C) and (X; 2) in our CIBMs reflect the retention of meaningful pieces that contribute to better concept quality and downstream task performance. These findings demonstrate that reducing concept leakage requires selectively preserving relevant information rather than minimizing mutual information indiscriminately.
Our findings align with recent theoretical insights on the Information Bottleneck principle in deep learning (Kawaguchi et al., 2023), which emphasize that indiscriminately minimizing the mutual information between the data and the latent representations, (X; 2), does not guarantee expressive or generalizable representations. Instead, effective models must selectively compress task-irrelevant information while retaining essential features for decision-making. Our results (cf. Table and Fig. H.1) support this trade-off by demonstrating that CBMs, despite lower [(X; C) and [X; Z), do not necessarily achieve superior concept representations or intervention efficacy in comparison to their IB regularized counterparts. In contrast, our IB-based CBMs, which balance information retention and compression, lead to improved alignment between concepts and final predictions, reinforcing the importance controlled, task-relevant compression rather than absolute mutual information minimization.
I DISCUSSION ABOUT CBMs SETUPS
Hard CBMs use hard concept representations, meaning that instead of producing a probabilistic output (as in soft concepts in soft CBM), each concept prediction is treated as a discrete binary or categorical value. These hard predictions are used as inputs the downstream task (class prediction), making the pipeline interpretable and less expressive, thus less prone to information leakage.
When compared with soft CBMs and Soft CIBMs:
Representation:
Hard CBMs: Use discrete hard values for concepts (e.g., 0 or binary concepts).
Soft CBMs: Use continuous values (e.g., logits probabilities).
Soft CIBMs: Similar to soft CBMs but use IB to minimize irrelevant information, reducing concept leakage.
Information Flow:
Hard CBMs: Compress information into discrete concept values, which prevents information leakage but risks losing useful details for downstream tasks.
Soff CBMs: Retain richer information but more prone to concept leakage.
-Soft CIBMs: Balance retaining relevant information while mitigating leakage through the IB framework.
Interventions:
Hard CBMs: Explicitly rely on discrete corrections during interventions, which can have a significant impact.
Soft CBMs and CIBMs: Treat interventions as updates to probabilities or logits, which is more expressive, but could induce noise in concepts.
24

---

Published as a conference paper at ICLR 2026
CUB
AwA2
CUB
AwA2
1.0
1.0
InCEM
IC; Y
0.9
INCEMM
0.9
112:
10
0.8
0.7
InCEM++BB IICYY
0.6
Training progres
InCEM+IB [(2
0.6
1 3
0.5
0.5
0.4
0.4
0.3
JnCCMM+BB
ICC Y.
0.2
InCEM-iBa
1(2,0)
0.2
2
0.1
0.1
0.0
0.0
2.5
3.00
3.5
4.02.5
3.0
3.5
4.0
2.5
3.0 3.5
4.02.5
3.0
3.5
4.0
((X;C)
C(
7(X; (2)
r(X; 2)
(b)
5
1.0
I(C: V1
0.9
AR. CBM
1(2:0)
0.9
2 32
0.8
0.8
0.7
0.7
AR-CDM-1Bg
1(C:Y)
Training muggrss
AR-CBM-IB.
I1Z:C)
Trunning progres
13 00
0.5
0.5
0.4
0.4
0.3
0.3
AR BR-CBMiBBg
AR-CBM+IB2
I(C:Y]
[(2:C)
0.2
0.0
0.0
4.5
5.0
4.0
5.0
[(X:C)
I(X:))
1(X:2)
I(X;
2)
(d)
1.0
1.0
I(C:Y)
CBM (ST)
I(Z:0)
CDM
6
0.7
2
0.7
8
CBM B
[[:Y]
0.6
Triining progress
CBM (8)+BB
4
0.6
progress
0.5
6
0.1
65
9
CBM (S
CBMY SOyIiR
0.2
01
9
0.2
2
12:
0.1
50
0.1
0.0
2
5
6 7
5
6
5
6
7
C(X;C)
T(X: C)
7(X; 2)
T(X;
2)
(e)
(
Figure H.1: Information plane dynamics (in nats) for (a.b) IntCEM, (c,d) AR-CBM, (c, f) CBM (SJ) and our proposed methods, B and 18 Warmer colors denote later steps in training. We show the information plane of (a, c, e) the variables X, C. and Y; and (b, d, f) the variables X, Z, and C.
25

---

Published as a conference paper at ICLR 2026
Due to their rigidity, without enough interventions, hard CBMs cannot recover from errors noise in the predicted concepts because the discrete pipeline does not allow for soft adjustments.
But, as more concepts are corrected, the discrete nature of hard CBMs becomes an advantage together with its independent training: ground truth, hard values fully override noisy predictions, ensuring perfect imput for the downstream classifier, which was previously trained also on ground truth concepts from train set.
Soft CBMs and CIBMs, while retaining more information, still rely on probabilistic updates during interventions, which may not fully override noisy concept predictions.
Overall, CIBMs are superior because they combine the advantages of soft representations (expressive- ness, better performance) with mechanisms mitigate concept leakage (robustness, interpretability). Hard CBMs, while conceptually cleaner in avoiding leakage. fail to achieve the same level of downstream performance and adaptability, particularly in more realistic or challenging scenarios.
26


Reviews:
[{"review_id": "Dxg8F7JO6D", "reviewer": "Reviewer_DEB8", "summary": "Motivated by the information bottleneck principle, the paper proposes concepts’ information bottleneck model (CIBM), which uses an information bottleneck regularizer (IB regularizer) for training concept bottleneck models (CBMs) to mitigate the problem of concept leakage while preserving prediction accuracy. Based on theoretical observations, the authors give two types of regularizers, bounded and estimator based, for training CBMs. Through experiments, it is shown that IB regularizers maintain or improve prediction performance while mitigating concept leakage. Moreover, using CIBM, the authors propose a measure to quantify the quality of concept sets.", "questions": "1. Can the usage of two estimators (bounded CIB and estimator-based CIB) be theoretically justified, given that the theoretical analysis seems to be wrong?\n2. How does the IB-regularizer compare against other methods that mitigate leakage?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 1, "strengths": "1. The motivation for using information bottleneck principle to tackle concept leakage looks valid and interesting.\n2. The experiment results suggest that IB regularizers are indeed helpful to mitigate concept leakage while preserving final prediction accuracy.", "weaknesses": "1. As authors frame CIBM as theoretically principled integration of IB principle to CBM, examining the validity of the estimators is important. However, theoretical analysis for deriving the bounded CIB (section 3.1, section 3.2, appendix B) seems to have fatal errors.\n- The paper claims to upper bound the L_{UB-CIB}, but in eq (A.1), -\\beta H(C|X) should be +\\beta H(C|X), and in equations from (A.2a) to (A.4l), the minus sign should be added to all the right hand sides (definition of entropy). These errors seem to make the CIB upper bound in eq (3) or (A.5) completely wrong. \n- Also, due to above observations, equation (6) seems to be wrong.\n2. If the goal of the paper is to address concept leakage, comparing with previous works that address concept leakage would be a valuable addition. Also, these works do not seem to be addressed properly in the related work section.\n3. At section 4.3, using only random intervention seems insufficient. The authors should include experiment that uses more effective intervention strategies explored in [1] such as UCP.\n\nReference\n\n[1] Shin et al., A Closer Look at the Intervention Procedure of Concept Bottleneck Models, ICML 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PNwmX2cyrc", "reviewer": "Reviewer_zMXB", "summary": "This paper proposes to enhance the interpretability of Concept Bottleneck Models (CBMs) by applying an explicit information-bottleneck (IB) regularizer to the concept layer. It argues that traditional CBMs suffer from concept leakage (i.e., unintended information flowing into concept activations) and that minimizing I(X;C) while maximizing I(C;Y) and I(Z;C) yields cleaner and more faithful concept representations. They introduce two practical methods, integrate them into multiple CBM families and evaluate on three benchmark datasets. Experimental results suggest reductions in leakage metrics, modest improvements in classification accuracy, and more reliable concept-level interventions.", "questions": "1. Could the authors provide more detailed explanation or proof of under what conditions minimizing I(X;C) while maximizing I(C;Y) ensures minimal-sufficient concept representations (i.e., no leakage)? Are there assumptions (e.g., about concept annotation completeness, model capacity, independence among concepts) required for the theory to hold?\n2. In Table 1, for certain model-dataset combinations the accuracy difference between vanilla CBM and CBM+IBB/IBE is very small (e.g., <0.5%) and within one standard deviation. \n3. In Eq. (5) and (7), the paper includes an entropy term H(C).\nHow is this quantity computed or approximated when C is continuous or high-dimensional?\nDoes this require batch-level estimation or additional variance reduction techniques?\n4. What is the computational overhead of including IB regularization, in terms of runtime or memory?\nHave the authors observed any optimization instability due to the extra MI term?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper improve target accuracy, reduce concept leakage, and enhance intervention effectiveness.\n2. The information-theoretic framing is conceptually sound: targeting I(X;C) aligns well with the desired minimal-sufficient concept representation.\n3. The proposed methods are modular and broadly applicable: the regularizer is applied across many CBM variants without altering the architecture radically.\n4. Empirical evaluations are relatively comprehensive: multiple methods, datasets, metrics (leakage, concept accuracy, task accuracy, intervention AUC) are reported.", "weaknesses": "1. Stability of empirical results.\nThe empirical performance of different regularizers across datasets is not consistently positive.\nIn Table 1, certain variants of the proposed CIB regularizers improve accuracy or interpretability metrics on some datasets, but degrade or show negligible effects on others (e.g., AwA2).\nThis inconsistency suggests that the approach may be sensitive to dataset characteristics or model initialization, raising questions about its stability and generalization.\n2. Missing theoretical justification of guaranteed improvement.\nThe paper introduces the modified information-bottleneck objective\n$I(Z;C) + I(C;Y) - \\beta I(X;C)$,\nyet provides no theoretical guarantee, such as an error-bound or generalization-bound proof, showing that adding this IB term will necessarily improve the performance or faithfulness of existing CBMs.\n3. Overstated claims: The paper suggests that CIBMs “close the accuracy gap to black-box models without sacrificing\ninterpretability,” yet in many experiments the black-box baseline still significantly outperforms the IB-regularized CBMs. The practical significance of the interpretability-performance trade-off remains underexplored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HCNTKtX1u4", "reviewer": "Reviewer_urBx", "summary": "In this work, the authors propose CIBMs which utilize an Information Bottleneck regularizer on the concept layer to reduce concept leakage for a base CBM. Evaluating their approach on different CBMs (CBM, CEM, ProbCBM, AR-CBM), the authors show strong accuracy, concept leakage, and intervention capabilities.", "questions": "- What dataset is Table 2 for? Why is this run on only one dataset?\n- use beta=.5 for all experiments?\n- \"We also compare against more recent CBM variants such as ... intervention-aware CEM (Espinosa Zarlenga et al., 2022)\" - is this CEM or IntCEM [1]? Why evaluate on one but not both?\n- The motivation for the selected baselines in general is unclear, why is more recent work looking at intervention/leakage not included?\n- Some typos throughout the main text and appendix.\n\n[1] Espinosa Zarlenga, Mateo, et al. \"Learning to receive help: Intervention-aware concept embedding models.\" Advances in Neural Information Processing Systems 36 (2023): 37849-37875.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Concept leakage is an important problem within CBMs, undermining faithfulness and intervention guarantees. The authors present a simple approach for mitigating leakage, utilizing an Information Bottleneck regularizer.\n- Experimental details are verbose and the approach can be easily attached to prior works.", "weaknesses": "- Figure 3 does not really convey any information. The expanded results in the Appendix are useful but perhaps a table at different points along the x-axis would help for the main text?\n- Accuracy and intervention results both show minimal improvement. Concept leakage results seem like the strongest result, but are not explored in much detail. What does the improved OIS/NIS mean downstream if the intervention performance is not improved? Is there an interpretation of these two metrics that better quantifies the improvement of CIBMs?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2OWWElvNzO", "reviewer": "Reviewer_L5tE", "summary": "The paper introduces Concept Information Bottleneck Models, which incorporate a loss term aimed at reducing concept leakage in Concept Bottleneck Models. The objective is to maximise task-relevant information encoded in the concepts while suppressing task-irrelevant information present in the input X. The proposed regulariser is applied on top of standard CBM-like architectures. Through a series of experiments, the authors demonstrate that this additional term mitigates leakage.", "questions": "- It would be interesting to explore how the proposed method behaves when the concept set is substantially reduced or incomplete (i.e., in cases where some degree of concept leakage becomes necessary to achieve good task performance). Such a setting would better approximate real-world conditions, where not all relevant concepts can be fully annotated.\n- While the paper focuses on minimising information leakage from the input to the output, it does not address intra-concept leakage. Prior works [1,2] have shown that certain concepts can be predicted from others, which may undermine the independence assumption among concepts. It would be valuable to investigate whether the proposed method can also mitigate this form of leakage, thereby further improving intervention responsiveness.\n\n---------------------------\n\n[1] Gabriele Dominici, Pietro Barbiero, Mateo Espinosa Zarlenga, Alberto Termine, Martin Gjoreski, Giuseppe Marra, & Marc Langheinrich (2025). Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning. In The Thirteenth International Conference on Learning Representations.\n\n[2] Moritz Vandenhirtz, Sonia Laguna, Ričards Marcinkevi\\vcs, & Julia E Vogt (2024). Stochastic Concept Bottleneck Models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The authors address a key limitation of CBMs (concept leakage) through a loss-based approach rather than architectural modifications. This design choice enhances the applicability of the proposed method, allowing it to be integrated with existing CBM variants. Notably, the models most susceptible to leakage (e.g., CEM) show the greatest improvement, further validating the effectiveness of the approach. The paper presents extensive experiments on standard benchmarks, applying the proposed regularisation to several widely used CBM-like models. Moreover, the paper is well written and easy to follow.", "weaknesses": "- The authors evaluate their method on datasets where the set of concepts is extensive and highly representative of both the task and the concept space. However, it would be interesting to assess how the approach performs in scenarios where some degree of leakage is necessary to solve the task. Such a setting could be simulated by systematically removing a subset of concepts (e.g., half) from the training set. This would better reflect real-world conditions, where annotating a complete set of concepts is often infeasible.\n- The proposed metrics (AUC and NAUC) do not appear to represent a clear novelty. They essentially quantify what prior works have already assessed qualitatively (CBM responsiveness to concept interventions) by expressing it numerically rather than through visual plots.\n- The claim of improved accuracy (Section 4.1) seems somewhat overstated. The observed gains are marginal and, arguably, accuracy should not be the primary focus of this work. It would be more appropriate to emphasise that the proposed regularisation maintains task performance while effectively reducing concept leakage, rather than suggesting accuracy improvements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)