Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations
Authors: Xinyue Xu
Yi Qin
Lu Mi
Hao Wang
Xiaomeng Li
Venue: ICLR 2024 poster (2024)
Keywords: Interpretability
Concepts
Energy-Based Model
Probabilistic Methods
Primary area: visualization or interpretation of learned representations
TL;DR: We introduce Energy-Based Concept Bottleneck Models as a unified framework for concept-based prediction, concept correction, and fine-grained interpretations based on conditional probabilities.

Abstract: Existing methods, such as concept bottleneck models (CBMs), have been successful in providing concept-based interpretations for black-box deep learning models. They typically work by predicting concepts given the input and then predicting the final class label given the predicted concepts. However, (1) they often fail to capture the high-order, nonlinear interaction between concepts, e.g., correcting a predicted concept (e.g., “yellow breast”) does not help correct highly correlated concepts (e.g., “yellow belly”), leading to suboptimal final accuracy; (2) they cannot naturally quantify the complex conditional dependencies between different concepts and class labels (e.g., for an image with the class label “Kentucky Warbler” and a concept “black bill”, what is the probability that the model correctly predicts another concept “black crown”), therefore failing to provide deeper insight into how a black-box model works. In response to these limitations, we propose Energy-based Concept Bottleneck Models (ECBMs). Our ECBMs use a set of neural networks to define the joint energy of candidate (input, concept, class) tuples. With such a unified interface, prediction, concept correction, and conditional dependency quantification are then represented as conditional probabilities, which are generated by composing different energy functions. Our ECBMs address both limitations of existing CBMs, providing higher accuracy and richer concept interpretations. Empirical results show that our approach outperforms the state-of-the-art on real-world datasets.

Published as a conference paper at ICLR 2024
ENERGY-BASED CONCEPT BOTTLENECK MODELS: UNIFYING PREDICTION, CONCEPT INTERVENTION, AND PROBABILISTIC INTERPRETATIONS
Xinyue Xu', Yi Qin', Lu Mi, Hao Wang" Xiaomeng Li
'The Hong Kong University of Science and Technology. 'University of Washington, Rutgers University, 'Equal advising [xxucb, yoinar, eexmli fexmli]eust... Jeust. hk, mi willuee lu@uww.ww... lu@uu.edu,, edu, hwt.
ABSTRACT
Existing methods, such concept bottleneck models (CBMs), have been success- ful providing concept-based interpretations for black-box deep learning models. They typically work by predicting concepts given the input and then predicting the final class label given the predicted concepts. However, (1) they often fail to cap- ture the high-order, nonlinear interaction between concepts, e.g., correcting a pre- dicted concept (e.g.. "yellow breast") does not help correct highly correlated con- cepts (e.g.. "yellow belly"), leading to suboptimal final accuracy: (2) they cannot naturally quantify the complex conditional dependencies between different con- cepts and class labels (e.g., for an image with the class label "Kentucky Warbler" and concept "black bill", what is the probability that the model correctly pre- dicts another concept "black crown"), therefore failing to provide deeper insight propose
into how black-box model works. In response to these limitations, Energy-basee Concept Bottleneck Models (ECBMs). Our ECBMs use a set of neural networks to define the joint energy of candidate (input, concept, class) tuples. With such a unified interface, prediction, concept correction, and condi- tional dependency quantification are then represented conditional probabilities, which are generated by composing different energy functions. Our ECBMs ad- dress both limitations of existing CBMs, providing higher accuracy and richer concept interpretations. Empirical results show that our approach outperforms the state-of-thed -fettrrrrt on real-world datasets.
INTRODUCTION
Black-box models, while powerful, are often unable to explain their predictions in a way that is comprehensible to humans (Rudin, 2019). Concept-based models aim to address this limitation. Unlike traditional end-to-end models (Zhang et al., 2021) predicting output directly from input, concept-based models first predict intermediate concepts from input and then predict the final class labels from the predicted concepts (Koh et al., 2020; Kazhdan et al., 2020). These models aim emulate humans' cognitive process of distinguishing between different objects (e.g., zoologists classifying birds according to their heads, wings, and tails) by generating concepts that are visually comprehensible to humans as intermediate interpretations for their predictions.
Concept Bottleneck Models (CBMs) (Koh et al., 2020), as a representative class of models, operate by firstly generating concepts given the input and then using these concepts to predict the final label. The vanilla CBMs often fall short in final prediction accuracy compared to black-box models, creating a potentially unnecessary aerformance-intepprtaalltt trade-off (Rudin et al., 2022). To improve on such trade-off, Concept Embedding Models (CEMs) (Zarlenga et al., 2022) improve CBMs by including positive and negative semantics, while Post-hoc Concept Bottleneck Models (PCBMs) (Yuksekgonul et al., 2022) make of residual fitting to compensate for limitations in concept learning. Despite recent advances, existing CBM variants (including CEMs and PCBMs) still suffer from the following key limitations:

---

Published as a conference paper at ICLR 2024
1. Interpretability: They cannot effectively quantify the intricate relationships between various concepts and class labels (for example, in an image labeled "Kentucky Warbler", what the likelihood that the model accurately identifies the concept "black crown"). As result, they fall short of offering deeper understanding into the workings of a black-box model.
2. Intervention: They often struggle account for the complex interactions among concepts. Consequently, intervening to correct a misidentified concept (c.g., "yellow breast") does not necessarily improve the accuracy of closely related concepts (e.g., "yellow belly"). This limitation results in suboptimal accuracy for both individual concepts and the final class label. Performance: Current CBM variants suffer from a trade-off (Zarlenga et al., 2022) between model performance and interpretability. However, an ideal interpretable model should har- ness the synergy between performance and interpretability to get the best of both worlds.
In response to these limitations, we propose Energy-based Concept Bottleneck Models (ECBMs). Our ECBMs use a set of neural networks to deñne the joint energy of the input æ. concept c, and class label y. With such a unified interface, (I) prediction of the class label y. (2) prediction of concepts C. (i.e., all concepts except for oạ) after correcting concept O₁ for input æ, and (3) condi- tional interpretation among class label y, concept Ck. and another concept cg' all be naturally represented as conditional probabilities p(y]=)), ) p(c, |2. ct), and p(ck ly, ctr), respectively; these probabilities then easily computed by composing different energy functions.
We summarize our contributions as follows:
Beyond typical concept-based prediction, we identify the problems of concept correction and conditional interpretation as valuable tools to provide concept-based interpretations. We propose Energy-based Concept Bottleneck Models (ECBMs), the first general method to unify concept-based prediction, concept correction, and conditional interpretation as condi- tional probabilities under joint energy formulation.
With ECBM's unified interface, we derive a set of algorithms to compute different conditional probabilities by composing different energy functionn.
Empirical results show that our ECBMs significantly outperform the state state-of-thee -of-the-art on real- world datasets. Code available at https//gtthhh.coo/mmeeeaa//////
2 RELATED WORK
Concept Bottleneck Models (CBMs) (Koh et al., 2020; Kumar et al., 2009; Lampert et al., 2009) use feature extractor and a concept predictor to generate the "bottleneck" concepts, which are fed into a predictor to predict the final class labels. Concept Embedding Models (CEMs) (Zarlenga et al., 2022) build on CBMs to characterize each concept through a pair of positive and negative concept embeddings. Post-hoc Concept Bottleneck Models (PCBMs) (Yuksekgonul et al., 2022) use a post-hoc explanation model with additional residual fitting to further improve final accuracy. Probabilistic Concept Bottleneck Models (ProbCBMs) (Kim et al., 2023) incorporate probabilistic embeddings to enable uncertainty estimation of concept prediction. There are a diverse set of CBM variants (Barbiero et al., 2023; 2022; Havasi al., 2022: Ghosh et al., 2023a;b; Yang et al., 2023; Sarkar et al., 2022; Oikarinen et al.. 2023), each addressing problems from their unique perspectives. This diversity underscores the vitality of research within this field.
Here we note several key differences between the methods above and our ECBMs. (1) These ap- proaches are inadequate at accounting for the complex, nonlinear interplay among concepts. For example, correcting a mispredicted concept does not necessarily improve the accuracy of related concepts, leading suboptimal final accuracy. (2) They cannot effectively quantify the complex con- ditional dependencies (detailed explanations in Appendix C.4) between different concepts and class labels, therefore failing to offer conditional interpretation on how black-box model works. In contrast, our ECBMs address these limitations by defining the joint energy of candidate (input, con- cept, class) tuples and unify both concept correction and conditional interpretation as conditional probabilities, which are generated by composing different energy functions.
Energy-Basse Based Models (LeCun et al., 2006; Tu et al., 2020; Deng et al., 2020; Nijkamp et al., 2020) leverage Boltzmann distributions to decide the likelihood of input samples, mapping each sample to a scalar energy value through an energy function. The development of energy- models have been signficantly influenced by pioneering works such as (Xie et al., 2016) and (Xie et al., 2018).
2

---

Published as a conference paper at ICLR 2024
Beyond classification (Li et al., 2022; Grathwohl al., 2019), energy-based models have also been applied to structured prediction tasks (Belanger & McCallum, 2016; Rooshenas et al., 2019; Tu & Gimpel, 2019). Xie et al. and Du et al. use energy-basee models for the distribution of data and labels, which also capture concepts. These methods use energy functions to improve prediction performance, but cannot provide concept-based interpretations. In contrast, our ECBMs estimate the joint energy of input, concepts, and class labels, thereby naturally providing comprehensive concept-based interpretations that align well with human intuition.
Unsupervised Concept-Based Models, unlike CBMs, aim to extract concepts without concept an- notations. This is achieved by introducing inductive bias based on Bayesian deep learning with probabilistic graphical models (Wang et al., 2019; Wang & Yeung, 2016; 2020; Wang & Yan. 2023; Xu et al., 2023), causal structure (L.in et al., 2022), clustering structure (Chen et al., 2019; Ma et al., 2023), generative models (Du et al., 2021; Liu et al., 2023a) or interpretability desiderata (Al- varez. Melis Jaakkola, 2018).
ENERGY-BASED CONCEPT BOTTLENECK MODELS
In this section, we introduce the notation, problem settings, and then our proposed ECBMs in detail. Notation. We consider supervised classification setting with N data points, K concepts, and M classes, namely D = (ar!), cld), y where the j-th data point consists of the input (5 E X, the label g0و E y c {0, 1MMM and the concept c( E C = [0,1)* note that y is the space of M- M-dimensional one-hot vectors while C is not. We denote ym E y the M -ddimensionall one-hot vector with the m-th dimension set to 1, where m {1,.... M}.
(3) denotes the k-th dimension of
Ck
the concept vector cli), where & {l,.... K}. We denote (c.")
(5) for brevity. A pretrained backbone neural network F: : X' + 2 is used to extract the features z E 2 from the input æ E X. Finally, the structured energy network Eo(:,)) parameterized by 9, maps the (x. y), (æx c), or (c. y) to real-valued scalar energy values. We omit the superseript (i) when the context is clear.
C.
Problem Settings. For each data point, we consider three problem settings:
1. Prediction (c( y ()). This is the typical setting for concept- based models; given the input 2, the goal is to predict the class label y and the associated concepts c to interpret the pre- dicted class label. Note that CBMs decompose p(c, y/(z)) to predict p(c) (z) and then p(v c). (e.g.. p(c.t (20, cx)). Given the input x and a corrected
2. Concept Correction/Intereenteoe concept Ck. predict all the other concepts C.
3. Conditional Interpretations (Wang et al., 2019) (e.g., p(cly) or p(ck ly, c). Interpret the model using conditional probabilities such as p(c) l,, cw) (i.e., given an image with class label and concept ck', what is the probability that the model correctly predicts concept ct)).
3.1 STRUCTURED ENERGY-BASER CONCEPT BOTTLENECK MODELS
Overview. Our ECBM consists of three energy networks collectively parameterized by O: (1) class energy network crglas(xx y) that measures the compatibility of input æ and class label y, (2) concept energy network Econcept (x, e) that measures the compatibility of input æ and the K concepts c, and (3) global energy network Fg Eglobal (c,y) that measures the compatability of the K concepts e and class label The class and concept energy networks model class labels and concepts separately; labels and concepts. For all three energy networks, lower energy indicates better compatibility. ECBM is trained by minimizing the following total loss function:
contrast, the global energy network model the global relation between class
Call
(Ltotat c, y)]
Crotat c, Letnes (x,z) + >eooomoept c) + AggCttora (c,
(2)
where Lelase. Leoncept, and Cglobel denote the loss for training the three energy networks Eglass (x. y). Eg Econcept (xx c). and Eg Eggloall (c. y). respectively. de and by are hyperparameters. Fig. shows overview of our ECBM. Below we discuss the three loss terms (Eqn. 1) in detail.
Class Energy Network Eglass (x. y). In our ECBM, each class m is associated with a trainable class embedding denoted un- As shown in Fig. (ttpp), given the input x and candidate label
3

---

Published as a conference paper at ICLR 2024
Training
Concept Emrheskings
A
Concepts
Feature Embeddinn
min
Ep E (a,e)
Contegp Energy
Input Image
M
Fecture Extractor
Eg (c.y)
F
Clasa Embeediing
Class
Concept Class Energy
Big (ar,
Truinable Frowen
y
Class Energy
14
Inference
Concept Fonhesklings
V
Probability
Concapt
Feature Embedding 2
min
Eg (ze Concept Energy
Input Image
min
Feature F
Eg (c.y) Canceyi-lass Eacrgy
Class Embeekiings
Probability
Eight (2r,y)
Class Enerrge
Figure I: Overview of our ECBM. Top: During training, ECBM learns positive concept embed- dings o (+) (in black), negative concept embeddings e (in white), the class embeddings um (in black), and the three energy networks by minimizing the three energy functions, Eglas* y), Fig Francept (x. c), and Fig Figlobal (c, using Eqn. 1. The concept and class label y are treated as con- stants. Bottom: During inference, we (I) freeze all concept and class embeddings as well all networks, and (2) update the predicted concept probabilities è and class probabilities ÿ by minimiz- ing the three energy functions using Eqn.
y. the feature extractor F first compute the features = F(x). We then feed y's associated class label embedding u along with the features z into a neural network Gzu(z, u)) to obtain the final Felass (r. y). Formmaly we have,
Fig Filaas (x. y) Gz,(z,
where Gzu(-.)) a trainable neural network. To train the class energy network, we use the Boltz- mann distribution to define the conditional likelihood of given input x::
exp( Egians (w.,y)
Po po (z)
(4)
S exp( Enginn (I..
where the denominator serves normalizing constant. Vm y one-hot vector with the m-th dimension set to The class energy network Eglasa (xc y) is parameterized by 0; it maps the input-
class pair (xc y) real-valued scalar energy. Our ECBM the negative log-likelihood as the loss function; for an input-class pair (æc, y):
Letass (x, y) log Po (3 a)) Egg Eeloss (x, log Σ M
Egg (a.
(5)
Concept Energy Network F (a., c). Our concept energy network Fig Feancept (xr, c) consists Eg Econcept (x),cu) cu) where k. (1,..., Each sub-network E pconcept (æt, en) measures the compatibility of the input æ and the k-th concept Cs E 10. Each concept k is associated with positive embedding ok
of K sub-networks,
(+) and a negative embedding u. We define the k-th concept embedding Us as a combination of positive and negative embeddings, weighted by the concept probability Ck, i.e., 84 Ct c. x and an concept Ck. the feature extractor first compute the features z = F(x). We then feed cæ
(+) (1 (1-ck) -x). of As shown in Fig. ((top), given the input
associated concept embedding (v(+)) ( if cæ and (vf-)) ()) if c along with the features into

---

Published as a conference paper at ICLR 2024
neural network to obtain the final Exmnsep
Eg
(z, Ck). Formally, have
a
booncept
(x, ct.) Gzu (z, vk);
where G-v(-:) -) is a trainable neural network. Similar to the class energy network (Eqn. 5), the loss function for training the k-th sub-network Eg Eooncepl (x, ck) is
((b) conceepe (xe, cx) = 0
peoncept (x, + log (Eeve(o))
Bconcept
(2.00)).
Therefore, for each input-concept pair (x, c), the loss function for training Eg
Enoncepi (x,c) is
(k) Leoncepe (2, c) = =>x k=1 concepp (I, Ck).
Global Energy Network Egglbbll (c, y). The class energy network learns the dependency between the input and the class label, while the concept energy network learns the dependency between the input and each concept separately. In contrast, our global energy network learns (1) the interaction between different concepts and (2) the interaction between all concepts and the class label.
Given the class label y and the concepts e [cu] we will feed y's associated class label em- bedding u along with e's associated K concept embeddings [mu]k= (1%- = = + ) if c 1 and (Uk = =/6 if cj into a neural network to compute the global energy Eg Eglobal (c. y). Formally, have
Eg Egglobal (c, = Gov ((u)ke u)
where :) is a trainable neural network. [ww]k- denotes the concatenation of all concept embeddings. For each concept-class pair (c, y), the loss function for training Eglobal (c, is
Fighobal (c, y) log < Em e -EEgiobal (e' yy =mm)).
Lglobal y) Fig
(10)
where c enumerates all concept combinations in the space C. In practice, we employ a negative sampling strategy to enumerate subset of possible combinations for computational efficiency.
Inference Phase. After training ECBM using Eqn. 1, we can obtain the feature extractor and energy network parameters 0 (including class embeddings [4mm==11 concept embeddings [e as well as the parameters of neural networks (s )) Gs and (c)-). During inference, we will freeze all parameters F and 0 to perform (1) prediction of concepts and class labels (Sec. 3.2), (2) concept correction/interventoon (Sec. 3.3), and (3) conditional interpretations (Sec. 3.4). Below we provide details on these three inference problems.
3.2 PREDICTION
To predict and y given the input x, freeze the feature extractor F and the energy network parameters and search for the optimal prediction of concepts ê and the class label ÿ as follows:
arg miggin (ataso(x, D) Acheoncepe (x. 2) + Ag y)
(11)
where Letase (.. Leoncept (.) and Lotobal(:,) the instance-level loss functions in Eqn. 5, Eqn. 8, and Eqn. 10, respectively. Since the second during inference, one only needs minimize the joint energy below:
second term these three loss functions remain constant
Eg
Ejoint
(I, & Eglass (Ix,) + Ae Eggoneept (x. (x,c) + Ag Eglohat
(c. y).
(12)
Therefore Eqn. 11 is simplified to arg mina.j Eyoint (x, C,,,C,1 C, T. To make the optimization tractable, we relax the support of è from 1) to [0, 1] similarly we relax the support of ÿ from ) C {0, 1 to [0, 1 (with the constraint that all entries of ÿ sum up to 1). We use backpropagation to search for the optimal é and ÿ. After obtaining the optimal č and ÿ. round them back to the binary vector space (0, 1} and the one-hot vector space y as the final prediction. More details are provided in Algorithm 1 of Appendix B. Comprehensive details about the hyperparameters used in this work can be found in Appendix B.I. Additionally, we present an ablation study that analyzes hyperparameter sensitivity in Table of Appendix C.2.
5

---

Published as a conference paper at ICLR 2024
3.3 CONCEPT INTERVENTION AND CORRECTION
Similar to most concept-based models, our ECBMs also supports test-time intervention. Specifi- cally, after ECBM predicts the concepts and class label y. practitioners can examine e and y to intervene on some of the concepts (e.g.. correcting an incorrectly predicted concept). However, existing concept-based models do capture the interaction between concepts; therefore correcting a concept does not help correct highly correlated concepts, leading to suboptimal concept and class aceuracy. In contrast, our ECBMs are able to propagate the corrected concept(s) to other correlated concepts, thereby improving both concept and class accuracy. Proposition 3.1 below shows how our ECBMs automatically correct correlated concepts after test-time intervention and then leverage all corrected concepts to further improve final classification accuracy.
Proposition 3.1 (Joint Missing Concept and Class Probability). Given the ground-truth values of 1
concepts [eu]k=n" the joint probability of the remaining concepts [c -++1 and the class label can be computed as follows:
plcck-- --rk---++1 Bla, (ck)==1)))
(0.0.9)
(13)
Σ MM
Σ o fto 68
gninnt (0.0.um))
where Egoint (xe, c, c) is the joint energy defined in Eqn. 12.
3.4 CONDITIONAL INTERPRETATIONS
ECBMs are capable of providing a range of conditional probabilities that effectively quantify the complex conditional dependencies between different concepts and class labels. These probabilities can be represented by energy levels. For example, Proposition 3.2 below computes p(ckly) to interpret the importance the concept сę to a specific class label y in an ECBM.
Proposition 3.2 (Marginal Class-Specific Concept Importance). Given the target class y, the marginal concept importance (significance of each individual concept) can be expressed as:
Egiornl (e,w)
Σn
(c.
(14)
p(ckly)=
C.N.
Σe
Eg
(w.3)) .((x)
where e represents the full vector of concepts and can be broken down into [ck, c-k].
Proposition 3.2 above interprets the importance of each concept c separately. In contrast, Proposi- tion 3.3 below computes the joint distribution of all concepts p(cly) identify which combination of concepts e best represents a specific class
Proposition 3.3 (Joint Class-Specific Concept Importance). Given the target class y, the joint concept importance (significance of combined concepts) can be computed as:
Eg gglobal (e.w)
Σx
EM Egglbbll (c. [e" x "6
neancent (m.ok) ( p(()))
(15)
p(cly) x
(a (o.)
ECBMs can also provide interpretation on the probability a correct concept prediction Ck2 given the class label and another concept сg- This is computed as p(ck/ow y) using Proposition 3.4 below. This demonstrates our ECBM's capability to reason about additional concepts when we have knowledge of specific labels and concepts.
Proposition 3.4 Class-Specifie Conditional Probability among Concepts), Given a concept la- bel ev and the class label y, the probability of predicting another concept Ck is:
globab (c:w)
(e MN, E ggoncep (w) (e p(())
Eis (1 2 Ex
-1
(c.um)
p(cx/Cw, v) x
Eg Lilna (o.
EN
Eg geoncept (e.01) )p(())))(())))))))))))))))))))))
(e
E (c.y..)
2mm=
Proposition 3.5 computes the conditional probability of one concept given another concept p(ck(ok) which interprets the interaction (correlation) among concepts in ECBM.

---

Published as a conference paper at ICLR 2024
Table 1: Accuracy Different Datasets. We report the mean and standard deviation from five runs with different random seeds. For ProbCBM (marked with "*",, we report the best results from the ProbCBM paper (Kim et al., 2023) for CUB and AWA2 datasets.
Model
Dam
OUB
Colen
AWAZ
Mari
Concret
Concept
Ovenal
Class
Connapt
Cossept
Ovecall
Clas
Cunaupi
Concept
Ovzrall
Class
0,964
0,070
0.759
0.37
0,581 0.006
0.246
0.005
000022
0.800
=00004
PCBM
0,635
0.150
0,010
0.862 0.003
09955
0.396
0,05
0.796
0.867
0.001
0.457
0.005
0.330
00000
0:973
0.000
0,796
0.000
BCRM
0L933
0.01
0.713 1 0.009
0.822
0.06
0.876
0.000
0.478
0.000
0.343 0.0000
0,979
: 0.000
0.854
0.000
0,912 8 0.000
Proposition 3.5 (Class- Conditional Probability among Concepts). Given one concept Ck. the conditional probability of another concept CN' can be compuated as:
globat (c.w)
1 E Pleg1
JAL E10,1)K
globbal (o.Vm))
le 2i=1 Fo
p(aeleg))
ER Eomery (miee) (m.e)) p((e))
20m1 Ligg 3 -101)--- Ee
glonnt (c.30m)
Besides global interpretation above, ECBMs can also provide instance-level interpretation. For ex- ample, Proposition A.2 in Appendix A shows how ECBMs reason about the conditional probability of the class label y given the input æ and a known concept Cj. More ECBM conditional interpre- tations and all related proofs are included in Appendix A.
4 EXPERIMENTS
In this section, we compare our ECBM with existing methods on real-world datasets.
4.1 EXPERIMENT SETUP
Datasets. We evaluate different methods on three real- world datasets:
Caltech-UCSD Birds-200-2011 (CUB) (Wah et al., 2011) is fine-grained bird classification dataset with 11,788 images, 200 classes and 312 annotated attributes. Following CBM (Koh et al., 2020), ProbCBM (Kim et al., 2023) and CEM (Zarlenga et al., 2022), we select 112 attributes as the concepts and use the same data splits
Animals with Attributes 2 (AWA2) (Xian et al., 2018) is zero-shot learning dataset con- taining 37, ,22 images and 50 animal classes. We use all 85 attributes as concepts.
Large-scale CelebFaces Attributes (CelebA) (Liu et al., 2015) contains 200, 000 images, each annotated with 40 face attributes. Following the setting in CEM (Zarlenga et al., 2022), we use the 8 most balanced attributes as the target concepts and 256 classes for the classification task.
Baselines and Implementation Details. We compare our ECBM with state-of-the-art methods, i.e.. concept bottleneck model (CBM) (Koh et al., 2020), concept embedding model (CEM) (Zarlenga et al.. 2022), post-hoc concept bottleneck model (PCBM) (Yuksekgonul et al., 2022), and proba- bilistic concept bottleneck model (ProbCB.)) (Kim et al., 2023). We use ResNet101 (He et al., 2016) as the feature extractor F for all evaluated methods. We use the SGD optimizer during the training process. We use A = 0.3 and Ag = 0.3. For the propositions, we have implemented a hard version (yielding 0/1 output results) for computing probabilities. See Appendix B for more details.
Evaluation Metrics. With (z)(), c(ع) ) as the dataset, we denote as (20), و the
N j=1 (1) model prediction for concepts and class labels. Cg and dه is the k-th dimension of cli) and 20,)
respectively. We use the following three metrics evaluate different methods.
Concept Accuracy evaluates the model's predictions for each concept individually:
Coce = "لح Mim 2(c 222)//KK),,
(16)
where 1(-) is the indicator function.
Overall Concept Accuracy evaluates the model's ability to correctly predict all concepts for each input x()) Higher overall concept accuracy indicates the model's ability to mine the latent correla- tion between concepts for a more accurate interpretation for each concepts. It is defined as:
Coverall EY 1(c()=80))
(17)
7

---

Published as a conference paper at ICLR 2024
Concept Accuracy
Concept.Accurec) Accuracy
Accuracy
Group Intervention
CRM
1.62
1.00
cuB dataset
5555
Accuracy
333
3331
0522
0465
3.155
Individul nntervetion
Ce ebu dalaset
0382
0.42
Accuracy
0.00
6332
1150
6225
0.2
0.4
0.5
0.0
Ind Intervention
1.00
3.000
ANN12 delaset
Accuracy
01900
2.05
6400
04
0.6
1.
-44 Intervention Ratio
26
Intervention Ratio
Intervention Ratio
Figure 2: Performance with different ratios of intervened concepts on three datasets (with error bars). The intervention ratio denotes the proportion of provided correct concepts. We use CEM with RandInt. CelebA and AWA2 do not have grouped concepts; thus we adopt individual intervention.
Class
Footed
Sooty Albatros
Blackend White Wirbler
Kentuday Warbler
Image
Eye
Eye
Dye
Brosit oolon--ydlow (Ous))
Concept
pottren patnen-sedll Rall (Ounda)
gettens (Cundee)
Eyenzilon
tall
Broast oxton--pdllow
colocavdioo
Importance
colon.black (Crade
colon.vdioo
pattrresalld (Curr)
Loleepatts (Oun)
pabarresalld (Chncia)
Uippappats coleecyylooo (Chncko
Figure 3: Marginal concept importance (p(c* 1/3))) for top 3 concepts of 4 different classes com-
puted using Proposition 3.2. ECBM's estimation (Ours) very close to the ground truth (Oracle).
Class Accuracy evaluates the model's prediction accuracy for the class label:
Aace Σÿ =g)0)))00)))))///
(18)
4.2
RESULTS
Concept and Class Label Prediction. Table shows different types of accuracy of the evaluated methods. Concept accuracy across various methods is similar, with ECBM slightly outper- forming others. Interestingly, ECBM significantly outperforms other methods in terms of overall concept accuracy, especially in CUB (71.3% for ECBM versus 39.6% for the best baseline CEM); this shows that ECBM successfully captures the interaction (and correlation) among the concepts, thereby leveraging one correctly predicted concept to help correct other concepts' prediction. Such an advantage also helps improve ECBM's 's class accuracy upon other methods. We have conducted an ablation study for each component of our ECBM architecture (including a comparison with tra- ditional black-box models) in Table 4 of Appendix C.2, verifying our design's effectiveness.
Concept Intervention and Correction. Problem Setting in Sec. and Proposition 3.1 introduce the scenario where a practitioner (e.g., a clinician) examine the predicted concepts (and class labels) and intervene on (correct) the concept prediction. An ideal model should leverage such intervention to automatically correct other concepts, thereby improving both interpretability and class prediction accuracy. Additional experiments (for the background shift dataset (Koh et al., 2020)) in the Ap- pendix C.3 demonstrate the potential of our ECBM to enhance the robustness of CBMs. Fig. 2 shows three types of accuracy for different methods after intervening on (correcting) different proportions of the concepts, i.e., intervention ratios. In terms of both concept accuracy and overall concept
8

---

Published as a conference paper at ICLR 2024
(A) Joine class-specifi concepts Prediction (L Emor 000331
(b) Conditional chas-spaciflc concapts Prediction (11 Enroi: 0.006))
(4) Conditional cass-agnesdic concapes
Prediction (LT Emor 0,00171
15
20
10
15
100
15 20
10
10
10
04
15
02
15-
62
15
02
0.0
00
00
Ground Truth
Ground Truth
Ground Truth
0
10
15
20
1.0
0-
10
15
1.0
0.
10
15
20
10
10-
04
04
15
02
15
02
15
02
00
20
70
00
Figure 4: We selected the class "Black and White Warbler" in CUB for illustration. (a) Joint class- specific concept importance p(cy = Lx = 1ly) for ECBM's prediction and ground truth derived from Proposition 3.3, (b) Class-specific conditional probability among concepts p(ck 1/e y, for ECBM's prediction and ground truth derived from Proposition 3.4. (c) Class-agnostic conditional probability among concepts p(ck 1/ck 1) for ECBM's prediction and ground truth derived from Proposition 3.5.
accuracy, we can see that our ECBM outperforms the baselines across all intervention ratios. In terms of class accuracy. ECBM underperforms the vanilla CBM and the state-of-the-arr CEM (with RandInt); this is because they have striet concept bottlenecks, and therefore even very few correct concepts can significantly improve class aceuracy. Note that the primary focus of our ECBM is not class accuracy enhancement (detailed explanations and individual intervention on the CUB dataset (Fig. 12) can be found in Appendix C.5). We also provide further evidence demonstrating how our model can mitigate concept leakage in Fig. 11 of Appendix C.5.
Conditional Interpretations. Fig. shows the marginal concept importance (p(cx ly)) for top concepts of 4 different classes, computed using Proposition 3.2. Our ECBM can provide interpre- tation on which concepts are the most important for predicting each class. For example, ECBM correctly identifies "eye color:black" and "bill color:blacck" as top concepts for "Black and White Warble"; for similar class "Kentucky Warble", ECBM correctly identifies "breast color:yellow" and "belly color:yellow" as its top concepts. Quantitatively, ECBM's estimation (Ours) is very close to the ground truth (Oracle).
Fig, 4(a) and Fig. 4(b) show how ECBM interprets concept relations for specific class. We show results for the first 20 concepts in CUB (see Table 3 in Appendix C for the concept list); we include full results (ECBM, CBM and CEM) on all 112 concepts in Appendix C. Specifically, Fig. 4(a) shows the joint class-specific concept importance, i.e., p(cy' = 1, 1, = 1 (with as "Black and White Warble"), computed using Proposition 3.3 versus the ground truth. For example, ECBM cor- rectly estimates that for the class "Black and White Warble", concept "belly color" and "under tail color" have high joint probability; this is intuitive since different parts of bird usually have the same color. Similarly, Fig. 4(b) shows class-specifi conditional probability between different concepts, i... p(c* 1|ck. 1. (with y as "Black and White Warble"), computed using Proposition 3.4. Besides class-specific interpretation, Fig. 4(c) shows how ECBM interprets concept relations in gen- eral using conditional probability between concepts, i.e., p(ck ((w)), computed using Proposition 3.5. Quantitatively, the average LI error (in the range (0, 1)) for Fig. 4(a-c) is 0.0033, 0.0096, and 0.0017, respectively, demonstrating ECBM's accurate conditional interpretation.
5 CONCLUSION AND LIMITATIONS
In this paper, we go beyond typical concept- based prediction to identify the problems of concept correction and conditional interpretation as valuable tools to provide concept-based interpretations. We propose ECBM, the first general method to unify concept-based prediction, concept correction, and conditional interpretation as conditional probabilities under a joint energy formulation. Fu- ture work may include extending ECBM to handle uncertainty quantification using Bayesian neural networks (Wang& & Wang, 2023), enable unsupervised learning of concepts (Ma al., 2023) via graphical models within the hierarchical Bayesian deep learning framework (Wang & Yeung, 2016; 2020), and enable cross-domain interpretation (Wang al., 2020; Xu et al., 2022; Liu et al., 2023b).

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENT
The authors thank the reviewers/ACs for the constructive comments improve the paper. The authors are also grateful to Min Shi and Yueying Hu for their comments to improve this paper. This work is supported in part by the National Natural Science Foundation of China under Grant 62306254 and in part by the Hong Kong Innovation and Technology Fund under Grant ITS/O30/21. Xinyue Xu is supported by the Hong Kong PhD Fellowship Scheme (HKPFS) from Hong Kong Research Grants Council (RGC).
REFERENCES
David Alvarez Melis and Tommi Jaakkola. Towards robust interpretability with self-explaining neural networks. Advances in neural information processing systems, 31, 2018.
Pietro Barbiero, Gabriele Ciravegna, Francesco Giannini, Pietro Lió, Marco Gori, and Stefano Melacci. Entropy-based logic explanations of neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 6046-6054, 2022.
Pietro Barbiero, Gabriele Ciravegna, Francesco Giannini, Mateo Espinosa Zarlenga, Lucie Charlotte Magister, Alberto Tonda, Pietro Lio, Frederic Precioso, Mateja Jamnik, and Giuseppe Marra. Interpretable neural-symbolic concept reasoning. arXiv preprinl 1 14068, 2023.
David Belanger and Andrew McCallum. Structured prediction energy networks. In International Conference on Machine Learning, pp. 983-9993999 982.992.110 PMLR, 2016.
Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019.
Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc' Aurelio Ranzato. Residual energy-based models for text generation. arXiv preprint arXiv:2004.11714, 2020.
Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based mod- els. Advances in Neural Information Processing Systems, 33:6637-6647, 2020.
Yilun Du, Shuang Li, Yash Sharma, Josh Tenenbaum, and Igor Mordatch. Unsupervised learning of compositional energy concepts. Advances in Neural Information Processing Systems, 34:15608- 15620, 2021.
Shantanu Ghosh, Ke Yu, Forough Arabshahi, and Kayhan Batmanghelich. Dividing and conquering blackbox to mixture of interpretable models: route, interpret, repeat. In Proceedings of the International Conference on Machine Learning. International Conference on Machine Learning, volume 202, pp. 11360. NIH Public Access, 2023a.
Shantanu Ghosh, Ke Yu, and Kayhan Batmanghelich. Distilling blackbox interpretable models for efficient transfer learning, arXiv preprint arXiv:2305.173000 2023b.
Will Grathwohl, Kuan-Chieh Wang, Jörn- Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprini arXiv: 19t.::262..00..... 03263, 2019.
Marton Havasi, Sonali Parbhoo, and Finale Relei.Velez. Addressing leakage in concept bottleneck models. Advances in Neural Information Processing Systems, 35:23386-23397, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Dmitry Kazhdan, Botty Dimanov, Mateja Jamnik, Pietro Liò, and Adrian Weller. Now you see me (cme): concept-based model extraction. arXiv preprint arXiv:2010.13233 2020.
Eunji Kim, Dabuin Jung, Sangha Park, Siwon Kim, and Sungroh Yoon. Probabilistic concept bot- tleneck models. arXiv preprint arX1v:2306.0/574, 2023.
10

---

Published as a conference paper at ICLR 2024
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. International Conference on Machine Learning, pp- 5338-5348 PMLR, 2020.
Neeraj Kumar, Alexander C Berg, Peter N Belhumeur, and Shree K Nayar. Attribute and simile classifiers for face verification. In 2009 IEEE 12th international conference on computer vision, pp. 365-372. IEEE, 2009.
Christoph Lampert, Hannes Nickisch, and Stefan Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In 2009 IEEE conference computer vision and pattem recognition, pp. 951-958. IEEE, 2009.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang. A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
Shuang Li, Yilun Du, Gido van de Ven, and Igor Mordatch. Energy-based models for continual learning. In Conference on Lifelong Learning Agents, pp. 1-22. PMLR, 2022.
Wanyu Lin, Hao Lan, Hao Wang, and Baochun Li. Orphicx: A causality-nsspired latent variable model for interpreting graph neural networks. In CVPR, 2022.
Nan Liu, Yilun Du, Shuang Li, Joshua B Tenenbaum, and Antonio Torralba. Unsupervised composi- tional concepts discovery with text-to-image generative models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2085-2095, 2023a.
Tianyi Liu, Zihao Xu, Hao He, Guangyuan Hao, Guang-He Lee, and Hao Wang. Taxonomy- structured domain adaptation. ICML, 2023b.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730-3738, 2015.
Chiyu Ma, Brandon Zhao, Chaofan Chen, and Cynthia Rudin. This looks like those: Illuminating prototypical concepts using multiple visualizations. arXiv preprint arXiv:2310. 18589, 2023.
Erik Nijkamp. Mitch Hill, Tian Han, Song- Chug-ChunnnCa Zhu, and Ying Nian Wu. On the anatomy of meme-based maximum likelihood learning of energy- models. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pp. 5272-5280, 2020.
Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui-Wei Weng. Label-free concept bottle- neck models. arXiv preprint SeXiv:2330.061299 2023.
Amirmohammad Rooshenas, Dongxu Zhang, Gopal Sharma, and Andrew McCallum. Search- guided, lightly-supervised training of structured prediction energy networks. Advances Neural Information Processing Systems, 32, 2019.
Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1(5):206-215. 2019.
Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. In- terpretable machine learning: Fundamental principles and 10 grand challenges. Statistic Surveys, 16:1-85, 2022.
Anirban Sarkar, Deepak Vijaykeerthy, Anindya Sarkar, and Vineeth N Balasubramanian. A frame- work for learning ante-hoc explainable models via concepts. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10286-10295 2022.
Lifu Tu and Kevin Gimpel. Benchmarking approximate inference methods for neural structured prediction. arXiv preprint 1004 2019.
Lifu Tu, Richard Yuanzhe Pang, Sam Wiseman, and Kevin Gimpel. Engine: Energy-based inference networks for non-autoregressive machine translation. arXiv preprint wwXiv:205500085/ 2020.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucs/ birds-200-2011 dataset. 2011.
11

---

Published as a conference paper at ICLR 2024
Hao Wang and Jingquan Yan. Self-interpretable time series prediction with counterfactual explana- tions. In JCML, 2023.
Hao Wang and Dit- Yan Yeung. Towards bayesian deep learning: A framework and some existing methods. TDKE, 28(12)3335--3488 2016.
Hao Wang and Yan Yeung. A survey on bayesian deep learning. CSUR, 53(5):1-37, 2020.
Hao Wang. Chengzhi Mao, Hao He, Mingmin Zhao, Tommi Jaakkola, and Dina Katabi. Bidi- rectional inference networks: A class of deep bayesian networks for health profiling. In AAAI, volume 33, pp. 766-773, 2019.
Hao Wang, Hao He, and Dina Katabi. Continuously indexed domain adaptation. ICML, 2020.
Ziyan Wang and Hao Wang. Variational imbalanced regression: Fair uncertainty quantification via probabilistic smoothing. NeurIPS, 2023.
Yongqin Xian, Christoph H Lampert, Bernt Schicle, and Zeynep Akata. Zero-shot learning comprehensive evaluation evaluattion of the good, the bad and the ugly. IEEE transactions on pattern analysis and machine intelligence, 41(9):2251 2265, 2018.
Jianwen Xie, Yang Lu, Song-Chun Zhu, and Yingnian Wu. theory of generative convnet. In International Conference Machine Learning, pp. 2635-2644, PMLR, 2016.
Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chun Zhu, and Ying Nian Wu. Cooperative training of de- scriptor and generator networks. IEEE transactions on paltern analysis and machine intelligence, 42(1):27-45, 2018.
Jianwen Xie, Zilong Zheng, Xiaolin Fang, Song-Chun Zhu, and Ying Nian Wu. Cooperative training of fast thinking initializer and slow thinking solver for multi modal conditional learning. arXiv preprint arXiv: 1902.022220 2019.
Zihao Xu, Guang-He Lee, Yuyang Wang, Hao Wang, et al. Graph-relational domain adaptation. In ICLR, 2022.
Zihao Xu, Guangyuan Hao, Hao He, and Hao Wang. Domain indexing variational bayes: Inter- pretable domain index for domain adaptation. In ICLR, 2023.
Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison- and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable im- age classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19187-19197, 2023.
Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. arXiv preprint arXiv:2205. 2022.
Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Gian- nini, Michelangelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, et al. Concept embedding models. arXiv preprint arXiv:2209.0908: 2022.
Tian Zhang, Dongliang Chang, Zhanyu Ma. and Jun Guo. Progressive co-attention network for fine- fine-graing grained visual classification. In 2021 International Conference on Visual Communications and Image Processing (VCIP), pp. 1-5, IEEE, 2021.
A ADDITIONAL CONDITIONAL INTERPRETATIONS AND PROOFS
Given the input x and label y. we propose to use the Boltzmann distribution to define the conditional likelihood of label y given a::
exp( Expp-EEgiaay Exgiazaaaaa..
(
(19)
ΣM exp( - Eggaae (zz.ym))
where ym )' one-hot vector with the m-th dimension set 1.
12

---

Published as a conference paper at ICLR 2024
We use the negative log-likelihood the loss function for an input-class pair (æ, y), and the panded form is:
Lelasa (2. y) = - log po (y)
exp( EE6 (1n
log Σ exp( Eglassg (2
M
log (exp (o (-Eglase (xz, w))) + log
Σ m=
exp (- -Eg
(x, ym))
(20)
Lelass (x, y) log
Eglane (I.
Thus, we can have:
p(y x) x e (pg'a"((x.1))
(21)
which connect our energy function Eglass (x, y) to the conditional probability p(y(æ)).
Similarly, we denote local concept energy of the energy network parameterized by between
input æ and the k-th dimension of concept C as Fo Econcept (x). We can obtain:
p(ck x)) & e E Eg (x.cx))
Enoncspl
(22)
We denote the global concept-class energy of the energy network parameterized by between input e and the label y as Eng Egilbbll (c, y). Similarly, we then have:
8 pglloba
plylc)
(23)
ΣM Fg tobao (c.um)
Similarly, with the joint energy Eqn. 12, we have
2o E sgoine (e.c.um)
plx,c,y) EM
(24)
Proposition 3.1 (Joint Missing Concept and Class Probability). Given the ground-truth values of concepts [e the joint probability of the remaining concepts [ck]k= K-4+1 and the class label can be computed as follows:
K( y/a, [c
K-*)
28
goint (w.c.))
(13)
Em=r E Eenk Ec (e
By
conin (0.c.mm)
where Fig
Fojint (x., c. c) is the joint energy defined in Eqn. 12.
Proof. By definition of joint energy, we have that
p([ck)<= -+11: y x. (c)) x (a. (a.c.))
K
(25)
Therefore by Bayes rule, we then have
proint (1.0.))
p([cklk K R - u/a, [ck (====
(26)
MM 2i onl
(e ng (x.e
concluding the proof.
We initially establish the Joint Class Class-Specific Concept Importance (Proposition 3.3), which we sub- sequently employ to marginalize e₁ and demonstrate the Marginal Class-Specific Concept Impor- tance (Proposition 3.2).
Proposition 3.3 (Joint Class-Specific Concept Importance). Given the target class y, the joint concept importance (significance of combined concepts) can be computed as:
Eg gllobal (c.v)
-e 2 gcancent (s.ck) )p(())
Σn
Ema Eg
Egtobnl (c.ym)
(15)
p(cly) x
Σg
10 Saa (mn)
13

---

Published as a conference paper at ICLR 2024
Proof. Given Eqn. 21, Equ. 22 and Eqn. 23, we have
p(cly)
p(yle)pie)
p(U)
Σn plee)))
p(P)
Σn p(ylle)
p(M)
(c.w)
(27)
Σ.
p(cx)=)))p(())
M
obot
c.
Ply i p((z)
Rofobol (c.0)
EE
rouncepp (m.
m
Eg
c.
Σn.
(a.3)) p(x)
where
plu)
Σ p(ar, y)
Σ (3) p(x)
Σ e
Edlea (2.9) (2.0) p(x),
(28)
or
Σ plylc) p(c)
Σ
Eg zglobal (c.0)
p(c),
M
Fig
Eallobut (c.ymm)
concluding the proof.
Proposition 3.2 (Marginal Class-Specific Concept Importance). Given the target class the marginal concept importance (significance of each individual concept) can be expressed as:
Σn
Egiobal (e.w))
Eg
{e ER sgoneent
(n pp(())
3f
moat
(c.
(14)
p(ckly) Σen
Σe (m -p(x))
Proof. Given Eqn. 21, Eqn. 22, Eqn. 23 and Proposition 3.3, marginal class-specific concept impor- tance is marginalize E c of Proposition 3.3, we have
p(culy)= Σ p(ckk cxly)
Eg global (c.1)
(e EB_ Fg Egeecct (argg)) p(x)
(29)
Σn
mmz Eg
Eglobal (c.)mm)
En (2)) p(x)
concluding the proof.
Proposition 3.4 (Class-Specific Conditional Probability among Concepts). Given a concept la- bel cu and the class label y, the probability of predicting another concept Cg is:
R Ralobal (c.a)
{e Eim pconeept ( (m.)))))p)))
Eis E 1 E(0.))x En
(c.ym)
p(cx/ow, v) x
pulobbl
(e EK (2))) )((())
DeylK -00,1)--1 En
Egglobll (c.ym))
5m=
14

---

Published as a conference paper at ICLR 2024
Proof. Given Eqn. 22, Equ. 23, and Proposition 3.3, have p(ck ck' y) p(ococll) plewr19) V)
Eigyl So a/ ((0.)) ple,, poo v)
=
2 51 ((0.)/= p(c,,,. p(c2,
(c.v)
E %
pooncept (m.)t))
Aig nbai
(c.
E (111 St0.)1
(30)
Exe
go (0. p(w)
x
(o) (0:2)
0 Egiobal
En
(c:Um))
Ex
(m..M) p(x))
Eglanat (c.w) e E
(e o (a))) p((x)
E1eg En
ggglbbll (c. Eg
Egiobol (elobol(e..))
Eie E10.// Ex
EM E Ealobal (c.
(e- > Fg (x.))))
concluding the proof.
Proposition 3.5 (Class-Agnostic Conditional Probability among Concepts). Given one concept Clo the conditional probability of another concept CN can be compuated as:
Eglobot (c.ss)
gronespt (m.re) pp(m))
EM 1,,,k 37 110,1) Ex
M
ggiobal (c.sm))
de
p(apley))
(e.y)
ie E ggoneept (m. p-p(s))
SM
En
Y- aaglobat (c.3m))
Proof. Given Proposition 3.3 and Propostion 3.4, we have
p(ckle))
plec ieg (
MM plea Mm (((m)))
DM=n Fey K Jan. ((01)) plau plorieg deyl pitk, & (ww) (lmm)
2mmi Sye ( plogiei ii ((hha
(31)
(e.v)
Ex 10 reancept p(o) pium)
> 2i
st/ke E(0. 2ie
mm
(e.wm)
gioba (c.w)
Ei Eg (goncep((w.4)) p(a))
Dm= Ekyy E/0.)) Kc En
3-m Ngtoball (0.0m))
concluding the proof.
Proposition A.1 (Missing Concept Probability). Given the ground-truth values of concepts K-a
[ck]k=1 the joint probability of the remaining concepts og can be computed as follows:
pooint e.mmm
p(ckk st |x. (=9)
MM
(32)
ΣM
E
(e Eg joint (e.e..))))))
where Ejoint (x, y) is the joint energy defined in Eqn. 12.
Proof. This follows directly after marginalizing y out from p[[ca] K-411 y/æ, [co))= in Propo- sition 3.1.
15

---

Published as a conference paper at ICLR 2024
Proposition A.2 (Conditional Class Probability Given Known Concept). Given the input and a concept Cj2 the conditional probability of label y is:
Log
2 -M P
(33)
ply x., (k)
F (0.0g))
Proof. Given Eqn. 22 and Eqn. 24, marginalize o E of Eqn. 24, we have
plyla ca)
p(y.a.ca)
p(w,ek)
Eeg p(y.0.c)
p(z,0c)
Een ply.o,cu,c..)
p(a,ow))
% (m..))
(34)
Ees y -m
plcn (()))
Lek
S
Min
X
5
(0.og)
concluding the proof.
B IMPLEMENTATION DETAILS
B.1 HYPERPARAMETERS
For fair comparison, we use ResNet101 (He et al., 2016) as the backbone and 299 x 299 as the input size for all evaluated methods, except CelebA with 64 64. We use the SGD optimizer to train the model. We use X_ 0.3. X 0.3, batch size 64, learning rate of x 10 10-2, and at most 300 epochs. the gradient inference process, we use the Adam optimizer, A 0.49, Ag = 0.004 (this is equivalent to applying ratio of 1:1:0.01 for the class, concept and global energy terms, respectively). batch size 64, and learning rate of 1 x 10-1, We run all experiments on an NVIDIA RTX3090 GPU, In order to enhance the robustness of the c-y energy head, we deployed perturbation augmentation when training ECBMs. Specifically, we perturbed 20% noisy pairs at probability p 0.2 the input of c-y energy head during the training phase. These not incorporated during the phases of inference and intervention.
Regarding hyperparameter A, 0.49, this is because in implementation, (12) in the paper, resulting in: a₁ A Ayglass(a (xx y) + >e Aggoeeent (x, e) >, Aggobal! (c. y). We then \ = 1, X = and Ag 0.01. This is therefore equivalent to setting X = 1111+1+0.01 0.49 for the original Eq (12) without by: Feclse( (xr y) + Xe proncept (æ. c) A Fginbal (c,
introduce X to Eq
16

---

Published as a conference paper at ICLR 2024
Algorithm 1 Gradient-Based Inference Algorithm
1: Input: Image æ. positive and negative concept embedding [v] - K and [e] 1
k=1 label em- weight parameters Ae, >g. learning rate 17, concept and class number K, M.
bedding
2: Output: Concept and label probability 2, ÿ
3: Initialize un nn-normalized concept probability č
4: Initialize un un-nommalized class probability y.
5: while not converge do
6: ē < Sigmoid ()
7: y Softmar
8:
for k + 1 To K do
9:
UR= Cx x Uk (1 -1) Uk
10: end for
for m To M do
12:
um t Fm x Um
13:
end for
14:
Calculate Lelass (xc, 1) based on Eqn. 5.
15:
Calculate Lconcept ( 2) based on Eqn.
16:
Calculate Lglobal C, y) based Eqn. 10.
17:
Ltotal = Lelass + de Scooncep + Aggglobal
18:
ct V Liotal
19: y + y V/ Leotal
20: end while
21: C < Sigmoid(()
D - --oottmar())
23: return č. C
B.2 ENERGY MODEL ARCHITECTURES
Table 2 shows the neural network architectures for different energy functions.
Table 2: Energy Model Architecture.
(a) æ-y energy network.
(b) æ-c energy network.
(c) c-y energy network.
z FeatureExtractor(z)
x FeatureExtractor((z)
v Embedding(e)
z FC(Input, hidden) (2)
2 PC(Input, hidden) (z)
Embedding(y)
2 Dropout(p=0.22 (z)
Dropout(p=0.2) (+)
cy u Norm2(v) u
u Embedding(y))
v Embedding(e)
Relu (cy)
* Norm2(u)
Norm2(v)
energy FC(hidden, 1) (cy)
z Relu (x)
Relu (z)
energy FC(hidden, 1) (z)
energy FC(hidden, 1) (z)
C MORE RESULTS
C.I VISUALIZATION RESULTS
We selected the ground truth and prediction results of the class "Black and White Warbler" in CUB dataset as a representative case for visualization. We provide the complete 112 concepts heatmaps of joint class-specific of concepts importance (p(cyr
1, 1/y)), class-specific conditional probability among concepts p(ck = 1|cr = 1, 1,y) and class agnsotic conditional probability among concepts (p(ck 1|ck/ 1)) in Fig. 5, Fig. 6 and Fig. 7, respectively. Note that p(ck 1, C 1 1 = leads to p(c; 1/ck- = 1, y): since most entries (p(cer = 1, C₁ = 1ly)) in Fig. 4(a) are close to so is Fig. 4(b). Table 3 shows the 20 specific concept names used in Fig. 4.
17

---

Published as a conference paper at ICLR 2024
30
10
20
30
40
50
60
70
80
90
100
110
10
20
0.8
0.8
30
30
0.6
40
0.6
50
50
60
0.4
70
70
80
90
-0.2
90
100
(a) Ground Truth
(b) Prediction
Figure 5: Joint class specifcc of concepts importance heatmap (p(cer 1. ground truth and prediction derived from Proposition 3.3.
1ly)) for ECBM's
10
20
30
40
50
60
70
80
90
100
110
10
20
30
50
60
70
80
90 100
110
0.8
30
30
40
40
60
70
0.4
70
0.4
80
90
100
110
0.0
110
0.0
(a) Ground Truth
(b) Prediction
Figure 6: Class-specific conditional probability among concepts heatmap p(ck = 1|cer = 1, y) for ECBM's ground truth and prediction derived from Proposition 3.4.
10
20
30
40
50
60
70
80
90
100
110
10
20
30
40
50
60
90 100 110
0.8
20
0.8
30
30
40
40
60
60
70
0.4
70
0.4
80
80
100
100
110
0.0
110
0.0
(a) Ground Truth
(b) Prediction
Figure 7: Class-agnostic conditional probability among concepts heatmap (p(c: = 1/ck = 1)) for ECBM's ground truth and prediction derived from Proposition 3.5.
18

---

Published as a conference paper at ICLR 2024
Table 20 concepts shown in Fig. 4.
Index
Concept Name
has bill shape::curved (up or down)
has shapphoolsh
4
has bill shape:nnedle
5
has www.billshhhpeeehhooe.....aabaaa
6
has bail spaut::paauule
has mus_bbll_shappe silapee:alllpuupoos
has bill shape::cone
9
has bill shape:specialized
10
mas.wing.coolr:bblu
11
has wing color:brown
12
has wing color:iridescent
13
has..wing.color:purpi
14
has wing color::ufoous
15
has wing color:grey
16
has
17
has bas.wing coing.color:
18
has has_wing wwwg.cooooorgreenn
19
has gas__wnng.colo
20
has has hns.wwing.color:orragg
We further provide two baseline results (CBM and CEM) on the complete 1 112 concepts heatmaps Fig. 8, Fig. and Fig. 10, respectively.
10
20
50
60
90 100
110
1.0
0
1.0
10
10
20
40
0.6
0.6
50
50
60
80
90
90
0.2
100
110
(a) CBM's Prediction
(b) CEM's Prediction
Figure 8: Joint class-specific of concepts importance heatmap (p(ckr 1, ck 1ly)) for CBM's and CEM's predictions.
19

---

Published as a conference paper at ICLR 2024
10
20
40
50 60
70
90
100
110
10
20
40
50
60
80
90
100
11.0
0
1.0
0.8
30
40
40
0.6
0.6
50
0.4
80
90
0.2
90
0.2
100
(a) CBM's Prediction
(b) CEM's Prediction
Figure 9: Class-specific conditional probability among concepts heatmap p(ck = 1/ck = 1, y) for CBM's and CEM's predictions.
20
30
60
70
90
100
110
30
40
50
70 80 90 100 110
10
20
0.8
0.8
30
30
40
40
0.6
50
50
60
70
0.4
70
80
80
90
90
0.2
100
100
110
(a) CBM's Prediction
(b) CEM's Prediction
Figure 10: Class-agnostic conditional probability among concepts heatmap (p(ca = 1/c = 1)) for CBM's and CEM's predictions.
C.2 ABLATION STUDY
Table 4 shows the performance of single-branch ECBMs and the black-box model. Table shows the sensitivity of ECBM to its hyperparameters.
Table 4: Ablation study single-
single-branch branch ECBMs and the black-box model.
Data
CUB
CelebA
AWA2
Model
Overall Concept
Overall Concept
Overall Concept
Metric
Concept
Class
Concept
Class
Concept
Class
ECBM (ær only)
0.825
0.265
0.909
BCBM (z e y only)
0.968
0.680
0.726
0.464
0.175
0.979
0.864
0.905
ECBM
0.973
0.713
0.812
0.876
0.478
0.343
0.979
0.854
0.912
CBM CEM
0.964
0.364
0.759
0.837
0.381
0.246
0.979
0.803
0.907
0.965
0.396
0.796
0.867
0.457
0.330
0.978
0.796
0.908
Black-box
0.826
0.291
0.929
20

---

Published as a conference paper at ICLR 2024
Table 5: Hyperparameter Sensitivity Analysis.
Data
CUB
CelebA
AWA2
Hyperparameter
Overall Concept
Overall Concept
Overall
Metric
Concept
Class
Concept
Class
Concept
Concept
Class
0.01
0.971
0.679
0.756
0.872
0.456
0.166
0.979
0.854
0.907
À 0.1
0.971
0.680
0.795
0.872
0.456
0.322
0.979
0.854
0.908
À 1
0.973
0.713
0.812
0.876
0.478
0.343
0.979
0.854
0.912
A
0.971
0.679
0.872
0.455
0.854
A 3
0.971
0.679
0,808
0.872
0.455
0.326
0.979
0.854
0.911
A 4
0.971
0.679
0.806
0.872
0.455
0.327
0.979
0.854
0.911
0.01
0.971
0.679
0.799
0.872
0,456
0.329
0.979
0.854
0.912
X 0.1
0.971
0.679
0.799
0.872
01.456
0.166
0.979
0.864
0.912
Xe 1
0,973
0.713
0.812
0.876
0.478
0.343
0.979
0.854
0.912
0.971
0.679
0.455
X =
0.971
0.679
0.798
0.872
0.455
0.329
0.979
0.854
0.912
A 4
0.971
0.679
0.798
0.872
0.455
0.329
0.979
0.854
0.912
b 0.0001
0.971
0.679
0.805
0.872
0.455
0.326
0.979
0.854
A 0.001
0.971
0.679
0,805
0.872
0.455
0.326
0.979
0.854
0.911
b 0.01
0.973
0.713
0.812
0.876
0.478
0.343
0,979
0.854
0.912
0,1
0.971
0,680
0.795
0.872
0,456
0.322
0.979
0.854
0.909
0.971
0.680
0,756
0.872
0.456
0,166
0.979
0.854
0.907
C.3 ROBUSTNESS
We believe our ECBMs do potentially enjoy stronger robustness to adversarial attacks compared to existing CBM variants. Specifically, our ECBMs designed to understand the relationships between different concepts, as well as the relationships between concepts and labels. As a result, during inference, ECBMs can leverage these relationships to automatically correct concepts that may be influenced by adversarial attacks. Our preliminary results suggest that our ECBM can potentially improve the robustness against adversarial attacks compared existing CBM variants.
Table 6: Accuracy on TravelingBirds (Koh al., 2020) (background shift).
We report the CBM
results using Table 3 of (Koh et al., 2020).
Overall Concept
Model
Concept
Class
Standard Joint (CBM)
0.373
0.931
0.518
Sequential (CBM)
0.928
0,504
Independent (CBM)
0.928
0.518
ECBM
0.945
0.416
0.584
Furthermore, we conducted additional experiments on the TravelingBirds dataset following the ro- bustness experiments of CBM (Koh et al., 2020) concerning background shifts. The results (Table 6) reveal that our ECBM significantly outperforms CBMs in this regard. These findings underscore our model's ' superior robustness spurious correlations.
C.4 THE NOTION OF COMPLEX CONDITIONAL RELATIONS
Previous concept-based methods do not allow dencies (as mentioned the abstract), such as p(c p(cxly. Crr), and p(c-ks y æ. ct). Post- hoc CBMs and vanilla CBMs, with their interpretative linear layers, provide way to understand concept-label relationships. In fact, in PCBMs, the weights can indicate each concept's importance for a given label (p(cly))) However, these existing CBM variants cannot provide more complex conditional dependencies such as p(ck y. ci) and p(c-k, 2/2z ck). In contrast, our model, which relies on the Energy-Baan -Baed Model (EBM) structure, can naturally provide such comprehensive con- ditional interpretations.
understand "complex" conditional depen-
C.5 INTERVENTION
Leakage in Models.
We conducted experiments that address the leakage problem as described in (Havasi al., 2022). When fewer concepts were given in the training process, the model should
21

---

Published as a conference paper at ICLR 2024
not learn good as the with all concepts known because of less conceptual information. If the model holds strong performance regardless of concept quantities, is more likely to suffer from information leakage, i.c. learning concept-ireelevat irrelevant features directly from image input. This will potentially detriment the accountability of the model interpretation. During training, we provide different quantities of concept groups available for model to learn, and then test the models based on these available concepts. Fig. II shows the results. We found that our model performs gradually better when more concept groups are given during training process, instead of intervention-invarian high performances. This indicates that our model suffers less from information leakage, hence providing more reliable interpretations.
0.800
0.700
Accuracy
0.600
0.500
0.400
0.300
CEM
ECBM
0.200
0.2
0.4
0.6
0.8
Concepts ratio given during training
Figure 11: The predictive performance of CEM and hard ECBM on the CUB dataset. The horizontal axis denotes the ratio of concept groups given during the training process, and the vertical axis denotes the "Class Accuracy".
Individual Intervention on the CUB dataset. We intervened our model based on individual con- cepts of the CUB dataset. Fig. 12 shows the intervention results.
Individual Intervention for the CUB dataset
Concept Accuracy
Overall Concept Accuracy
Class Accuracy
1.000
1.000
1.000
0.955
0.980
Accuracy
0.990
0.900
0.960
0.975
0.850
0.540
CEM
0.870
0.965
0.8
Intervention Ratio
Intervention Ratio
Intervention Ratio
Figure 12: Individual intervention on the CUB dataset (with error bars). The horizontal axis denotes the ratio of the concept group's 's ground truth given during the inference process. We have several observations based on these intervention results.
1. Our ECBM underperforms CEM with RandInt in terms of class accuracy. This is expected since CEM is a strong, state-of-the-art baseline with RandInt particularly improve inter- vention accuracy. In contrast, our ECBM did not use RandInt. This demonstrates the effec- tiveness of the RandInt technique that the CEM authors proposed.
2. Even without RandInt, ECBM can outperform both CBM and CEM in terms of "concept accuracy" and "overall concept accuracy", demonstrating the effective of our ECBM when comes to concept prediction and interpretation.
3. We would like to reiterate that ECBM's main focus is not to improve class accuracy, but provide complex conditional interpretation (conditional dependencies) such p(cly),
22

---

Published as a conference paper at ICLR 2024
p(ck M, Ce),, and p(c-k, 2/2x cx). cx). Therefore, our ECBM is actually complementary to sem- inal works such as CBM and CEM which focus more on class accuracy and intervention accuracy.
23


Reviews:
[{"review_id": "j2iANHLXH1", "reviewer": "Reviewer_XFNY", "summary": "This paper has introduced the idea of enery-based models in the concept bottleneck models to capture the interaction between the so-called \"concepts\" and quantify the dependencies between the \"concepts\" and class labels.", "questions": "1. As demonstrated in Table 1, the gap in performance between CBM and ECBM is noticeably smaller on the AWA2 dataset compared to that observed on the CUB and CelebA datasets. Is there any explanation for this phenomenon?\n2. How can the model ensure a stable solution during inference while searching for the optimal values of 'c' and 'y'? Is there any complexity analysis in B.1.\n3. Is it possible for the model to detect undefined 'concepts,' or is it capable of exploring new 'concepts'?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper enhances the state-of-the-art Concept Bottleneck Models by introducing a unified energy formulation that integrates concept-based prediction, concept correction, and conditional interpretation into a cohesive framework.", "weaknesses": "1. I find it difficult to fully embrace the idea of Concept Bottleneck Modeling (CBM) presented in this paper, as well as in prior research, due to the absence of a clear definition for the term \"concepts.\" From my perspective, what is referred to as \"concepts\" in this context appears to be nothing more than additional labels or explicit features that can be readily discerned by humans. However, one of the notable advantages of deep learning networks lies in their ability to automatically extract features, even those that may not be easily identifiable by humans. Consequently, I believe that the primary reason for the observed decline in performance of CBMs, when compared to recent work such as [1] on CUB, is that the model is exclusively directed towards learning these so-called \"concepts\" and neglects the acquisition of features that can be explicitly identified.\n2. There missing baseline models that are not based on CBM in the experiments.\n3. There missing ablation study on the proposed method with only a single branch, i.e, the 'concepts' branch and the 'class' branch as shown in Figure 1.\n4. The \"related work\" section about energy-based models is not complete. It misses some pioneering works and representative works to build a full history of the advancement of EBMs. For example, [2] [3] are pioneering EBM works, [4] also use an EBM for the distribution of labels (or other information) and data, which also captures high-level concepts and data. It is better to have a comparative discussion on both models if they share a similar concept. \n\n[1] Progressive co-attention network for fine-grained visual classification. 2021 International Conference on Visual Communications and Image Processing (VCIP). IEEE, 2021.\n\n[2] A Theory of Generative ConvNet. ICML 2016\n\n[3] Cooperative Training of Descriptor and Generator Networks. TPAMI 2018.\n\n[4] Cooperative Training of Fast Thinking Initializer and Slow Thinking Solver for Conditional Learning. TPAMI 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eu5Qpc16CU", "reviewer": "Reviewer_tSsK", "summary": "The article \"Energy-based Concept Bottleneck Models (ECBMs)\" presents a novel approach to addressing the limitations of existing concept bottleneck models (CBMs) in providing interpretable insights for black-box deep learning models. The authors identify three key issues with conventional CBMs, namely their inability to capture high-order interactions between concepts, quantify complex conditional dependencies, and strike a balance between interpretability and performance.\n\nThe proposed ECBMs leverage neural networks to define a joint energy function for input, concept, and class label tuples. This unified interface allows for the representation of class label prediction, concept correction, and conditional interpretation as conditional probabilities, enabling a more comprehensive understanding of the model's behavior.", "questions": "See weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. To my knowledge, utilizing energy based models in the context of CBMs is novel. To my knowledge, this is the first paper that does so.\n2. The writing is good.\n3. The experiments are pretty comprehensive but i have some points", "weaknesses": "One caveat is I am not that familiar with energy based models, so I wont be able to comment on the inherent problems of energy based models. However, given the context of CBMs, i want to have the following criticism:\n\n1. The second gap: Given the class label and one concept, the probability of predicting another concept? is not clear \n2. Incomplete literature review. The authors should add the following papers as each of them has a unique perspective to solve the problem.\n\n[1] Interpretable Neural-Symbolic Concept Reasoning. Barberio et al. ICML 2023 \n\n[2] Entropy-based Logic Explanations of Neural Networks. Barberio et al. AAAI 2021 \n\n[3] Addressing Leakage in Concept Bottleneck Models. Havasi et. al., Neurips 2022 \n\n[4] Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat. Ghosh et al. ICML 2023 \n\n[5] Distilling BlackBox to Interpretable models for Efficient Transfer Learning. Ghosh et al. MICCAI 2023 \n\n[6] Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification .Yang et al. CVPR 2023 \n\n[8] A Framework for Learning Ante-hoc Explainable Models via Concepts Sarkar et al. CVPR 2022 \n\n[9] Label-Free Concept Bottleneck Models. Oikarinen et al. ICLR 2023\n\n3. How does each of three conditional probabilities - p(y|x), p(c-k|x, ck), and p(ck|y, ck′)—mitigate the 3 limitations - Interpretability, Intervention, Performance? \n4. How does this method impact label free CBM or Language in a bottle kind of framework? There concepts are retrieved from the language model. because in more realistic setting like medical images having ground truth concepts is expensive.\n5. I believe this is an inherently interpretable method. How to integrate it in a posthoc setting like PCBM? Also, if it can be integrated in a posthoc setting, i would like to see the comparison of completeness scores.\n6. There is a new notion of difficulty estimation for samples using concept bottleneck models in . Can the energy based models be extended to that?\n7. One limitation of concept bottleneck models is the limited applicability in a real-life scenario. I would like to see one experiment on an application - for example shortcut identification as in [4] and Discover and Cure: Concept-aware Mitigation of Spurious Correlation. Wu et al. ICML 2023\n8. Also, there is a problem of leakage in CBM as in [3]. Will the ECBM be helpful in that regard? If the authors can show ECBM is better for the experiments depicted in fig. 2 and 3 in [3], this paper will be very strong.\n\n\nOverall, the contribuition of energy based method for CBM is novel. However, the x-ai community has identified the two setbacks of concept bottleneck models - 1) the cost of getting the ground truth concepts (for inherently interpretable setup) 2) the concepts are purely correlation not causal (for posthoc setup), so it is not guranteed if the concepts are actually utilized by the blackbox. This paper does not answer this two core research gap through their method as how ECBMs can help in either of the two question is still not clear to me.\n\n**Extremely pleased with response from the authors and especially with the leakage experiments. This paper deserves a 7 for sure and 8 is too high. So i am increasing my score**", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JDAeBr9oZL", "reviewer": "Reviewer_JBWu", "summary": "Concept Bottleneck Models often fail to capture complex interactions between the concepts and the interactions between concepts and labels. The paper proposes ECMBs that define an energy function over input concepts & class for generalizing & addressing the shortcomings of CMBs. Empirical results show that the proposed approach outperforms the CBMs on real-world datasets.", "questions": "Refer to Weaknesses", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Recently, there has been a lot of interest in developing models explaining themselves while making predictions. CBMs were proposed with such criteria in mind. \n- The proposed approach addresses some of the shortcomings of CBMs, expanding their availability.\n- The paper is well-written and easy to follow.\n- Empirical results show that the proposed approach outperforms the CBMs on real-world datasets.", "weaknesses": "One of the critical issues with the paper is how they evaluate & the choice of baselines. Even though they consider a diverse set of tasks, the authors must add additional experiments to strengthen the paper.\n\n- Does the architecture of the network affect the performance? Does it need to be shallow or deeper? What are the design considerations for these networks?\n- Is the model robust towards background shifts (ref. experiments in [1])\n- What is the compute overhead in comparison to CBMs?\n- One of the main shortcomings of CBMs is the need for concept annotations. There has been a lot of work on addressing it. Why do the authors build on top of vanilla CBMs, not the improved frameworks [3]?\nIn addition to CBMs, SENNs also offer a way to develop models that can implicitly explain their predictions, and many of their variants can efficiently encode the relationships between concepts ([2]), as well as between class & concepts. How does the proposed framework compare against those?\n- Many of the existing explanation methods aren't robust in practice. Are ECBMs robust to adversarial attacks?\n\n[1] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, Percy Liang: Concept Bottleneck Models. ICML 2020\n[2] Anirban Sarkar, Deepak Vijaykeerthy, Anindya Sarkar, Vineeth N. Balasubramanian: A Framework for Learning Ante-hoc Explainable Models via Concepts. CVPR 2022\n[3] Tuomas P. Oikarinen, Subhro Das, Lam M. Nguyen, Tsui-Wei Weng: Label-free Concept Bottleneck Models. ICLR 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vLMD2LBoO3", "reviewer": "Reviewer_WpSc", "summary": "This paper proposes Energy-based Concept Bottleneck Models (ECBMs), a novel concept-based architecture for predicting task labels and providing concept-based explanations for those predictions. The proposed model addresses (1) the lack of concept-based models that can capture relationships between concepts and labels and (2) the inability of test-time concept interventions in existing concept-based models to propagate to concepts besides those being intervened on. ECBMs achieve these goals via the use of energy functions that capture the joint probabilities between input features, concepts, and task labels. By learning these functions in an end-to-end fashion with an aggregate loss function, and composing these energy functions at inference time, one can (1) produce task predictions with corresponding concept explanations, (2) estimate concept importance for specific labels, and (3) impute mispredicted unintervened concepts after an intervention has been performed, therefore improving the impact of interventions on a model's performance. This paper evaluates ECBMs on three real-world datasets and shows that they can outperform existing baselines on both task and concept accuracy.", "questions": "The following questions, in no particular order, would help clarify some of the doubts on this work. If possible, I would appreciate it if the authors could elaborate on these concerns as they may serve as a good starting point for a discussion during rebuttal:\n\n1. Could you please elaborate on the seemingly contradicting baseline results discussed in the weaknesses? In particular, for the CEM interventions, the shown results seem to match the interventions in CEMs when `RandInt` is not used for training (something the authors explicitly discuss is necessary for interventions to work properly). Is it the case that you trained CEMs with RandInt? I tried to look for code as part of this submission but could not find it in the supplement (please let me know if I missed it; in which case, I am very sorry for that!).\n2. Related to the question above, which dataset is used in the evaluation of Figure 2? Is this CUB? I could not find this information anywhere. Also, when intervening on a bottleneck, are interventions performed on groups of concepts or on individual concepts at a time? In CUB, in particular, most previous work focuses on group-level interventions, so clarifying this is very important. Based on the left-most plot in Figure 1 increasing linearly, I am assuming interventions are done at an individual level rather than at a group level, but clarifying this is very important.\n3. Similarly, could you please elaborate on how ECBMs fare against competing baselines when intervened on in the other datasets?\n4. Could you please elaborate on the lack of results for PCBMs and ProbCBMs given my comments on the weaknesses indicating that they are in fact baselines that could be evaluated in the setups used in this paper?\n5. Could you please elaborate more on the specifics of the negative sampling strategy used to learn $E^\\text{global}_\\theta$? This is a critical practical element of this method and including details for this is crucial if you want to ensure reproducibility. Furthermore, understanding how this is done to scale for large concept spaces is extremely important to understand some of the limitations of this work.\n6. The outer summation of Preposition 3.2 is very unclear. It seems to be “iterating” over all concept vectors without considering the $k$-th dimension, but it is not immediately clear if this is the case or whether these vectors are what $\\mathbf{c}$ is referred to inside the summation. Can you please clarify what are this summation’s bounds and iteration variables?\n7. When learning the positive and negative embeddings for each concept, I was wondering if you have an intuition as to what inductive bias in your model is responsible for forcing the distribution of positive and negative embeddings to be easily separable? For example, in ProbCBMs the use of a contrastive loss with two “anchor” embeddings leads to this separation. Similarly, in CEMs the use of RandInt at training-time leads to this separation.  Understanding this is crucial for understanding why a given model performs well or badly when intervened on and may help clarify the design choice/process of this architecture to future readers hoping to build on top of this work.\n\nBesides these questions, I found the following typos/minor errors that, if fixed, could improve the quality of the current manuscript:\n\n1. In section 3.2 (page 5), “the second terms” should be “the second term”\n2. In section 3.3 (page 6), ”Proposition 3.2 below compute” should be “Proposition 3.2 below computes”\n3. In proposition 3.2, the variable $k$ is already captured when using $c_k$ but it is then overwritten in the summation over all concepts in the RHS’s numerator that uses $k$ again to indicate the iteration index. The same applies for propositions 3.4 and 3.5!\n4. In section 3.4 (page 6), “Proposition 3.3 below compute” should be “Proposition 3.3 below computes”", "limitations": "", "rating": 8, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "Thank you for submitting this work. I found the general main idea to be very interesting and worth exploring. In particular, I believe that the following are the main strengths of this paper:\n\n1. **Originality**: the idea of incorporating energy functions to model inputs, concepts, and task labels concurrently is, to the best of my knowledge, something that has not been attempted in previous work in the concept-based literature. In particular, I found the use of these energy functions to propagate interventions to be both original and interesting.\n2. **Quality**: although the quality of the experiments could be significantly improved on (see below), the quality of the presented idea, as the presentation of the idea itself, meets the standard of what I would expect in this field and is very clearly elaborated.\n3. **Clarity**: the paper’s writing is extremely clear and easy to follow. This helps the authors clearly communicate their ideas and the motivation behind the ideas. Although this work could benefit from further details when it comes to elements crucial to reproducibility (see below), I deeply thank the authors for writing in a very concise and clear manner throughout the manuscript. This lack of details, however, is the reason behind my \"fair\" score in the presentation field above.\n4. **Significance**: the paper’s main contribution, that of reframing concept learning as learning sets of joint probability distributions, can lead to novel and significant ideas in the future. It is certainly a novel and significant way to think about concept learning. Nevertheless, this significance is contingent on the paper’s core idea being carefully proven to work as expected (see weaknesses below).  If the experiments are carefully corrected for the weaknesses stated below, and the paper’s main claims still hold, then I believe this paper would be of good value to the community.", "weaknesses": "Although I found several parts of this work to be strong and well-defined, I have the following hesitations regarding this paper in its current state:\n\n1. [Critical] In my opinion, the biggest weakness of this paper is its evaluation and the fairness of the evaluation itself. As discussed in the questions below, this paper seems to compare baselines with significantly varying capacities and fine-tuning processes ( e.g., ProbCBM is trained with a ResNet18 backbone in the source of the results reported in this paper vs ECBM trained with a ResNet101 backbone) as well as failing to evaluate key contributions across several tasks (e.g., interventions are evaluated on a single task). This yields a lot of the evaluation inconclusive as it makes the source of the improvements shown in the proposed model hard to identify.\n2. [Critical] Against common practices in the field, none of the results in the evaluation section include error bars. Without such error bars, it is very hard to judge for statistical significance. This is particularly important for the paper’s main evaluation in Table 1 where differences between ECBMs and other baselines are small for task (e.g., ”class”) accuracies on some tasks.\n3. [Critical] Some of the key baseline results in this paper seem to contradict the results presented in previous literature. For example, CEM’s intervention performance in Figure 2 is in direct contradiction to those reported in the original paper and in subsequent works ([1, 2, 3]). Furthermore, the authors claim that ProbCBMs cannot be trained on CelebA due to the binary nature of the concept annotations. This however, to the best of my understanding, is not a requirement for ProbCBM (an example is CUB itself which is used as a baseline for this method even though it has not uncertainty labels as claimed by the manuscript). Similarly, results for Post-hoc CBMs are missing for CelebA and AWA2 as the authors claim that Post-hoc CBMs require \"predefined concept activation vectors.\" This is not true as the CAVs needed for PCBMs can be easily learnt from a dataset with binary concept annotations as those in CelebA and AWA2. In fact, the authors of that paper do this exact process for CUB.\n4. [Critical] A lot of details required for reproducibility are missing. There is no code attached to this submission as part of the supplement. There are no details on the details and training procedures of any of the baselines. The hyperparameter selection is not discussed anywhere, which brings doubt to the very specific precision indicated for some of ECBM’s selected hyperparameters (e.g., $\\lambda_c = 0.49$).\n5. [Major] Related to the point above, there are no ablation studies showing the sensitivity of the proposed method to its hyperparameters ($\\lambda_{\\{c, g\\}}$).\n6. [Major] The claim that previous concept-based methods do not allow one to understand concept to label relationships is not entirely true. Post-hoc CBMs enable this via their sparse interpretable predictor and even vanilla CBMs tend to be trained with a linear layer on top of their concept bottleneck, rendering an easy mechanism to obtain concept importance for a given label via the linear layer's weights.\n7. [Major] The proposed evaluation is limited in the number and quality of the competing baselines. For example, several of the baselines are missing values in Table 1.\n8. [Minor] Inference on ECBMs requires one to solve an optimization problem via a relaxation that is solved with gradient descent + back-propagation. This means that this proposed approach may not scale well to a setup where one is expected to run inference quickly or as part of a separate pipeline, which needs to be end-to-end differentiable.\n9. [Minor] The paper lacks any discussion of the limitations of its method. The last section, which has “limitations” on its title, fails to discuss any of the limitations.\n\nThese weaknesses, in my opinion, lead to significant doubt on the soundness of the discussed results and therefore contribute to my decision leading towards rejection. Nevertheless, if I misunderstood any significant pieces of this work, or if some of these weaknesses are void, I am happy to be convinced otherwise if the appropriate evidence is provided.\n\n### References\n\n[1] Sheth, Ivaxi, and Samira Ebrahimi Kahou. \"Overcoming Interpretability and Accuracy Trade-off in Medical Imaging.\" *Medical Imaging with Deep Learning, short paper track*. 2023.\n\n[2] Collins, Katherine Maeve, et al. \"Human uncertainty in concept-based ai systems.\" *Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society*. 2023.\n\n[3] Zarlenga, Mateo Espinosa, et al. \"Learning to Receive Help: Intervention-Aware Concept Embedding Models.\" *arXiv preprint arXiv:2309.16928* (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uljoHSnFhi", "reviewer": "Reviewer_doCC", "summary": "The paper proposes a new CBM, Energy-Based concept bottleneck model ECBMs.\n\n## Model architecture and training:\n\nBefore ECBM, the feature extractor network extracts feature embedding of the input $z$ ECBMs consists of 3 energy networks:\n- Class Energy Network: Each label is associated class label embedding $u$, this is inputed to the trainable NN along with $z$, $E_\\theta^{class}=G_{zu}(z,u)$. To train the class energy network, the Boltzmann distribution is used to define the conditional likelihood of y given input x. The class energy network maps the input class pair (x, y) to real-valued scalar energy, negative log-likelihood is used as a loss function.\n- Concept Energy Network: It consists of K sub-networks (one for each concept), the input is an embedding that represents the aggregate of positive and negative concept embedding $v_k$ (similar to CEM) along with $z$ $E_\\theta^{concept}=G_{zv}(z,v_k)$. As done for the class network negative log-likelihood is used as a loss function.\n\n- Global Energy Network: this network learns the dependencies between the concepts and the output. As an input it takes the class embedding  $u$ all concepts embeddings  such that   $E_\\theta^{global}=G_{vu}([v_{k}]_{k=1}^{K},u)$. The negative log-likelihood loss is calculated for each concept-class pair. \n\n\n## Prediction:\nTo make a prediction feature, the feature network and energy networks are frozen then search for optimal prediction of concepts $\\hat{c}$ and the class label $\\hat{y}$ by minimizing equation 12.\n\n## Interventions:\nOne of the main advantages of ECBMs over other methods is that they are able to propagate the corrected concept to other correlated\nconcepts, this is done automatically while optimizing for $\\hat{c}$ and $\\hat{y}$ and changing some values in $\\hat{c}$ to ground-truth $c$.\n\n## Conditional interpretations:\nBy using the energy functions given a target class one can calculate the marginal concept importance for that class, the joint concepts importance for that class, the conditional probability between concepts for that class and the conditional probability between concepts (without taking class into consideration i.e the conditional probability summed over classes).\n\n## Experiments:\n- The paper compared ECBMs with CBMs, CEMs, PCBMs and ProbCBM; across 3 datasets.\n- For evaluation, the paper compared (a) Concept accuracy (i.e. predict each concept independently). (B) The overall concept accuracy, this evaluates that all concepts are predicted correctly for a given sample i.e concept vector == ground-truth vector. (C) The class accuracy.\n- For the experiments they looked at concept and class label prediction (without any interventions), concept and class label prediction with interventions and conditional interpretation.", "questions": "- What is the accuracy of a black-box model for datasets in table 1?\n- Results in figure 2 are from which dataset?\n- Aren't $E_\\theta^{class}$, $E_\\theta^{concept}$,$E_\\theta^{global}$ neural networks? If so why don't they have trainable or frozen signs in figure 1?\n- Equation 9 is $[v_k]_{k=1}^{K}$ this the concatenation of all concepts embeddings?\n- Please clearly state the output dimensions of each energy model.\n- For the gradient-based inference Algorithm 1 line 14,15,16 since this is done during inference the algorithm shouldn't have access to $c,y$ ? Is this a typo?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "## Originality-Excellent:\n- The idea to use energy-based models for concepts is quite original.\n- The training and inference algorithm is also original and the idea has not been explored before.\n\n## Novelty-Excellent: \nBesides the model architecture and loss being novel there is two additional aspects that are also novel about this work: \n- As far as I know, this is the first work that captures concept interaction during intervention.\n- The conditional interpretations have not been investigated previously.\n\n## Quality-Good:\n- Based on the result section, for  ECBMs outperform baselines for concept and class label prediction experiments. For the interventions, ECBMs seem to outperform baselines in concept accuracy and most interestingly the overall concept accuracy, which supports the intilal claim of the paper that by correcting one class will help other classes can be corrected as well. However, CBMs outperform in class accuracy which is somewhat expected.", "weaknesses": "## Clarity -- Poor:\n- In page 1, under limitations of CBMs the paper mentioned interpretability and said \"They cannot effectively quantify the intricate relationships between various concepts and class labels\". I disagree, in the original CBM on top of the CBM layer there is a linear which quantifies how each concept affects the final output so if a model classifies an image with label \"A\" we can look at the weight matrix and see which concepts where used in this classification yes at inference time (assuming we don't have the ground truth concepts) we don't know if the model correctly classified the concepts or not or the likelihood of a concept being correctly classified but we do know the relationship between the concepts and the final output.\n- The paper is not very clear, please see the questions section.\n\n## Significance --Fair:\n- ECBMs Training: Training is much more complicated than regular CBMs and probably much more expensive particularly optimizing equation 10.\n- ECBMs Inference requires optimization not just a forward pass through the network.\n- Usage and reproducibility: There was no code to reproduce the experiment. It is very difficult to imagine how all of this is trained together without the code.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)