Faithful Vision-Language Interpretation via Concept Bottleneck Models
Authors: Songning Lai
Lijie Hu
Junxiao Wang
Laure Berti-Equille
Di Wang
Venue: ICLR 2024 poster (2024)
Keywords: Interpretability ML
Explainability ML
Concept Bottleneck Model
Primary area: visualization or interpretation of learned representations

Abstract: The demand for transparency in healthcare and finance has led to interpretable machine learning (IML) models, notably the concept bottleneck models (CBMs), valued for their potential in performance and insights into deep neural networks. However, CBM's reliance on manually annotated data poses challenges. Label-free CBMs have emerged to address this, but they remain unstable, affecting their faithfulness as explanatory tools. To address this issue of inherent instability, we introduce a formal definition for an alternative concept called the Faithful Vision-Language Concept (FVLC) model. We present a methodology for constructing an FVLC that satisfies four critical properties. Our extensive experiments on four benchmark datasets using Label-free CBM model architectures demonstrate that our FVLC outperforms other baselines regarding stability against input and concept set perturbations. Our approach incurs minimal accuracy degradation compared to the vanilla CBM, making it a promising solution for reliable and faithful model interpretation.

Published as a conference paper at ICLR 2024
FAITHFUL VISION-LANGUAGE INTERPRETATION VIA CONCEPT BOTTLENECK MODELS
Songning Lai*12255 Lijie Hu Hu Junxiao Wang 1.2.4, Laure Berti-Equille", and Di Wang
1 Provable Responsible Al and Data Analytics (PRADA) Lab 2 King Abdullah University of Science and Technology
IIRD Institut de Recherche pour le Développement, Montpellier, France "SAAAA-KASS AI "Shandong University
ABSTRACT
The demand for transparency in healthcare and finance has led to interpretable machine learning (IMI.) models, notably the concept bottleneck models (CBMs), valued for their potential in performance and insights into deep neural networks. However, CBM's reliance on manually annotated data poses challenges. Label- free CBMs have emerged to address this, but they remain unstable, affecting their faithfulness as explanatory tools. To address this issue of inherent instability, we introduce formal definition for an alternative concept called the Faithful Vision- Language Concept (FVLC) model. We present methodology for constructing an FVLC that satisfies four critical properties. Our extensive experiments on four benchmark datasets using Label-free CBM model architectures demonstrate that our FVLC outperforms other baselines regarding stability against input and con- cept set perturbations. Our approach incurs minimal accuracy degradation pared to the vanilla CBM, making it a promising solution for reliable and faithful model interpretation.
"The greatest obstacle to discovery not ignorance; it the illusion of knowledge."
Daniel J. Boorstin
INTRODUCTION
Contemporary machine learning models, like deep neural networks (DNNs), often rely on intricate nonlinear structures, making them challenging for end-users to understand and trust. This lack of interpretability hinders their adoption, especially in critical domains like healthcare (Ahmad al.,
2018 Yu et al. [2018] Lai et al., 2022) and finance (Cao, 2022), where transparency is crucial. In response to this demand, interpretable machine learning (IML) models (Doshi Velez Kim 2017) offer explanations for their behavior and insights into their inner workings. Concept Bottleneck Models (CBMs) have emerged as a noteworthy IML model, focusing on enhancing system safety and building trust among stakcholders (Koh et al. (020). Unlike traditional IML models, CBMs leverage elevated concepts, featuring a concept layer that aids in downstream tasks and produces eas- ily understandable deep features, facilitating users' comprehension of the model's decision-making processes and concept relationships with input data.
CBMs have recently advanced significantly. driven by notable studies (Yuksekgonul et al. 2022 Oikarinen et al. 2023). 'hese advancements are partly attributed to pre-trained large language mod-
els (LL.Ms) and vision-language models (VLMs) (Menon & Vondrick) 2023 Yang et al. 2024a b
Xu et al. 2023b). Traditional CBMs require substantial manual annotation, which label-free CBM OOikarinen et al.) 2023) effectively addresses by leveraging factual information from pre-trained models, reducing the need for domain-specific experts. However, this convenience comes with in- herent instability in pre-trained models, which we investigate this paper. Instability is common issue in deep learning interpretation methods, making it challenging to understand model reason- ing (Hu et al. 2023c b), especially with unlabeled data and self-supervised training (Ghorbani al.,
"The first two authors contributed equally to this work.
1 "ooresponneence Lijie Hu (lijie.uu @ kaust, kaust.edu.sa. edu.sa) and Di Wang (di.wang@@ kaang@kaust.cdu.sa)

---

Published as a conference paper at ICLR 2024
8:38- Lockeh
wwrkondatrr
Hooppa pour
Sum cow
San of 2347
Figure 1: This is an example using the Places365 dataset (Zhou et al. 2017). The leftmost fig- ure displays the input image, while the adjacent one on the left shows the concept output without perturbations. In contrast, the figure on the right presents the concept output after applying concept words and input perturbations, resulting in noticeable changes. These alterations include shifts in concept positioning along the ordinate and adjustments the ranks of their respective weights, as exemplified by the concept "surgery". The prediction has also changed under slight perturbations.
2019 Dombrowski et al. 2019 Ych et al. 2019). This instability issue also affects Label-free CBM, as Figure I illustrates.
To ensure a "faithful concept" for explaining deep features from the CBMs, we need a precise def- inition. Ideally, a "faithful concept" should have these four properties within the same concept set, which is generated by the LLM according to classes: (i) Significant overlap between the top-k in- dices of the "faithful concept" and the original concept, ensuring interpretability. (ii) Inherent stabil- ity, with the concept vector remaining robust against random noise and perturbations during L.M concept set generation. (iii) prediction distribution close to that of the vanilla CBMs, preserving its outstanding performance. (iv) Stable output distribution, remaining robust during self-supervised learning and LLM concept set generation, even in the presence of perturbations.
We have identified significant faithfulness issue in generating textual descriptions from visual data due unstable alignment be- tween text and images. (2) Rigorous Definition of FVLC: Building on our findings, we offer formal definition of faithful vision-language interpretation with four essential properties, providing clear framework for assessing VLM faithfulness.
Our Contributions: (1) In-Depth Analysis of Faithfulness in VLMs:
(3) Innovative Faithful Mapping Module: To tackle VLM faithfulness issues, we present an adaptable and flexible mapping module. It optimizes pa- rameters minimally, preserving the integrity of the large model's pre-trained parameters, and aligns with our faithfulness definition. smoothly integrates with classifier-fre vision-language inter- pretation models without altering the original model structure, simplifying faithful interpretation in downstream tasks. (4) Proposed Faithful Evaluation Metric: Alongside the mapping module, propose original metric tailored for evaluating vision-language interpretation faithfulness, measuring alignment stability between textual information and visual content for more precise as- sessments. More details about the setting, implementation, and additional experimental results can be found in Appendix.
2 RELATED WORK
Concept Bottleneck Models. Considerable research has explored Concept Bottleneck Models (CBMs) recently (Havasi al., 2022 Kim al., 2023 Keser al.) 2023) However, CBMs face two primary challenges. Firstly, their performance often falls short compared to models without the concept bottleneck layer due to incomplete extraction of information from the original data into bot- tleneck features. Secondly, CBMs require laborious dataset annotation by human experts. Numerous studies have tackled these challenges and proposed potential solutions.
Chauhan et al. 2023) extend CBMs to interactive prediction settings by developing an interaction policy that determines which concepts to request labels for, thereby improving the final prediction. Post-hoe Concept Bottleneck models (PCBMs) (Yuksekgonul et al. 2022) can be applied to any neural network without sacrificing model performance while still providing interpretability benefits.
It's important to highlight that the noise can originate not only from the input data hut also from other sources like perturbed concept sets (c.g., Figure T emphasizing distinctions from adversarial robustness.
2

---

Published as a conference paper at ICLR 2024
When concept annotations are unavailable, PCBMs demonstrate the ability to transfer concepts from other datasets or utilize natural language descriptions through multimodal models. (Sawada Naka- mura (2022) integrate supervised concepts with unsupervised ones trained using self-explaiming neural networks (SENNs) to obtain both types of concepts simultaneously. Oikarinen et al. (2023) discuss the limitations of existing CBMs and propose Label-free CBM, which allows any neural network to be transformed into an interpretable CBM without the need for labeled concept data while maintaining high accuracy. Label-free CBM achieves state-of-the-art performance against other CBMs on classification tasks and represents an efficient unsupervised CBM. However, it is important to note that current CBM models do not consider the instability of concept sets' words generated by large language models or the instability caused by self-supervised training. Perturba- tions to the words in the concept set and input can impact the interpretability of the output. this paper, our focus is primarily on enhancing the faithfulness and stability of Label-free CBMs.
Faithfulness in explanation methods. Faithfulness, as a crucial property of explanation models, entails accurately reflecting the true reasoning process of the underlying model in the explanation (Wiegreffe & Pinter 2019 Herman 2017 Jacovi & Goldberg 2020; Lyu et al. 2022). is closely related to other principles such as sensitivity, implementation invariance, input invariance, and com- pleteness (Yeh et al. 2019). Completeness, for instance, signifies that an explanation should compre- hensively encompass all relevant factors contributing to the prediction (Sundararajan et al.] 2017). The remaining three principles pertain to stability concerning different types of perturbations. An explanation should undergo changes if significant features that influence the prediction are heavily perturbed (Adebayo et al. 2018) while remaining stable in the face of small perturbations (Yin et al. 2022 Therefore, stability plays pivotal role in ensuring explanation faithfulness.
Several preliminary approaches have been proposed to achieve stable interpretations. For example, Yeh al. (2019) theoretically analyzes the stability of post-hoc interpretations and proposes the use of smoothing techniques to improve interpretation stability. Additionally. Yin et al. (2022) design an iterative gradient descent algorithm to obtain counterfactual interpretations exhibiting desirable stability characteristics. However, these techniques are primarily designed for post-hoc interpreta- tion and may not directly apply to CBMs. Novel methodologies need to be explored to address the stability requirements in CBMs. This paper aims to develop techniques that enhance stability and faithfulness within the context of CBMs. By considering the unique characteristics and challenges posed by CBMs, we seek to establish framework that achieves stable interpretations while ad- hering to the principles of faithfulness, completeness, and sensitivity. Our proposed approach will facilitate reliable explanations in CBMs and contribute to the interpretable machine learning field.
3 PRELIMINARY
Concept Bottleneck Models. To introduce the original CBMs, we will adopt the notations used by Koh et al. (2020). We consider a classification task with a concept set denoted as C = {PI..... Pp} and a training dataset represented as {I. y/, c.) where for [N], 24 E Rd is the feature vector, y/ Rdd: denotes the label, where d, corresponds to the number of classes, and e; E R denotes the concept vector whose j-th entry represents the weight of the concept pj. In CBMs, we aim learn two representations. One transforms from the input space to the concept space, which is represented by g : d + R² The other one maps from the concept space to the prediction space, which can be denoted by f : R dRd.. For any input æ, we aim to make its predicted concept vector è = g(x) and prediction ý = (g(()) to be close to its underlying ones.
Label-free CBMs. The label-free CBM proposed by Oikarinen al. (2023) has four steps:
Step 1: Concept set creation and filtering. In traditional CBMs, the concept set is typically gen- erated through annotations by human experts. Instead, we propose utilizing the OpenAl API and leveraging GPT-3 (Brown et al. (202)) to automatically generate the concept set based on the classes, which are the dataset's labels. GPT-3 possesses a substantial amount of domain knowledge and can effectively identify important concepts associated with detecting each class when prompted appro- priately. To accomplish this, we ask GPT-3 the following:
(i) List the most important features for recognizing something as a {class); (ii) List the things most commonly seen around {class); (iii) Give superclasses for the word {class]. After Step 1, we get a concept set C = {pr.... .Ps)- See the Appendix for additional details and sample output.
3

---

Published as a conference paper at ICLR 2024
Step 2 and 3: Learning the Concept Bottleneck Layer (CBL). After obtaining the concept set, the following step involves acquiring a projection from the feature space of the backbone model into space where interpretable concepts correspond to axis directions. To achieve this, we should learn the projection weights Wa without relying on any labeled concept data by using CLIP-Dissect (Oikarinen & Weng 2022] Guo al. 2023bla). Initially, we require concept set that the bottleneck layer is supposed to represent as C (P1, p. and a training dataset (such as images) without O; of the original task. We then calculate and save the CLIP concept activation matrix M, where Ma., = (x,)) (P)), and E, and Er are the CLIP image and text encoders respectively. W initialized as a random
matrix, where is the dimensionality of backbone features (- (Backbone is the part that extracts features from the input image for the network). The concept set C is created Step 1, and the training dataset provided by the downstream task. We define g(.) = W.b We use E [d] to denote a neuron of interest in the projection layer and its activation pattern is denoted as qi where qi = (b: (z1()) ....)................... ... g: ( with qi E R and gi is the I-th coordinate of g(r). Our optimization goal is to minimize the defined objective L over Wo. which is as follows:
Th
M
L(W.)
sim (am(P,,0) P.,
1 12 M
112
Here, 9 indicates vector q normalized to have mean O and standard deviation 1.
Step 4: After successfully learning the Concept Bottleneck Layer, the next step involves training the final predictor using the fully connected layer WF E R where d, represents the number of output classes. For each input v, we can get the class prediction distribution y(2r c) = WFg(x)- This process allows map the extracted concepts the output classes, enabling the model to make accurate predictions based on the learned features.
4 FAITHFUL VISION-LANGUAGE CONCEPT
Faithfulness issues in Label-free CBMs. The Label-free CBM, as currently implemented using GPT-3, has inherent limitations that can impact its stability and reliability. One such limitation is the potential introduction of instability and perturbation to the concept sets due the reliance on GPT-3 for concept generation. Additionally, it is challenging to prevent input images from being perturbed, which further compounds the issue. These limitations can result in unstable performance, under- mining the trustworthiness and faithfulness of the model. While Label-free CBM shows promise in explaining model behavior without relying on labeled training datasets, it suffers from conceptual instability when faced with slight perturbations in the concept set and input data. This instability renders it unsuitable for real-world applications where robustness and reliability are crucial. By addressing these challenges, can advance the field towards more trustworthy and robust deep learning models.
What is a "faithful concept"? A "faithful concept" is crucial for model interpretation. "faithful concept" encapsulates a weight vector that exhibits stability and reliability, thereby instilling trust in its ability to offer accurate explanations of model behavior. When we refer to a "faithful concept", we mean a concept vector that remains stable even perturbations introduced to the input data or modifications applied to the concept set itself. These perturbations may arise due various factors, such as noise or slight alterations in the wording of the concept set. However, "faithful concept" should demonstrate resilience and maintain its explanatory power within an acceptable range despite these perturbations. By ensuring the stability and reliability of the concept, we can establish foundation for faithful interpretable model explanations, thereby enhancing our understanding of complex deep learning systems. See Appendix A details intuition of "faithful concept".
Definition 1 (Top-k overlaps). This definition mainly measures the overlap of the first concept weights. We need to use the overlaps of top-k indices to measure the similarity concept. For vector x E Rdd, we define the set of top-k component Tx(.) as follow.
Te(x)) {i i E [d] and (Kæ, 2 24 j E [4]] k)).
And for two vectors x', the top-k overlap function Vi(z, x²) is defined by the overlapping ratio between the top-k components of two vectors, i.e.. Vi 2') = 1 (x) [z(z'))
Note that, in sancept-based models, c could be seen as a function of input x, i.c., c == g(x). Thus, the faithful concept ē can also be seen function of xr, denoted ğ(z). Moreover, since our goal
4

---

Published as a conference paper at ICLR 2024
is to replace the concept vector with more robust one, we follow the previous model except for the procedure to produce the vector č. We define an FVLC as follows.
Definition 2 (Faithfull Vision-Language Concept). Under the same concept space, i.e., under the set of concepts generated by GPT3 al one time, we call a matrix W. is (D, R, or, B, ki, (h,kk))-Fon -2)-Faithful Dision-Languee Concept (FVLC) model for the vanilla concept if satisfies for any input x:
(Similarity of Explanation) Vk, (g)). gx)) 2 $1 for some 1 2 B1 2 0;
(Stability of Explanation) Vk, ()) - p) 2 B2 for some 2 B2 2 0 and all ||ll| S
R1, where || || a norm and R1 2
(Closeness of Prediction) D(y(a, (lu(z,)).fc y(, c)) S 0n for some an 2 where D some
probability distance or divergence;
(Stability of Prediction) Diy( é) y(z, ē + 8)) S a2 for all ||ō|| S R2, where D is some probability distance or divergence, || is a norm and R₂ 2 0,
where ((x) (x) y(2, c) WFg(z)) and y(2, - y(2, Moreover, for any given C π(x) is a (D, R. 3, kn, k2) -Faitfuul Vision-language Concept (FVLC). Here, a min (01, 022,, 33 = max ( [1, 32], and R = min min R1, R2}.
00)
5 FVLC FRAMEWORK
Sensitivity, In the above definitions, two properties are considered: similarity and stability, relevant to both prediction and explanation. Notably, within the domain of explanation, our aim extends beyond ensuring explanation stability alone. We also strive for explanation sensitivity, a distinct criterion from that of prediction. The concept should demonstrate sensitivity when crucial features are excluded while maintaining stability when subjected to minor perturbations.
The top-k approach. To achieve this, we employ the top-k approach to preserve these crucial fea- tures, ensuring that the modified concept ē maintains similar explainability to the original concept. The top-k approach involves two parameters, k1 and 31. k1 can be considered as prior knowledge, indicating that we believe the top-kı indices of the concept play the most crucial role in making the prediction that their corresponding features almost entirely determine the prediction. On the other hand, 31 quantifies the extent to which ē inherits explainability from the original concept. When 3 1, it signifies that the top-kı order of the entries in (1x) remains the same as in the original concept. Therefore, it is desirable for 31 to be close to 1.
Stability. The stability condition involves two parameters, Rí and 32, representing the robust region and level of stability, respectively. Ideally, if č satisfies this condition with R1 oo and Bg = 1, it indicates an extremely stable concept, resilient to any randomness or perturbations. In practice, we strive to have Ri as large as possible and (32 sufficiently close to 1.
Prediction.
The last two conditions pertain to the similarity and stability of concept-based predic- tion. In the third condition, quantifies the proximity between the prediction distribution based on and that based on the original concept. A value of ai implies that غ = c. Therefore, our goal is to minimize ai as much as possible. Similarly, the stability condition involves two parameters, R2 and 022, representing the robust region and level of stability, respectively. Ideally, if satisfies this condition with R₂ do and a2 = 0. it indicates randomness or perturbations. In practice, we aim for R2 to be as large as possible and a₂ to be sufficiently small. Based these discussions, it is evident that Definition provides a reasonable and consistent definition for ensuring the required properties of a faithful concept.
extremely stable concept, resistant any
Framework. We have proposed strict definition of FVLC. To build the FVLC, freeze the image encoder and derive a minimum-naximum optimization problem with four conditions in the Definition 2 Specifically, the formula optimization problem targets the third condition (predicted proximity) and makes it subject to the other three conditions. So, by definitions, we can get rough optimization problem and derive the following overall objective function:
min E, D(, 2). v(z, c)) Ve, (z)x), (z)) + A max D(v(2, C), (0, 8)) W.
-M max Ve, (o())) + p)).
SR,
5

---

Published as a conference paper at ICLR 2024
M E
Concept
Text Encoder
Prod: rock
Wy
We W
Image Encoder
y()
g() - a()
Backbone bf()
L(W
Figure 2: An overview of our pipcline for creating FVLC. For the concept set C output by GPT-3, is input into the text encoder of CLIP to obtain Fy. The input image is processed by the image encoder of CLIP obtain Fj. To accomplish the image classification task, the image is fed into the backbone to extract image features. The obtained activation matrix M from E₁ and Er is used learn the mappings g(-) and We from the feature space the concept space. Furthermore, the mapping Wp from the concept space to the image category is learned. Then L//Cs/Cs/C, are employed to enhance the model's faithfulness (W: - We. g + (())). We introduce noise interference in the concept set, text encoder, and input image to validate the faithfulness of our model. The box mixed with black and white dots represents the noise, and snowflakes represent the freezing parameters.
where A E [4] are hyperparameters. To solve this min-max max optimization problem, we can gener- ally use stochastic gradient descent-based methods to obtain the solution for the outer minimization and use PSGD (Projected Stochastic Gradient Descent) to solve the inner maximization. However, a major challenge that the top-k overlap functions Ve, (g(x), ((f()))rr))) and Vi, (G)), g(x) + p) are non-differentiable, which makes it difficult to use gradient descent. Therefore, need to consider
a surrogate loss for .... In Appendix B we show details on such a surrogate loss and the optimization procedure. Finally, have the following objective function:
min E. (1 D(y(I, 2), u(x, c)) + A2 Ls, (0(x).((x) + As max D(y(a, C), vI, ē+ 8))
W.
|
+ A4 max Cx, (F(x). + p)], R
(2)
where Lk is the surrogate loss and D is Kullback-Leibler Divergence.
EXPERIMENTS
6.1 EXPERIMENTAL SETUP
Datasets. We conducted comprehensive evaluation of our approach by training our model on four diverse datasets: CIFAR-10. CIFAR-100 (Krizhevsky al. 2009), CUB (Wah et al.) 2011), and Places365 (Zhou et al. 2017). See Appendix D for more details, such as datasets, backbone, baselines, and other settings.
Perturbations. First, word perturbation. For the concept set generated by Step in LCBM, we perform word perturbation. Taking advantage of the power of GPT-3 (Brown al. 2020). enter

---

Published as a conference paper at ICLR 2024
CIFAR10
CI
CUB
Places365
Figure 3:
sample from each dataset. The sequence, from left to right, includes the input image x, concept weight visualization without perturbation (c), concept weight visualization with perturbation (c + 8), and optimized con- cept weight visualization with perturbation (č + 6). See Appendix for a larger version these subfigures.
The visualizations for concept weights and final layer weights
Method
CIFARIO
CIFARIO0
CUB
Places365
Standard (No interpretability)
88.80%
70.10%
76.70%
48.56%
P-CBM (CLIP)
84.50%
56.00%
N/A
N/A
Label-free CBM
86.32%
65.42%
74.23%
43.63%
WP1(5%) base
86.47%
65.13%
74.08%
43.57%
WP1(5%) FVLC
65.43%
73.96%
43.67%
WPI(10%) base
86 86.25%
65.09%
73.97%
43.67%
WPI(10%) ( FVLC
86.39%
64.90%
73.92%
43.62%
WP2 base
86.41%
65 16%
73. 73.96%
43.54%
FVL-
86.22%
65.34%
74.44
44.55%
IP base
86 66.62%
65.36%
74.39%
43.64%
IP FVLC
86,88%
65.29%
74.01%
43.71%
WPI base
86.49%
65.17%
9..9%%%
43.67%
WPI FVLC
86.43%
63.33%
73.92%
43.49%
WPL(103)-W22 base
64.87%
73.82%
43.61%
(PI(10%)-WP2 F FVLC
86.38%
65.06%
74.01%
43.44%
- buse
85.96%
64.41%
73.74%
43.52%
WPP21P FVLC 86.70%
65.14%
74.36%
43.46%
The table presents accuracy for baselines and FVLC before and after the perturbations across four benchmark datasets. In the first row, we have the standard backbone image classifica- tion model, which lacks interpretability. The latest CBM models with interpretability are P-CBM YYuksekgonul et al., 2022) and Label-free CBM (Oikarinen et al. (203),, both of which employ an unsupervised process for generating concept sets, eliminating the need for manual labeling. The accuracy of Label-free CBM under various perturbations is displayed in while row color. The per- centages parentheses indicate the degree of added WP1 (The same as below).
Table I:
the full concept set into GPT-3 and ask it to replace 5%, and 10% of words with synonyms. By this operation, we obtain the concept set with 5%, and 10% word perturbation. We name this perturbation WP1 (Word Perturbation 1). Second, word embedding perturbation. Specifically, we choose the embedding of the last layer of the text encoding layer and then embed x' (N(0 a) with
7

---

Published as a conference paper at ICLR 2024
perturbation of certain radius 0.001. We name this perturbation WP2 (Word Perturbation 2).
Third, input perturbation. We add the perturbation of a certain radius o = 0.001 in input images (Input image pixels have been normalized). We name this perturbation IP (Input Perturbation).
CIFARIO
CIFAR100
CUB
Places365
Method
TCPC
TOPC
TCPC
TOPC
TCPC
TOPC
TCPC
TOPC
WPI(5%) - base
1,55E-01
6.32E-02
100E-01
7.17B-02
1.26E-01
1.85-001
1.59-001
6.40E-02
WPT(5%)
1.12E-03
8,55K003
2811303
4.510003
1,055-022
150K-03
1,388003
1,30K403
WP1(10%) base
1.99E-01
8.36E-02
1994-00
1.31B-01
2.32E-01
3.41E-01
2.26E-01
1.14E-01
WPI(10%) FVLC
1,19--03
7,40E-03
3.67E-03
4.55E-03
1.19E-02
1.53-003
1.39E-03
1.25E-03
WP2 base
1.53E-01
4.99E-02
1.36E-01
6.67B-02
1.43E-01
1.73E-01
1.40E-01
6.37E-02
WP2 PVLC
1.10E-02
8.72V-03
3.35K:03
4.55E:03
1O55442
1.533-03
1.55E:03
1.29F:40
IP- base
1668-00
6.28E-02
1.38E-01
8.81B-02
1.71E-01
2.23E-01
1.73E-01
8.09E-02
IP
EVECC
8.02E:003
8.29
3.248:03
4.56E:03
LO4K40
1,59,00
1.50F:03
1.25V:03
WPH5%)4TZ2 base
188E-01
3.50F-02
1.28E-01
6.65B-02
1.44-001
1.79E-01
1.60E-01
6.32F-02
WPI(5S)+WP2
PVLC
1.205-02
7,46F:03
3.67E:03
4.55V:03
9.816-2
LSIK-03
1.54F:03
1,281:433
WPI(103-WPP2 base
1.17-001
8.62E-02
1.93E-01
1.325-01
1.76--01
3.45E-01
2.526-01
1.17E-01
WPI[1OS) FVLC
L18E-00
9.41E-03
2.06E-02
1.4E-002
187E-02
3.79E-02
2.74E-02
1.18E-02
- base
1.36F-01
1.05E-01
2.226-01
1.551-011
1.958-01
3.54E-01
2.626-01
1.44E-01
WPPHIP FVLC
143E-02
1.11E-02
2.39E-02
1.77E-02
2.21E-02
4.54E-02
3.35E-02
1.34E-02
Table 2:
The table demonstrates the stability (TCPC and TOPC) results for both the baselines and FVLC approaches across four benchmark datasets subjected to various perturbations.
Evaluation metrics. In experiments, assessed the stability of model using two metries: Total Concept Perturbation Change (TCPC) and Total Output Perturbation Change (TOPC).
TCPC measures the stability of model interpretability between concept weight vectors, namely Ci (the concept weight vector before perturbation) and 22 (the concept weight vector after pertur- bation). It is calculated as TCPC(C1, 02) ||eı e2ll/||c where C1, C2 represents the complete concept weight vectors. TOPC assesses the stability of the model's output results by comparing two sets of outputs: y1 (the output results before perturbation) and y/2 (the output results after perturba- tion). It is defined as TOPC(11 92) = |ly] y/2l/ [|3/ Il. where y1, 32 represents the complete last layer output vector (i.e. the classes prediction output vector).
6.2 UTILITY EVALUATION
Table presents the accuracy results of our proposed FVLC method and the baseline approach on four datasets with different levels of perturbations. The table clearly demonstrates that our method maintains a consistent and high accuracy across all datasets without any significant changes or losses. This highlights the robustness of our approach in terms of accuracy preservation. More- over, our FVLC method outperforms the baseline approach in terms of accuracy in some cases. This indicates that our approach not only maintains stability but also achieves improved accuracy com- pared to the baseline method. Overall, the results presented in Table the effectiveness of our FVLC method, which successfully combines high accuracy with interpretability while preserving stability across various datasets.
6.3 STABILITY EVALUATION
Table 2 analyzes the TCPC and TOPC values, which measure the stability of the CBMs under various perturbations. The table demonstrates that, in comparison to the baseline approach, FVLC exhibits higher stability in terms of concept weights and smaller differences in the output result matrix before and after perturbation. These findings indicate that FVLC combines interpretability with strong perturbation resistance, making it a faithful model. The experimental results reveal that, on average, FVLC outperforms the baseline model in terms of both TCPC and TOPC, with an average reduction of 90%.
For brevity, Figure 3 presents visualizations of the concept set differences and concept weight changes, both with and without fine-tuning, before and after perturbation for each dataset. These visualizations provide additional insights into the effectiveness and stability of FVLC in handling perturbations. The results presented in Table and Figure 3 collectively demonstrate the superior stability of FVLC compared to the baseline model. FVLC successfully achieves faithful interpre- tation, making it a promising approach for concept-based modeling. We also carried out repeated experiments in multiple concept spaces to further verify that our method conforms to Definition 2 More experiments are in the Appendix
8

---

Published as a conference paper at ICLR 2024
Method
Setting
CIFARIO
CIFARIO0
Places66
C La C1
TCPC
TOPC
TCPC
TOPC
TCPC
TOPC
TCPC
TOPC
WPH10S)- FVLC
1.99-001
8.36F-02
1.94F.01
1.310-01
2.326-01
3411-01
2.265.01
1.1E.01
2,09E-02
3.148-02
28818-02
4.885-02
4.085-01
7.565-02
4.696-02
6.145-02
L80E-02
1.79E-02
2.01E-02
2.85E-02
3.778-01
4.50E-02
4.48E-02
3.68E-02
4.78E-03
3.11E-02
1.67E-02
2.19E.02
5.691-02
5.526-03
6.04E-03
4.98E-03
1.67H-02
1.118-02
5,398-02
638--002
1.60H-01
2.125-02
1.988-02
1.818-02
L655-03
1.0E-00
5.08E-03
6.43E-03
1.615-02
2,14B-03
1.95E-03
1.72E-03
L63E-03
1.02E-02
5.04E-03
6.27E-03
1635--02
2.10E-03
1.94E-03
1.71E-03
1.19F-03
7.40R-03
3.67443
4.55E:00
1.195-02
1.538.03
1.39
1.25%:03
WP2- FVLC
1.53E-01
4,99E-02
1.36E-01
6.67E-02
1.43-001
1.73-001
1.40E-01
6.37E-02
7.62E-022
2.2911-02
2.02F-02
2.08E.02
6.091--02
5.226-03
1.04-002
5.31E-03
3.377-02
1.84--02
1466-02
1.165-02
6.80-0
2955-08
1.06F-02
2,368-03
5. 5.196-02
1.46--22
1.31E-02
9.3 9.34E-03
5.30E-02
3,24E-03
6.83E-03
2.12E-03
4.57E-02
12
02
1.01E.02
5.015-02
3.46-03
6.300.03
2.94E-03
2.73--02
7,108-03
40000-0
2.424-02
1,91:088
3.9718-33
2. 2,366-03
2.476-02
8.00E-03
5.29E-03
S.25E-03
2.40B-02
1. 1.70E-03
3.83E-03
1 1.965-03
1.10-002
8.72E-00
3.35E:03
4.55E-03
1.050-02
1.33-003
1.55E.03
1229:-03
-V-C
1.68--01
1,388-011
811--0022
1
2,236-01
1,238-01
8,095-02
6.39E-02
2. 2.55E-02
2.56E-02
3.5TE-02
8.04E--2
1.2 L.27--02
L.6E-02
9. 80E-00
3.826-02
3.91E-02
1.535-02
2.165-02
1.776-02
7666E-03
6.99E-03
5 5.888-03
163E-02
1.73E-02
6.35E-03
0-35
3-305-03
3.01E-03
2.59E-03
2.04F-022
2.10--02
8.351-00
1.88.00
2.675-02
3.77P-03
3.188.03
1.238-02
1.277--02
5.095-03
6. 6.948-008
2448-018
2.35E-03
1,94E-03
1.26E-02
1.09E-02
5.07E-03
7.00E-03
1.596-02
2.536-03
2.29E-03
1995-00
8.02E-03
8.2913-03
3.24FF-03
1.36F-03
1.046:022
1.598.03
1.50F-03
1.25E:03
NPI1050NN2-PVVL FVLC
1.17B-01
8 6262-002
1.938-01
1 1.32B-01
1.765-01
3-45E-01
2.53E-01
1,175-01
5.97E-02
6.280-02
1.4E-01
9,94E-02
1-44--0
2.58E001
1.86E-01
8,24E-02
3 3.76R-022
6. 6.34
18P-001
1.03-001
1:11F-01
2638-1
1.90P-01
8.13R-02
3.528-00
2.54E-02
6.03E-022
4. 4.135-02
5.085-02
1,77-001
7.636-02
3.318-02
9,21E-02
5-45-00
1,28E-01
S.56E-02
1044-001
2.24B-01
1600-01
6.82E-02
3.15E-02
2.002-02
6.881-022
4.718.02
5.761--02
1.218.11
8.86E-02
3.72E-02
1.53H-02
18188-02
2,788:022
898-02
2.5785-02
4886-02
3,626-02
1.55K-02
L18E-02
9.41E-03
2.06E-02
L44E-02
L87E-02
3.79E-02
2.74E-02
L18E-02
WPH:0S)) FVLC
1. 1.36F-01
1.05P-01
2.22F:011
LSE..01
1.955-01
3.54P-01
2.62P-01
1.44-01
7.378-00
5.75E-02
1,29E-01
9 9.24E-02
1.218-01
2.345-01
1,74-011
7,308-02
7.85E-02
5.85E-02
1.23--01
9.17E-02
1.14--01
2,43-001
L70E-01
6.E0-202
4.59|E-02
3.670-002
7.621--02
5,57E-02
6.74B-02
L41E01
1.88-01
4.23E-02
5.818-02
4, 4.52H-02
699--02
7.118-02
8.995-02
1383-001
1,345-01
5
5.84E--2
4.53E-02
9.75E-02
7.17E-02
8.9-1B-02
L8SE-01
1336-01
5.1E5002
2.91E-02
2.31E-02
4.85T-022
3. 3.56E-02
4.46E-02
9.330-02
6.78E.02
2.818.02
1433-002
1.118-02
2.39F:02
1.778-02
2.211:.02
4.54E-02
3.353-02
L344-02
Table 3: The ablation study of FVLC. We assess the efficacy of Lz, Lz, and La in Equation when applied alongside perturbations to both the word and input.
6.4 ABLATION STUDY
In the ablation study, conducted a thorough evaluation of each module (regularization) outlined in Equation ( Our evaluation aimed to assess the significance and effectiveness of each module in enhancing the performance of our model. To accomplish this, we initially designated Lı as the primary loss function, which is defined as the first term in Equation (2). Subsequently. we systemat- ically examined various combinations by selectively removing L2. C3. and L,. The outcomes of our study, presented in Table unequivocally demonstrate that each regularizer incorporated in our ob- jective function is both indispensable and effective. Each module contributes uniquely to the overall performance improvement of the model. Of particular significance is the inclusion of L4. which sig- nificantly enhances the model's stability. By ensuring the preservation of important concepts even under certain perturbations, the addition of La guarantees a higher level of robustness and stability in the model's predictions. This finding showcases the critical role played by this regularization module in refining the model's performance.
7 CONCLUSION
In this paper, we present comprehensive definition of the Faithful Vision-Language Concept (FVLC), which establishes a robust and accurate interpretation of deep learning predictions. Our definition encompasses four key properties: similarity of explanation, stability of explanation, close- ness of prediction, and stability of prediction. To obtain FVLC, we propose a novel method and evaluate it extensively through comprehensive experiments. Experimental results demonstrate that FVLC achieves comparable accuracy to the baseline while exhibiting stronger concept stability, indicating that FVLC is more faithful explanation tool.

---

Published as a conference paper at ICLR 2024
8 ACKNOWLEDGEMENTS
Di Wang, Junxiao Wang, Songning Lai, and Lijie Hu are supported in part by the baseline fund- ing BAS/I/1689-01-01, funding from the CRG grand URF/14466--11001 FCC/1/1976-49-01 from CBRC, and funding from the Al Initiative DE1/1/4811-10000 of King Abdullah University of Sci- ence and Technology (KAUST). Di Wang, Junxiao Wang, and Lijie Hu are also supported by the funding of the SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence (SDAIA-KAUST AI).
REFERENCES
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
Muhammad Aurangzeb Ahmad, Carly Eckert, and Ankur Teredesai. Interpretable machine learn- ing in healthcare. In Proceedings of the 2018 ACM international conference bioinformatics, computational biology, and health informatics, pp. 559-560, 2018.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Asso- ciates, Inc., 2020. URL https::/procesd/t/e www.innn....euureeeee cctppaper/2220//ilee 1457c0d6bfcb49611 1o1b04011110aa44223622
22t.a//eec..44A1111ooaaa11142222 42f64a Paper. Paper.
Longbing Cao. Ai in finance: challenges, techniques, and opportunities. ACM Computing Surveys (CSUR), 55(3):1-38, 2022.
Kushal Chauhan, Rishabh Tiwari, Jan Freyberg, Pradeep Shenoy, and Krishnamurthy Dvijotham. Interactive concept bottleneck models. In Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 37((), pp. 1948-5955 2023.
Ann-Kathrin Dombrowski, Maximilian Alber, Christopher J. Anders, Marcel Ackermann, Klaus- Robert Müller, and Pan Kessel. Explanations can be manipulated and geometry is to blame. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Informa- tion P Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 13567-13578, 2019.
Finale Doshi Velez and Been Kim. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv: 1702.08608. 2017.
Amirata Ghorbani, Abubakar Abid, and James Y. Zou. Interpretation of neural networks is fragile. In The Thirty-Thine AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-Firsl Inno- vative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu. Hawaii, USA, January 27 February 2019, pp. 3681 3388-3688. AAAI Press, 2019.
Tao Guo, Song Guo, and Junxiao Wang. pfedprompt: Learning personalized prompt for vision- language models in federated learning. In Proceedings of the ACM Web Conference 2023, pp- 1364-1374, 2023a.
Tao Guo, Song Guo, Junxiao Wang, Xueyang Tang, and Wenchao Xu. Promptfl: Let federated learning in age of foundation
participants cooperatively learn prompts instead of model. IEEE Transactions on Mobile Computing, 2023b.
Marton Havasi, Sonali Parbhoo, and Finale Doshi- Velez. Addressing leakage in concept bottleneck models. Advances in Neural Information Processing Systems, 55:23386-23397 2022.
10

---

Published as a conference paper at ICLR 2024
Bernease Herman. The promise and peril of human evaluation for model interpretability. arXiv preprint arXiv:1711.07411 2017.
Lijie Hu, Ivan Habernal, Lei Shen, and Di Wang. Differentially private natural language models: Recent advances and future directions. arXiv preprint arXiv:2301.0911 2023a.
Lijie Hu, Yixin Liu, Ninghao Liu, Mengdi Huai, Lichao Sun, and Di Wang. Improving faithfulness for vision transformers. arXiv preprint 2r 2023b.
Lijie Hu, Yixin Liu, Ninghao Liu, Mengdi Huai, Lichao Sun, and Di Wang. Seat: Stable and explainable attention. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11): 12907-12915, Jun. 2023c. doi: 10.1609/aaii v37il
Alon Jacovi and Yoav Goldberg. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4198 4205,, 2020.
Mert Keser, Gesina Schwalbe, Azarm Nowzad, and Alois Knoll. Interpretable model-agnostic plau- sibility verification for 2d object detectors using domain-invarianl concept bottleneck models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 3890-3899, 2023.
Eunji Kim, Dahuin Jung, Sangha Park, Siwon Kim, and Sungroh Yoon. Probabilistic concept bot- tleneck models. arXiv preprint arXiv:2306.01574 2023.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang. Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning, pp. 5338-5348. PMLR, 2020.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Songning Lai, Xifeng Hu, Jing Han, Chun Wang, Subhas Mukhopadhyay, Zhi Liu, and Lan Ye. Pre- dicting lysine phosphoglycerylation sites using bidirectional encoder representations with trans- formers & protein feature extraction and selection. In 2022 15th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BM)) BME-B, pp. 1-6. IEEE, 2022.
Songning Lai, Xifeng Hu, Yulong Li, Zhaoxia Ren, Zhi Liu, and Danmin Miao. Shared and pri- vate information learning in multimodal sentiment analysis with deep modal alignment and self- supervised multi-task learning. arXiv preprint arXiv:2305.08473. 2023a.
Songning Lai, Xifeng Hu, Haoxuan Xu, Zhaoxia Ren, and Zhi Liu. Multimodal sentiment analysis:
A survey. Displays, pp. 102563, 2023b.
Songning Lai, Jiakang Li, and Yonggang Lu. A comprehensive review of community detection in graphs. arXiv e-prints, pp. arXiv-2309, 2023c.
Jia Li, Lijie Hu, Jingfeng Zhang, Tianhang Zheng, Hua Zhang, and Di Wang. Fair text-to-image diffusion via fair mapping. arXiv preprint 21Xiv:2311.17695 2023.
Qing Lyu, Marianna Apidianaki, and Chris Callison- Burchh. Towards faithful model explanation in nlp: A survey, arXiv preprint arXiv:2209. 1132,, 2022.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
Sachit Menon and Carl Vondrick. Visual classification via description from large language models. In International Conference on Learning Representations, 2023.
Tuomas Oikarinen and Tsui- Wei Weng. Clip-dissect: Automatic description of neuron representa- tions in deep vision networks. arXiv preprint arXiv:2204. 10965, 2022.

---

Published as a conference paper at ICLR 2024
Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui-Wei Weng. Label-free concept bottle- neck models. arXiv preprint SxXiv:2304.061229 2023.
Yoshihide Sawada and Keigo Nakamura. Concept bottleneck model with additional unsupervised concepts. IEEE Access, 10:41758-41765, 2022.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 5319-3328. PMLR, 2017.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The caltech-uesd birds-200-2011 dataset. Technical Report CNS-TR-201 TR-2011 1-001, California Institute Technology. 2011.
Sarah Wiegreffe and Yuval Pinter. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNIPP), pp- 11-20, 2019.
Haoxuan Xu, Songning Lai, Xianyang Li, and Yang Yang. Cross-domain car detection model with integrated convolutional block attention mechanism. Image and Vision Computing. 140:104834, 2023a.
Xilie Xu, Keyi Kong, Ning Liu, Lizhen Cui, Di Wang, Jingfeng Zhang, and Mohan Kankanhalli. An Ilm can fool itself: prompt- based adversarial attack. arXiv preprint arXiv:2310.13345, 2023b.
Shu Yang, Muhammad Asif Ali, Cheng-Long Wang, Lijie Hu, and Di Wang. Moral: Moe augmented 1260, lora for IIms' lifelong learning. arXiv preprint arXiv:2402.1 12660, 2024a.
Shu Yang, Lijie Hu, Lu Yu, Muhammad Asif Ali, and Di Wang. Human-ai interactions in the communication era: Autophagy makes large models achieving local optima. arXiv preprint arXiv:2402.11271, 2024b.
Buih Yeh, Cheng- Yu Hsich, Arun Sai Suggala, David I. Inouye, and Pradeep Ravikumar. On the (in)fidelity and sensitivity of explanations. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 10965-10976, 2019.
Fan Yin, Zhouxing Shi, Cho-Jui Hsieh, and Kai-Wei Chang. On the sensitivity and stability of model interpretations in nlp. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 2631-2647, 2022.
Kun-Hsing Yu, Andrew I. Beam, and Isaac Kohane. Artificial intelligence in healthcare. Nature biomedical engineering, 2(10): 2018.
Mert Yuksekgonul, Maggie Wang. and James Zou. Post-hoc concept bottleneck models. arXiv preprint wxXi:22055.15480 2022.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 mil- lion image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
A MORE DISCUSSION ON FAITHFUL CONCEPT
What is "faithful concept"? As we can see from the previous discussion since the key component of Label-free CBM is the concept bottleneck layer, to get above faithful Label-free CBM, it is sufficient for us to get a faithful concept bottleneck layer, which called "faithful concept". Before diving into rigorous definition of "faithful concept", first need to intuitively think about what properties it should have.
The first one is keeping interpretability similar to the vanilla CBL. In the vanilla CBL for an input image, we can easily see that the rank of entries in the output of CBL, i.e., concept vector, can reflect the importance of each concept in the concept set. Thus, the "faithful concept" should also have almost the same order for each entry as in the vanilla one. However, keeping the rank for all entries is too stringent, motivated by the fact that the interpretability and the prediction always rely
12

---

Published as a conference paper at ICLR 2024
on the most important entries. Here, relax the requirement to keep the top-k indices almost unchanged.
In fact, such property is not enough its interpretability could also be unstable to different per- turbations. Modeling such instability is challenging as the perturbation could be caused by multi- resources, which is significantly different from the adversarial robustness. Our key observation is that wherever a perturbation comes from, will subsequently change the output of CBL. Thus, the interpretability of "faithful concept", i.e., the top-k indices, resilient to noise, we can naturally think it is robust to those different perturbations.
However, even with the previous properties, is still insufficient. The main reason is that keep- ing interpretability does not indicate keeping the prediction performance. This is because keeping interpretability can only guarantee the rank of indices unchanged, but cannot ensure the magnitude of these entries, which determine the prediction unchanged. For example, suppose the vanilla con- cept vector is (0.5, 0.3, 0.2), then the above "faithful concept" vector might be (0.9, 0.051, 0.049), which is significantly different from the original one. Based on these, we should also enforce the prediction performance, i.e., the output distribution, to almost the same as the vanilla one. More- over, its output distribution should also be robust to perturbations motivated by Figure
Based on our above discussion, our takeaway is that the "faithful concept" vector should make its top-k indices and output distribution almost the same the vanilla concept vector while also being robust to perturbations. Based on the above intuitions, we can translate the previous intuitions into rigorous mathematical definitions. Specifically, call the above "faithful concept" Faithful Vision-Language Concept (FVLC).
Differences with adversarial robustness. While both FVLC and adversarial robustness consider perturbations or noises on input data. There are many critical differences: (1) Not only the predic- tion, should also make the interpretability stable and close the vanilla concept vector. Due to these additional conditions, our method for achieving FVLC is totally different from the meth- ods in adversarial robustness, such as certified robustness or adversarial training. See Section B for details. (2) The way of modeling robustness in FVLC is also totally different from adversarial robustness. In adversarial robustness, it usually models the robustness to perturbation on input data. However, in FVLC, due to the requirement on interpretabilityy i.e., the top-k indices of the vector, we cannot adopt the same idea. Firstly, directly requiring the top-k indices robust to perturbation on the input will make the optimization procedure challenging (which is minimax optimization prob- lem) as the top-k indices function is non non-diffceet con-differentiable and calculating the gradient of the backbone neural network is costly. Secondly, rather than perturbation input data, perturbation could come from multi-resources, such as embedding perturbation or combination of perturbations. Thus, from this perspective, our stability of interpretability is more suitable for "faithful concept".
mentioned, the
B OPTIMIZATION FOR FVLC LAYER
In the last section, we presented rigorous definition of FVLC. To find such FVLC, propose to formulate a min-max optimization problem that involves the four conditions in Definition [ Specifically, the formulated optimization problem takes the third condition (closeness of prediction) as the objective and subjects it to the other three conditions. Thus, we can get rough optimization problem according to the definition. Specifically, we first have
min E, (l,(,,)), We
is the basic optimization goal, that is, we want to get vector which has similar output prediction with the vanilla CBM for all input æ. If there is no further constraint, then the minimizer of (3) is just the vanilla CBM matrix W, that optimizes] [ We then consider constraints for this objective function:
Equation 33
13

---

Published as a conference paper at ICLR 2024
Notation
Remark
Notation
Remark
concept set concept vector
an input data representation which transforms from the input space con- cept space
9 R2
R
f : IR - Rd.
representation which maps from the concept space the prediction space
è
(
f(g(z))
M
concept activation matrix
Fj
image encoder
Fr
text encoder original concept embedding matrix
bS())
backbone features extractor
W.
WF
weight matrix the output layer
W.
FVI.C surrogate loss of -V;
VE
top-k overlap function
Lk
divergence metric
g(x)
(ob
((z)
y(
Weg(a)
((
Wr9(2)
((x, E
W ( + 8)
A3,
regularization parameters
R, ax, 3, k1, 12
parameters in FVLC
Table 4: Table notations.
Va s.t. max D(y(z, ((, a2. 1oll
(4)
VA, (o()).g())) 31:
(5)
max VA, (f(x), + +p) 825
where ((x) (x), and y(2, 2) WFg(z)) v(x, (
Equation 4 is the constraint of stability, Equation 5 corresponds to the condition of similarity
of explanation, and Equation 66) links the stability of explanation. Combining equations (
(6) and using regularization to deal with constraints, we can get the following min-max stochastic optimization problem.
min EE (Dv(e, 2), v( c)) Vi (0(z).(()) + A2 max D(y(z,,),1le ( W.
|
- ~As max VA, ((x) ((x) p)]. R,
where 0, 0, and A are hyperparameters.
To solve this min-max optimization problem, we can generally use stochastic gradient descent- based methods to obtain the solution for the outer minimization and use PSGD (Projected Stochastic Gradient Descent) to solve the inner maximization. However, major challenge is that the top-k: overlap functions Vk, (z) (x)) and Via (g(x,, (z) + p) are non-differentiable, which makes impossible to use gradient descent. Therefore, need to consider surrogate loss for V++()).
Our goal is to design a surrogate loss function (x(-) for (-) that can be used training. We focus on the example of (C()) for - VV,(ē, c). One possible naive surrogate objective is to use distance metric, such as the fi-nomm, between č and c, i.e., L(ē) = l|è c/11- While this objec- tive preserve the top-k overlap when obtain the optimal near-optimal solution, it lacks consideration of the top-k information, making it a loose surrogate loss.
To address this issue, propose minimizing the distance between ē and constrained the top-k entries only. Specifically, we minimize ||csy Esslli |l1. where Cst, Čss E R are the vectors e and é, respectively, constrained on the top-k indices set S* of c. We use both top-k indices sets for both vectors to involve the top-k indices formation. Therefore, our surrogate loss function is:
Cx 27 (lles الyوة + ||āg
Coy I|)).
Note that besides the fo-nrmm achieves the best performance. Thus, throughout the paper, we only use L norm. By using this surrogate function, have our relaxed function:
other norms. However, in practice, find that -n-m
14

---

Published as a conference paper at ICLR 2024
min (2,l(a,,),,(a + A1 La, + max W.
Div(r, 2 6))
1
L3
+ A max Cb, (g()))())(()) +
R,
Inspired by PGD in Madry et al. (0188), to solve the above optimization problem, in each iteration, with fixed current model W (thus fixed ((x)) we have to first update ō and p. Specifically, in the y-th iteration for updating current noise 8 p-1 have:
6p = 6 6--1
Σ d BEA
6
A p-11
8 arg
min 115
8pl1
|
where A--1 is a batch and 7p a step size parameter for PGD.
Similarly, the PGD and the surrogate loss of Lx(.) get the optimal p* in the 1-th iteration of outer SGD.
Pq P BB--1
Σ 2E
V pp (2, ē Pq-1);
Poy arg arg min |p
Pqll:
||ellS
where By- is a batch and Ty is a parameter step size for PGD.
After we find 6p with P iterations and PQ with Q iterations, we then update W to W by using batched gradients. Details are given in Algorithm [ By using this algorithm, we obtain stable and faithful concept that can be trained in self-supervieed way on unlabeled data.
C EXAMPLE OF STEP 1
We apply various filters to enhance the quality and reduce the size of our concept set. The filters include:
Removing concepts longer than 30 characters.
Eliminating concepts that are too similar to target classes using cosine similarity in text embedding space, with a similarity threshold of 0.85.
Removing duplicate or synonymous concepts with cosine similarity threshold > 0.9.
For more information about the filters, please refer Oikarinen al. 2203
In Figure we show examples of the full prompt used for GPT-3 and partial output examples.
D MORE EXPERIMENTAL SETTINGS
D.I DATASETS
To evaluate the effectiveness our approach, we conducted training using the FVLC framework on four distinct datasets: CIFAR-10, CIFAR-100, CUB, and Places365. These datasets offer a di- verse range of tasks and challenges. CIFAR-10 10 and CIFAR-100, introduced by Krizhevsky et al. 2209), widely used for general image classification. They provide comprehensive set of la- beled images for training and evaluation purposes. CUB, on the other hand, focuses specifically on fininnd bird-species classification. It comprises dataset of 5900 training samples, each
15

---

Published as a conference paper at ICLR 2024
Algorithm Faithful Vision-Language Concept
Input: Weight matrix WF of the fully connected layer in a Label-Free CBM: Weight matrix W. for concept layer in a Label-Free CBM; backbone network bf(:); Training data D; parameters R1, R2, 22, h3, A1, Az, Às. Iterations number T, P.Q.
2: Initialize W via random sampling from the standard Gaussian matrix.
3: for t = 1, 2, ... I do
4: Initialize 66 and PG.
5: for 1,2, ... do
6:
Randomly sample batch Ap-1 C D.
7:
6p
00 5--1 1 +
Σ
VoD(y(z,,), (
Ap-1 #E
8 = arg
min 1 - Spl1
|| |
Sn
8:
end for
9:
for Q= 2, do
10:
Randomly sample batch By-1CU
Pg P
Tq Σ
V pphin Pq-1);
BBg-1
zEBp-1
Pg arg min l|ells
||p Pgll
12:
end for
13:
Randomly sample batch C D. Update é using Stochastic Gradient Descent
14:
Wt W T
Σ
V W. ((y((,,),)(())) W. + +1 V (((x) w.
aEC,
+ A2 W. D(y(z, vo, č 8p)) + As (7()) g(x) P w.
15: end for Return: ~ = Ōr.
16:
accompanied by annotations that describe 312 concepts associated with the bird species. These con- cepts include attributes such wing color (e.g.. blue) and head pattern (c.g., spotted). In contrast, Places365 primarily geared towards scene recogmition. encompasses a much larger scale, with 1-2 million training images available for use. The dataset also provides annotations that describe various concepts related to scenes. Despite the availability of concept names and annotations in the CUB and Places365 datasets, we intentionally choose not utilize this information during the training of our FVLC models. Our objective was to demonstrate the capability of our approach to perform effectively without relying on such labels. Surprisingly, our method similar concepts and achieve competitive performance when compared to methods that leverage the provided concept information. By training FVLC on these four datasets, which encompass gen- eral image classification (CIFAR- 10R100110 00), fine-grained bird-species classification (CUB), and scene recognition (Places365), we aimed to evaluate the effectiveness and versatility of our approach com- prehensively. For other fields, such as sentiment analysis (Lai et al. 2023a Hu et al.) 2023a), autonomous driving (Xu et al. 2023a), and social good (Lai et al. 2023c Li et al. 2023). ) we leave it in future work.
able discover
D.2 BACKBONE
To ensure fair and consistent comparison with previous work, we adopted specific backbone mod- els for the CIFAR and CUB datasets, as done by Yuksekgonul et al. (2022). For CIFAR, we utilized the CLIP (RN50) image encoder the backbone model. This choice allows us establish direct comparison with the results reported in the mentioned study. Regarding the CUB dataset, we em- ployed the 18 architecture trained CUB from fincclsob), which has been widely used
16

---

Published as a conference paper at ICLR 2024
List the most important features for recognizing something horse:
List the most important features for recognizing something dog:
Four legs
Fur hair
Mane and tail
Pointed ears
Hooves
Snout muzzle
Large, muscular body
Tail
List the things most commonly seen around horse:
List the things most commonly seen around a dog:
Saddle
Leashes or collars
Bridle
Dog food or treats
Hay bales
Toys, such as balls or chew toys
Water trough
Dog beds or blankets
Give superclasses for the word horse:
Give superclasses for the word dog:
Mammal
Living organism
-Animal
Animal
-Vertebrate
Natural object
Chordate
Vertebrate
Figure 4: Example of our Step 1.
in the literature for fine- fine-grained bird-species classification tasks, making a suitable choice for our evaluation. For Places365, we opted for the ResNet-50 architecture trained as our backbone net- work. This choice aligns with common practices in the field, ensuring consistency and allowing for meaningful comparisons with other approaches. In our framework, the number of concepts incorpo- rated into each model is roughly proportional to the number of output classes specifie to the task. each class introduces additional initial concepts, the number of concepts increases accordingly. For instance, the CIFAR-10 model integrates 128 concepts, while the CIFAR- 100 model employs 824 concepts. The CUB model incorporates 211 concepts, whereas the Places365 model incorporates 2202 concepts. pared the other datasets. This is because, in approach, only utilized the important features prompt for this particular dataset. Despite this reduction in the number of concepts, our method still achieves competitive performance CUB, showcasing the effectiveness of approach even with more limited concept space.
is worth noting that the CUB dataset has a smaller number of concepts com-
D.3 BASELINE
Standard. The standard model represents an image classification model, which extracts image features through the same backbone to complete the image classification task. fact, this is a common practice in the field of CBM. After adding the bottleneck layer, the accuracy of the image classification task will be affected due to the intervention of the concept task.
FVLC model and then connects fully connected layer
P-CBM. The Post-hoc Concept Bottleneck Model (P-CBM) is framework that converts pre- trained neural network into a concept bottleneck model. The PCBM consists of two main steps. First, there is pre-trained backbone model that maps inputs an embedding space. This backbone model can be any pre-trained model, such as an image encoder or a specific layer of a deep neural network. The second step involves learning a concept bank, which captures interpretable concepts relevant to the task. Concept representations can be obtained using techniques like Concept Activa- tion Vectors (CAVs) or multimodal models. CAVs are learned by training Support Vector Machines (SVMs) to distinguish embeddings with and without the concept. Multimodal models leverage text encoders to map concepts to vector representations. Once the concept bank is obtained, the em- beddings produced by the backbone model are projected onto the concept subspace defined by the concept vectors. This projection step aligns the embeddings with the interpretable concepts, creating bottleneck representation. Finally, interpretable predictor is trained to classify examples based on their projections onto the concept subspace. This predictor can be implemented using methods like sparse linear layers.
17

---

Published as a conference paper at ICLR 2024
D.4 SETTINGS
To give the details of our experimental setup, we provided Ta which lists the key parameters we used during our training and evaluation process. The values of these parameters have been selected based on previous research and experimental experience, and have been carefully adjusted to achieve optimal performance. Note that these parameters include only model architecture and optimizer type, but also important settings such as learning rate, batch size, number of training iterations, etc. Through Table the reader can understand the specific configuration of our experiment and can reproduce it if necessary.
Argument
Value
Remark
batch_size
512
hatch size used when saving model/CT.IP activations hatch size used when fitting final layer
saga batch size
256
proj batch size
5000
batch size use when learning projection layer concepts with smaller topS clip activation will be deleted how many steps to train the projection layer for concepts with smaller similarity to target concept will be deleted sparsity regularization parameter, higher- sparse how many iterations to run the final layer solver for
clip cutoff
0.25
proj_steps
1000
interpretability.cum .cutoff
0.45
lam
0.0007
n_iters
1000
rgd_radius
0.1
pgd_step
10
pgd_step_size
0.02
pgd norm type
l-infty
x pgd radius
0.05
x-pgd_step
10
x-pgd_step_size
0.01
x-pgd_norm_ttpe
l-infty
lambda_1
1.00--02
lambda
1 1.008-02
lambda_3
1.00E-02
lambda.4 4
1.00E-02
Table Model parameter configuration.
E DETAIL OF FIGURE
See in Figure 7678
MORE EXPERIMENTS
In order verify that FVLC faithful under the same concept space, we repeatedly generated concept sets to build different concept spaces and repeated the above experiments. The experimental results are shown the Table 6 7 seen that our FVI.C is more stable than other baselines against input perturbation and concept set perturbation, making it a more faithful interpretation. Moreover, our approach exhibits minimal accuracy degradation compared to vanilla CBM.
I0 [T] According the experimental results, it can be
G MORE EXPERIMENTS IN HAMI 0000
We report the performance of our model on the HAM10000 dataset in Table 12 interpretable output results samples HAM10000. The presentations are shown in the Figure
We also visualized
18

---

Published as a conference paper at ICLR 2024
Predttrck Card 0.955
Lagt:9.78 Brac--005
Thaller
CIFAR10
ditver
+3.35
NOT pueriaat wurs
+3.30
NOT
13.07
an engine
NOT passenger
+0.49
6.0
6.5
1.0
1.5
2.0
25
Concest conertuttions
Nedsocck Coné. 0.953 Logit9.62 Blas 0.90
Predtruck Coné: 0.954 Log4 Bles:0.94
trabler
NOT pointed earo
+3.17
driver
+1.18
an cogie
NOT poinoed cers
+1.31
MCT white the cost
+0.09
NOT of
+1.09
ongine
NOT nunanay
60.76
NOT nunsary
+0.74
0.0
0.5
1.0
1.5
7.0
2.5
0.0
11.5
1.0
1.5
7.0
2.5
Concact cont
Concept co butoors
The visualizations for concept weights and final layer weights on one sample from CI- FARIO. Top left, top right, bottom left and bottom right are the input image, the concept weight visualization without perturbation, the concept weight visualization with perturbation, and the opti- mized concept weight visualization with perturbation.
Figure 5:
Prodeeon
Legio4.35
CIFAR100
NOT a long, Bhick lal NOT various sizes shapes
+0.60
-0.53
-0.48
larga
a striped pattern
+0.29
NOT o large, HASASIA bedy -0.29
0.0
Prediman- Cont: 0.797 Logitt.00 D.a.
Predemann Cant. 0.518 Logg-.111 0lau
NOT lagg, Bony stapt
+0.00
NOT
NOT spocesri
love
NOT small round shape
+0.22
NO1 eboccores dantse
NDI
0.334
MDT stee
-0.34
ped pastern
other festunes
0.D
0.2
0.4
0.6
0.5
3.0
0.0
3/0
155
Cencept contrie
Figure 6: The visualizations for concept weights and final layer weights on one sample from CI- FAR100. Top left, top right, bottom left and bottom right are the input image, the concept weight visualization without perturbation, the concept weight visualization with perturbation, and the opti- mized concept weight visualization with perturbation.
19

---

Published as a conference paper at ICLR 2024
Bank, Swallww 0.346
Bas:0.02
sith acrobatic Right
+2.75
small, brewn
+1.92
CUB
NOT feathers
dark wingtips
NOT medium soed duck -0.24
05
0.0
2.0
23
2.0
25
30
Bank Swatlow
Cont 6,485 Logit 080
Biss: 0
Wrsl Bark Switlw
Card 0,444
Logit6 Bax
white
+2.69
sxtit. acmobaticc
+2.7
small.
white underparts
white from
HOT inddssoet floth ers
NOT red pellow expis
-0.25
NOT ned and yellow wingrip
+0.43
blue
NOT medium-soed duck
other festures
Sum 200 features
0.
0.5
1.0
1.5
2.5
2.5
-85
0.0
B5
1.0
2.0
25
Concept aiooo
Butter
Figure 7: The visualizations for concept weights and final layer weights sample from CUB. Top left, top right, bottom left and bottom right are the input image, the concept weight visualization without perturbation, the concept weight visualization with perturhation, and the optimized concept weight visualization with perturbation.
Prodsoperating room
0.352 Logit6.59 Blas-L.06
Places365
surgery
+2.52
anesthesth
couipment
large, starte space
bable
+0.99
hospital gowe
Sum of 2356 ether features
mndsspporting room
Logit Bios-1107
starle surfeces
#3.93
surgery
+2.75
laberstory
+1.57
coflty poog es and lab coats
+0.56
lange: stevile space
+2.00
lrrg storie space
+0.47
surgical table
+1.19
+0.14
gownn
ahoorerhead
a nursing station
Sum 21414 other features
+0.68
Sum of 2151 other features +0.1
Concect contributions
Figure 8: The visualizations for concept weights and final layer weights sample from Places365. Top left, top right, bottom left and bottom right are the input image, the concept weight visualization without perturbation, the concept weight visualization with perturbation, and the opti- mized concept weight visualization with perturbation.
20

---

Published as a conference paper at ICLR 2024
Method
CIFARIO
CIFARI00
CUB
Places365
(Na interpretabilitty
88.80%
70.10%
76.70%
48.56%
P-CBM(CLIP)
84.50%
56.00%
N/A
N/A
LabeI-frec CBM
86.32%
65.42%
74.23%
43.63%
WPI(5%) base
86.26%
65.25%
73.92%
43.52%
WPI(5%) FVLC
86.45%
65.59%
73.74%
43,47%
WPI(10%) base
85.88%
65.05%
74.27%
43.49%
WP1(10%) FVL.C
86.27%
64.65%
73.66%
43.46%
WP2
86.09%
65.27%
73.62%
43.42%
WP2 FVLC
86.41 %
65.52%
74.55
44.56%
IP base
86.31
65.67
74.51%
43.58%
IP FVLC
86.70% %
65.46%
74.11%
43.76% %
WPI(5%)+WP2 base
86.22%
65.18%
73.78%
43.66%
WPI(5%)+ WPP FVLC
86.50%
65.65%
73.66%
43.63%
WPI(10%) + base
86.21 %
65.11%
73.63%
43.63%
WPI(10%)+W12 FVLC
86.60%
65.18%
74.19%
43.60%
WPI(10S))P221IP base
85.63%
64.72%
74.05%
43.20%
WPI(I0%)+WP2-IP FVLC
86.48%
65.28%
74.42%
43.33%
Table 6: The table shows the accuracy of baseline and FVLC before and after perturbation under four benchmark datasets. The first is the standard backbone image classification model without in- terpretability. P-CBM (Yuksekgonul et al. 2022) and Label-free CBM OOkarinen et al. 2023) the latest CBM models with interpretability. and the process of generating concept sets is unsuper- vised and does not require manual labeling. The percentages in parentheses the degree of added WP1. should be emphasized that this is repeated experiment under the new-1 concept space.
Method
CIFARI0
CIFARI00
CUB
Places365
(Na interpretability)
88.80%
70.10%
76.70%
48.56%
P-CBM(CLIP)
84.50%
56.00%
N/A
N/A
Label-frec CBM
86.32%
65.42%
74.23%
43.63%
WP1(5%) base
86.65%
65.45 %
74.44%
43.40%
WP1(5%) FVLC
86.01 %
65.56%
73.83%
43.60%
WP1(10%) base
86.20%
65.08%
73.62%
43.51%
WPI(10%) FVLC
86.19%
64.72%
73.75%
43.77%
WP2
86.08%
64.89%
74.09%
43.63%
WP2 FVLC
86.31 %
65.07%
74.65%
44.73%
base
86.88%
65.43%
74.49%
43.59%
IP FVLC
86.95 %
65.37%
73.98%
43,90%
WP1(5%)+P22 base
86.45%
65.20%
73.71%
43.80%
WPI(5%)+WP2 FVLC
86.76%
65.12%
74.21%
43.56%
WPI(10%++WP2 base
86.11%
64.89%
74.18%
43.51%
WPIC10%)+WP2- FVLC
86.28%
65.31%
73.89%
43.46%
WP111012PP2 base
86.08%
64.59%
74.03%
43.12%
WPI(I0 FVLC
86.86%
64.87%
74.71
43.38%
Table 7: The table shows the accuracy of baseline and FVLC before and after perturbation under four benchmark datasets. The first is the standard backbone image classification model without in- terpretability. P-CBM (Yuksekgonul et al. 2022) and Label-free CBM (Oikarinen et al.) 2023) the latest CBM models with interpretability. and the process of generating concept sets is unsuper- vised and does not require manual labeling. The percentages in parentheses the degree of added WP1. should be emphasized that this is repeated experiment under the new-2 concept space.
21

---

Published as a conference paper at ICLR 2024
Method
CIFAR10
CIFAR100
CUB
Places365
(No interpretability)
88.80%
70.10%
76.70%
48.56%
P-CBM(CLIP)
84.50%
56.00%
N/A
N/A
Label-free CBM
86.32%
65.42%
74.23%
43.63%
WP1(5%) basc
86.29%
65.06%
73.81
43.74%
WP1(5%) FVLC
85.99%
65.59 %
74.12%
43.53%
WPI(100% base
85.91
65.14%
73.82%
43.83%
WPI(10%)- FVIC
86.02%
64.83%
73.74%
43.70%
WP2
86.40%
65.36%
74.04%
43.44%
WP2 FVLC
86.32%
65.51%
74.70%
44.48
IP base
86.46%
65.36%
74.66%
43.64%
IP FVLC
86.75%
65.60%
74.11%
43.74%
WPI(5%)+WP2 base
86.90%
65.40%
74.18%
43.52%
WPI(5%))WP2 FVLC
86.57%
65.20%
73.89%
43.53%
WPI(10%))=P2 base
86.25%
64.72%
73.82%
43.65%
WPI(10%))WP2- FVLC
86.53%
65.20%
73.75%
43.34%
WPI(100))WW2-1IP base
85.15%
64.18%
73.85%
43.43%
WPI(I003)WW22-PP FVLC
86.89%
65.29%
74.12%
43.56%
Table 8: The table shows the accuracy of baseline and FVLC before and after perturbation under four benchmark datasets. The first is the standard backbone image classification model without in- terpretability. P-CBM (Yuksekgonul et al. (2022) and Label-free CBM (Oikarinen et al., 2023) the latest CBM models with interpretability. and the process of generating concept sets is unsuper- vised and does require manual labeling. The percentages parentheses the degree of added WP1. It should be emphasized that this is repeated experiment under the new-3 concept space.
CIFARIO
CIFARIOD
CUB
Places365
Method
TCPC
TOPC
TCPC
TOPC
TCPC
ТОPC
TCPC
TOPC
WP1(5%) buse
1.1E-01
6.38E-02
L04E-01
6.92E-02
1.88-00
1.76-01
1.65--01
6.61E-02
WPI(5%) FVLC
LITE-03
8.96-03
2.776-03
4,38E:03
1085402
1.465-03
1,445-03
1.2515-33
WPI(10S) hase
2.06E-01
8.51E-02
1.926-01
1.32-001
2.39E:01
3.376-01
2.16E-01
1.15-01
WPI(10%) FYLC
L17E-03
7.07E-03
3.85E-03
4.34E-03
1.22-002
1.55E-03
1.33E-03
1.25E-03
WP2 hase
1660-01
4.77E-02
1.33--01
6.53E-02
1.388-01
1.718-01
1.43E-01
6.10F-02
WP2 FVLC
L06E-02
8.67E-03
3.11E-03
4,55E-03
L.7E-02
1.60E-03
1.56E-03
1.35E-03
IP base
L70E-01
6.02E-02
1.35E-01
8.87E-02
1.76E-01
2.19E-01
1.65E-01
7,94E-02
IP FVLC
7.826-03
8.43-033
3.3016-03
4,461-03
1.03E-02
1,551:03
1,53E-03
1.27E-03
WP105%)+ base
178E-01
3.60E-02
1.24E-01
6,58E-02
L46E-01
1.86E-01
1.64-001
6.36E-02
WPI(5%) WP2- FVLC
1255-00
7.76E-03
3.845-03
1,56B503
1,58E-03
1.251:-03
WPI(103>=W22 base
1.215-01
8.50E-02
1.938-01
1.28E-01
1.76E-01
3.326-01
2.53E-01
1.12--01
WPIC109HHP22 FVLC
L14E-02
9.78E-03
2.11E-02
1.45E-02
194E-02
3.88E-02
2.68E-02
1.21E-02
WPI(10%) WPI(1S))WPPHIIP FVLC
1.39E-01
1.03E-01
2.32E-01
1.56E-01
1.948-01
3.57E-01
2.55E-01
1.42E-01
1.415-02
0.15:-02
2.47H-02
1.75E-02
2.138-02
4.70F:02
3,44F-02
1,28E-02
Table 9: The table shows the TCPC and TOPC of baselines and FVLC under four benchmark datasets with different perturbations. The percentages in parentheses are the degree of added WP1. should be emphasized that this a repeated experiment under the new-l concept space.
22

---

Published as a conference paper at ICLR 2024
Method
CIFARIO
CIFARI00
CUB
Places365
TCPC
TOPC
TCPC
TOPC
TCPC
TOPC
TCPC
TOPC
WPI(5%) buse
1477-01
6.19E-02
1.07-001
7.10E-02
1,226-01
1.82-01
1,6E-01
6.77E-02
WPI(5%) FVLC
1.07E-03
8.95E
2,796-03
4.316:03
1.096:02
1.409-03
1466:03
1.26F-03
WPI(10%) base
2.12E-01
8,40E-02
184E-01
127E-01
2.37E-01
3.44E-01
2.09E-01
1100-01
WPK10%) FVLC
L.1E-03
6.73E-03
3.848-03
4.36E-03
1.16--02
1.56E-05
1.39E-03
1.24E-03
WP2 buse
161E-01
4.74E-02
1.361-01
6.23E-02
1.33E-01
1,6819001
1,46E-01
6.04E-02
WP2 FVLC
1.05E-02
8.57E-03
3.36E-03
4.60E-03
1.08E-02
1.67B-03
1 1.54E-03
1.35E-03
IP base
168E-01
5,93E-02
1.371-01
8,54E-02
1 16695-01
2.19E-01
1.71E-01
7.98E-02
IP FVLC
8.04E-03
8.04E-03
3.296:03
4.34E:03
1.05:002
1.565-03
1.49E:03
1.26F-03
WP1(5%)+ WPP base
187E-01
3.43E-02
1.20E-01
6.33E-02
1.33-01
1.87E-01
1.61E-01
6.33E-02
WPI(5%)=WP2 FVLC
1226-02
8.12E/03
3.85E-03
4.34E-03
9.40F:02
1,549-03
1.59F/03
1,22F:03
WPH100)) base
1220-01
8.33E-02
2.01E-01
1.33E-01
1.68E-01
3.43E-01
2.53E-01
1.12E-01
WPIC100)4W22- FVLC
L12E-02
9.32E-03
2.14E-02
1.41E-02
2.01E-02
3.72E-02
2,65E-02
1.17--02
(PI(105)=WP22HH hase
1.405001
9.84F-02
2.38F-01
1,50F-01
1.875-01
3.49H:01
2.638-01
1.35H-01
WPIC0000WP2HIP FVLC
1.35E-02
1.17E-02
2.59E-02
1.83E-02
2.10E-02
4.86E-02
3,45E-02
1.28E-02
Table 10: The table shows the TCPC and TOPC of baseline and FVLC under four benchmark datasets with different perturbations. The percentages in parentheses are the degree added WP1. It should be emphasized that this is repeated experiment under the new-2 concept space.
CIFARIO
CIFARI00
CUB
Places365
Method
TCPC
TOPC
TCPC
TOPC
TCPC
TOPC
TCPC
TOPC
WP1(5%) hase
L44E-01
6.44E-02
100B-01
6.61E-02
1,25E-01
1.81-01
1.72E-01
6.58E-02
W
1.1E-03
8,58E-03
2.68E-03
4,50E-03
L06E-02
1.5E-003
1.42E-03
1.30E-03
WPI(10%)- base
2.08E-01
8.31E-02
1855-01
1.34E-01
241E-01
3.40E-01
2,26E-01
1.21E-01
WPI(10) FVLCC
1.22F-03
6.73E-03
3.88
4.31E:03
1,258002
1,539-03
1.25E-03
WP2 buse
1.58--01
4.89E-02
1.376-01
6.83E-02
1.22-001
1.67E-01
1.44E-01
6.17E-02
WP2 EVEC
1.035-02
8.738403
3.24H-03
4.34F/03
1.04//22
1.659503
1,55E/03
1,35K-03
IP base
1.78-01
6.11E-02
1.30E-01
9.07E-02
1 1880-01
2.23E-01
1.62E-01
7.94E-02
IP FVLC
7,49-003
8.14E-03
3.42E-03
4.28E-03
1.4E-02
1.48E-03
1.54E-03
1.22E-03
WP1 base
1.82E-01
3.73E-02
1.30B-01
6.61E-02
1.50E-01
1.82E-01
1.63E-01
6.17E-02
WPI(5%)) FVLC
125E-02
8.1SE-03
3.91E-03
4.57E-03
9.85E-02
1.52E-03
1.64E-03
1.19E-03
WPI(10%) WP2 WP2
base
1.15-001
8.60E-02
1.87E-01
1.26E-01
1.76E-01
3.20E-01
2.64E-01
1.07E-01
WPICU090 FVLC
9,70F-03
2.176/02
2.02F-022
1,25E-02
WPI(10%)== WP2+II base
1.38E-01
1.07E-01
2.22E-01
1.62E-01
2.01E-01
3.61E-01
2.62E-01
1.41E-0
WPIC006)WP2-IIP FVLC
1.36--02
1,20E-02
2,39E-02
1.70E-02
2.22E-02
4.57E-02
3,50E-02
1.27E-02
The table shows the TCPC and TOPC of baselines and FVLC under four benchmark datasets with different perturbations. The percentages in parentheses are the degree added WP1. It should be emphasized that this is repeated experiment under the new-3 concept space.
Table 11:
Model
HAM100000
Standard (No interpretability)
96.30%
P-CBM
94.70 %
Label-free CBM
94.38%
FVLC
94.42%
Table 12: The performance of our model on the HAM10000 dataset,
23

---

Published as a conference paper at ICLR 2024
Look
HAM1 10000
orsted
+3.33
roleed uubhe
+2.23
afrmn hard consistancy
piomented demenologii
Concept Costribtions
dome Mape
+2.67
sough crolted botur
bump
whtte color
conaittency
+0.35
fimm of hard contieercy
pink, coler
unaied,, and oaaly surface
pigmonkal sigmanted
+0.25
piginerited or non-pigneetdd
+6.22
mund not Chape
+0.19
smooth surface
MOT a nucder
+0.37
o27 Reatures
Sum 126 other Reatures
1.0
30
Concest
Conkect C0TSTEEEEEO
Figure 9: Explanations for one randomly chosen input image for our model trained on HAM10000. The visualizations for concept weights and final layer weights on one sample from HAM10000. Top left, top right, bottom left and bottom right are the input image, the concept weight visualization without perturbation, the concept weight visualization with perturbation, and the optimized concept weight visualization with perturbation.
24


Reviews:
[{"review_id": "8zbFD4RE5q", "reviewer": "Reviewer_AK8W", "summary": "In this paper, the authors have introduced a new approach to ensure the stability and robustness of label-free concept bottleneck models. The authors introduced stability conditions for the concepts, similar to those in (Sundararajan et al., 2017), and incorporated them into their training objective function to enhance the robustness of the concepts. They have also demonstrated the numerical improvement in stability.", "questions": "1- Can the authors explain the choice of architecture for the text and image encoder and its impact on the model's performance, particularly in terms of accuracy?\n\n2- Also, which image model is used in \"Standard (No interpretability)\" in table 1.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1-I really liked the paper. The authors address a very important question of model interpretability, which pertains to the robustness of explanations, in this case, regarding concepts.\n\n2-The authors not only discussed conditions for ensuring the robustness of concepts but also incorporated them into a well-defined objective function.\n\n3-I also appreciated the numerical comparison of stability in the experiments, which many papers only illustrate with a few examples.\n\n4- Finally, in my opinion, the authors conducted significant experiments to support their claims.", "weaknesses": "The quality of the explanations for some parts of the paper could be enhanced.\n\n1- It took me a significant amount of time to follow the flow of data through the network. Perhaps adding concise explanations detailing how input images traverse the network during inference would make the explanation more accessible.\n\n2- Likewise, when explaining the framework, the authors could introduce all the key terms at the beginning of the section so that readers do not need to continuously reference earlier sections to grasp the meanings of these terms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OaUJqI0frU", "reviewer": "Reviewer_UywT", "summary": "The concept bottleneck models(CBMs) and the subsequent label free CBMs offer the concept based explanation for a given prediction. However, their explanations are not faithful if the input image or the concept gets perturbed. The authors tried to fill this gap.", "questions": "See weaknesses", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Concept based explanations are a hot topic at this time and the effect on explanations based on the input perturbations is fairly unexplored. So thats a major strength of the paper.", "weaknesses": "1. The writing is poor. It is hard to follow.\n2. The related work is not complete. The following important papers should have been included in the related work:\n\n[1] Concept Embedding models. Zarlenga et al. Neurips 2022\n[2] Interpretable Neural-Symbolic Concept Reasoning. Barberio et al. ICML 2023\n[3] Entropy-based Logic Explanations of Neural Networks. Barberio et al. AAAI 2021\n[4] Addressing Leakage in Concept Bottleneck Models. Havasi et. al., Neurips 2022\n[5] Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat. Ghosh et al. ICML 2023\n[6] Distilling BlackBox to Interpretable models for Efficient Transfer Learning. Ghosh et al. MICCAI 2023\n[7] Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification Yang et al. CVPR 2023\n[8] A Framework for Learning Ante-hoc Explainable Models via Concepts Sarkar et al. CVPR 2022\n\n3.  What is the difference b/w faithful and original concept in definition 1?\n4. Label free cbm step 1, page 3. I think it will be \"they propose\" instead of \"we propose\" as label free cbm is proposed by Oikarinen et al., 2023. All the in step 1, 2, 3, 4 in sect 3 \"we\" should be replaced by \"they\"\n5. I dont understand anything from fig 2. The caption should clearly state the objective of the figure.\n6. The terms in the loss function is not at all clear (Page 6). What is D? is this somedistance? Also how will the two constraint \\delta and \\phro will be going to affect the loss and the optimization. And its connection with the Fig.2 is completely unresolved to me.\n7. Can the authors use any difficult datasets like HAM10k or MIMIC-CXR to bolster their claims. Medical images will make the claims stronger because if the explanations are not stable, it will raise trust issue among the users.\n8. I am confused with table 1. If the main message is to showcase that their method is more superior compared to the baselines, they should have the best cofiguration of their method and the baselines. Also they should have another table where they should conduct an ablation study to include different configurations of their method with various perturbations and compare the performance. Also, only PCBM and LCBM baselines are not enough. I would like to see the performances for LCBM and language in a bottle especially when the inputs and the concepts will be perturbed. Also thourough comparison is needed with basic CBM and some other methods for datasets where concepts are annotated. Also missing variances in table 1. \n9. Also, for table 2, nothing is highlighted or bolded. With so many numbers, it is diffcult for me to see which is higher and which is lower.\n\n*Post Rebuttal*\nI saw the comments by the authors and i would like to thank them. The writing is clear and the captions are good but very long. Still some related work is missing. Importantly, i feel the datasets are fairly simple but i am increasing the score to 5.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1ThYgi01fx", "reviewer": "Reviewer_6N2M", "summary": "In this paper, the author proposed a solution for constructing faithful vision-language concept models. To achieve this, the authors formalized four properties that faithful concepts should respect and designed a min-max object function by following these four properties. In addition, the authors also proposed an algorithm to solve this object function. Through extensive experiments over multiple vision-language dataset, the authors can demonstrate the effectiveness of the proposed solution in maintaining good prediction performance while achieving faithfulness.", "questions": "See above", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "+ Label-free concept bottleneck model can be unstable and unfaithful. The authors provide a timely solution for tackling this problem\n+ The authors provide a very clear formulation of the faithful concept bottleneck model and the overall techniques are sound and convincing\n+ The authors also performed extensive experiments over various vision-language datasets to demonstrate the benefit of the proposed solution\n+ The overall presentation is clear and easy to follow.", "weaknesses": "+ I have one concern for the generalizability of the proposed solution. Although the authors claimed that they proposed a solution to deal with the stability issue of the label-free concept bottleneck model, the authors only focused on the prediction task over the vision dataset. I am wondering whether the proposed solution could be generalized to other settings such as the classification tasks over NLP data or tabular data.\n+ The metrics of evaluating the model stability are also concerning to me. I feel that this metric should be $\\epsilon$-invariant, which means that ideally the perturbation $\\epsilon$ should occur in the denominator of the metric. Otherwise, with larger epsilon, you can get larger stability, which seems to be counter-intuitive. Also, by looking at the other similar metrics in literature, such as the stability metric proposed in the model interpretability literature (Alvarez Melis, David, and Tommi Jaakkola. \"Towards robust interpretability with self-explaining neural networks.\" Advances in neural information processing systems 31 (2018).), they are also calculated by putting the input perturbation in the denominator. So it would be better if the authors could justify why they didn't choose to design their stability metric in this manner and ideally reference some other papers.\n+ The presentation could be further improved. For example, for Figure 3, the text in the figure is very tiny and it is very hard to tell what concepts they are. For Figure 2, I guess the box mixed with black and white dots should represent the noise but I cannot find any explanations for this. It would be better if the authors could clearly illustrate this figure in the caption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mJUW4fbV8r", "reviewer": "Reviewer_W5Db", "summary": "This paper introduces Faithful Vision-Language Concept (FVLC) models to overcome faithfulness and instability issues in Label-free Concept Bottleneck models (LCBMs). The authors also analyze the stability and the faithfulness of concepts introducing four desiderata, namely: (i) the degree of overlap with the ground-truth concepts, (ii) the robustness against random noise, (iii) the correctness of the predictions, and (iv) the stability of the output layer.\nWith a series of experiments, it is shown that the FVLC outperforms LCBMs both in accuracy and in stability of the concepts.", "questions": "How is the interpretability of the model evaluated in vision-language models?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The authors address the important problem on interpretability in CBMs. Ensuring concepts are of good quality is central to applications of CBMs and LCBMs in real-world scenarios where perturbations are likely to happen. The presentation is clear and all parts can be easily understood, although it could be improved, see weaknesses. \nThe analysis of the possible instability effects in the concepts is sound and motivated by examples where LCBMs effectively struggle to predict the correct concepts. The proposed method for concept refinement is simple and works well in a wide range of case studies. In addition, the ablation study reveals that all terms included in FVLC are needed to improve the stability and accuracy of the model.\nEssentially, FVLC paves the use of language-based CBMs with faithfulness guarantees, which could be very helpful in applications.", "weaknesses": "One major concern about the analysis is the question of the interpretability. The claim that if a concept $\\tilde c$ matches the top-k component of c leads to an equivalently interpretable concept depends on the assumption that c is already interpretable. This is implicit in the discussion and it seems to me that it is sidelined. Since the interpretability of the model is central in this paper, somehow this has to be verified, e.g., via concept accuracy, even for LCBMs. \n\nThe authors consider the LCBM as the standard reference for improving the concepts, but the optimization could have been done even without it since the alignment loss has to be w.r.t. concepts extracted with CLIP, avoiding the noise in fitting first the LCBM and then the FVLC. \n\nI feel the contribution would also improve by comparing with other SotA models, like Post-hoc CBMs and LaBo:\\\n[1] Yuksekgonul, Mert, Maggie Wang, and James Zou, \"Post-hoc concept bottleneck models.\" ICLR 2023 \\\n[2] Yang, Yue, et al. \"Language in a bottle: Language model guided concept bottlenecks for interpretable image classification.\" CVPR 2023\n\nAbout the presentation, it is somehow counterintuitive to mention $\\Tilde c$ and $c$ while there is a clear reference to the maps $\\Tilde g$ and $g$. Moreover, I am sure that all the extracted concepts are input-dependent, so it would be the case to indicate them with $g(x)$ or $c(x)$ to explicit their dependence.\nOne possible improvement of the presentation could be to comment on the definitions after their statements, indicating intuitive properties captured by them. Another point is the introduction of $ \\tilde c$ which is clear from the text but it should be stated before that the authors refer to the faithful concepts obtained with $\\TIlde g$.\n\nOne remark: the statement in section 5 that $\\tilde c = c$ in the limit of $\\alpha_1 \\to 0$ is not always true since it depends on the loss that is considered. What loss is used? In principle, different choices of the loss function would to more stable or less stable concepts for conditions (iii) and (iv).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)