CONDA: Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts
Authors: Jihye Choi
Jayaram Raghuram
Yixuan Li
Somesh Jha
Venue: ICLR 2025 Poster (2025)
Keywords: foundation models; concept bottleneck models; distribution shifts; concept-based explanations
Primary area: interpretability and explainable AI
TL;DR: We propose a test-time adaptation method for a concept bottleneck model (CBM) based prediction pipeline for foundation models

Abstract: Advancements in foundation models (FMs) have led to a paradigm shift in machine
learning. The rich, expressive feature representations from these pre-trained, large-
scale FMs are leveraged for multiple downstream tasks, usually via lightweight
fine-tuning of a shallow fully-connected network following the representation.
However, the non-interpretable, black-box nature of this prediction pipeline can be
a challenge, especially in critical domains, such as healthcare, finance, and security.
In this paper, we explore the potential of Concept Bottleneck Models (CBMs)
for transforming complex, non-interpretable foundation models into interpretable
decision-making pipelines using high-level concept vectors. Specifically, we focus
on the test-time deployment of such an interpretable CBM pipeline “in the wild”,
where the distribution of inputs often shifts from the original training distribution.
We first identify the potential failure modes of such pipelines under different types
of distribution shifts. Then we propose an adaptive concept bottleneck framework
to address these failure modes, that dynamically adapts the concept-vector bank
and the prediction layer based solely on unlabeled data from the target domain,
without access to the source dataset. Empirical evaluations with various real-world
distribution shifts show our framework produces concept-based interpretations
better aligned with the test data and boosts post-deployment accuracy by up to
28%, aligning CBM performance with that of non-interpretable classification.

Published as a conference paper at ICLR 2025
CONDA: ADAPTIVE CONCEPT BOTTLENECK FOR FOUNDATION MODELS UNDER DISTRIBUTION SHIFTS
Jihye Choi, Jayaram Raghuram, Yixuan Li & Somesh Jha
Department of Computer Sciences, University of Wisconsin Madison {jibye, jayaramr, sharonli, jha 0cs. wisc.edu
ABSTRACT
Advancements in foundation models (FMs) have led to paradigm shift in machine learning. The rich, expressive feature representations from these pre-trained, large- scale FMs leveraged for multiple downstream tasks, usually via lightweight fine-tuning of a shallow fully-connected network following the representation. However, the non-interpretable, black-box nature this prediction pipeline can be a challenge, especially in critical domains, such as healthcare, finance, and security. In this paper, we explore the potential of Concept Bottleneck Models (CBMs) for transforming complex, non-interpretable foundation models into interpretable decision-making pipelines using high-level concept vectors. Specifically, we focus the test-time deployment of such an interpretable CBM pipeline "in the wild", where the distribution of inputs often shifts from the original training distribution. We first identify the potential failure modes of such pipelines under different types of distribution shifts. Then we propose an adaptive concept bottleneck framework address these failure modes, that dynamically adapts the concept-vector bank and the prediction layer based solely on unlabeled data from the target domain, without access the source dataset. Empirical evaluations with various real-world distribution shifts show our framework produces concept-based interpretations better aligned with the test data and boosts post-deployment accuracy by up 28%, aligning CBM performance with that of non-interpretable classification
INTRODUCTION
Foundation Models (FMs), trained on vast data, powerful feature extractors applicable across diverse distributions and downstream tasks (Bommasani et al. 2021 Rombach et al.) 2022). They can be applied to classification tasks using task-specific fine-tuning data (Kumar et al., 2022 Radford et al. 202T). Despite these strong advantages, foundation model-based systems often operate as inscrutable black baxes,, presenting a barrier to user trust and wider deployment in safety-critical settings. Another challenge faced in the standard deployment of FM-based deep classifiers their vulnerability to distribution shifts at test time caused e.g., due to environmental changes, which can cause a drop in performance (Bommasani 2021 This is particularly challenging in high-stakes domains such as healthcare (AlBadawy
via zero-shot prediction, or via linear probing
ct al.
al.
2018 Eslami et al. 2023). autonomous driving YYu et al. 2020). and finance (Wu et al. 2023a).
In this work, we address these challenges by developing an interpretable classification framework that enjoys the rich, expressive feature representations of FMs, while also having enhanced robustness towards distribution shifts at test time. To tackle interpretability. we utilize Concept Bottleneck Models (CBMs) (Koh et al.) (2220), transforming FM-based classifiers into interpretable, concept- based prediction pipelines. With the rapid advancements in FMs, there is strong opportunity to utilize them as powerful backbones, providing robust feature representations from which high-quality concepts can be extracted. Unlike early CBM approaches that required expensive concept annotations, recent advances show potential for constructing concept bottlenecks without any annotations by leveraging vision- language models (Oikarinen et al. 22033 Wu et al. (023b), and achieving performance on par with non- inteerprrreee models. Concept- predictions provide not only interpretability, but are
'hee code repository for work is available https://githuu..cmm///

---

Published as a conference paper at ICLR 2025
Input
Foundation Model
Feature
Prediction
Input
Foundation Model
"Seak"
Prodiction
"tarbor
Feature-ben based Predictionn Pipalliee
Concept-based Prediction Pipeline
Source
Source
AVS
ANG
AVG
AvC-va
AVIG
Sarget
VIG
AVG
Target
Target
WG
Low-level
Zaro Shot Linear Probing
PCBM
CONDA
Concept-loval (e.g., Waterbirds)
Zuro Bhat Linear Proting Feature based
FCBM
CONDA
(egg CIFAR-10)
Feature-based
Concept-based
Concept-based
Figure 1: Concept-based predictions are not inherently more robust to distribution shifts than feature- based predictions, necessitating dynamic adaptation after deployment. We observe significant drops in the averaged group accuracy (AVG) and worst- gorst-group accuracy (WG) from the source to the target (test) domain under two types of distribution shifts: (1) low-level shift (left), where inputs are perturbed without modifying class- semantics (e.g., Gaussian noise); and (2) concept-leve shift (right), where some high-level semantics change. On the left, predictions made through high-level concepts (e.g. by PCBM (Yuksekgonul et al.) (023) here) are not necessarily more robust to low-level input perturbations. On the right, the performance of concept-basee predictions suffers high-level features, and falling behind direct fcature- predictions (here zero-shot and linear- probbrg based classification). However, with CONDA, we can boost the performance the deployed concept- -bsed predictor to be on par with, or even better than, its non non-inton -on-interpretable counterparts.
more drastic drop, failing leverage the expressiveness of the foundation model's
also beneficial for robustness; central premise of CBMs is that as complex feature embeddings go through the concept bottleneck, the resulting predictions should, in theory, become more invariant to inconsequential input changes (Kim et al. 2018 Adeeayy et al. 2020).
However, we observe that CBMs directly deployed under distribution shifts often do not produce more robust predictions compared to FM-based classifiers (either zero-shot or fine-tuned configurations). For instance, illustrated in Figure I even when concept- based prediction pipeline matches or outperforms a Seature-based prediction pipeline in the training (source) domain, its test-time drop significantly under distribution shifts. This highlights that
(deployment) performance naive adoption of CBMs is insufficient for fully leveraging the robustness and expressiveness of FM features under test-time shifts, necessitating dynamic approach for adapting concept-based predictions in real-world deployments.
The problem of test-time (or source-free domain) adaptation (TTA) has recently been explored extensively (Wang et al.] 2021 Jung et al.) 2023 Liang et al.) 2023). The goal of TTA is to adapt deep classifier, trained on source domain data, test-time deployment setting where there could be distribution shifts (e.g., corruptions, environment changes), and given access to only unlabeled test data and the source domain classifier. While the main focus of TTA methods non-interpretable. deep classifier networks, we present the first approach (to our knowledge) for 77A of concept bottlenecks with a foundation model backbone. Our key contributions are summarized as follows: given unlabeled test data, a frozen FM, and a pre-constructed concept bottleneck, we
been on
1. formally categorize the types of distribution shifts expected during deployment, identifying possible failure modes of the concept bottleneck pipeline under these shifts (Section 2
2. propose novel framework CONDA (CONcept-based Dynamic Adaptation), where each component of the framework is adapted based on the identified failure modes, without requiring access to the source dataset or any labels for the test data (Section 3)
3. empirically demonstrate the robustness and interpretability of CONDA across various FM backbones (e.g., CLIP:VYT-L/44 and concept bottleneck construction methods (e.g., post- hoc CBM), showing that CONDA improves the test-time accuracy by up to 28%, and provides concept-based interpretations better tailored towards the test inputs (Section
Related Work. Distribution shifts occur when the data distribution during deployment differs from that during training, leading to degraded model performance (Quiñonero-Camm et al. (002)). To address this issue, TTA methods adapt the model parameters using only unlaheled test data; via
2

---

Published as a conference paper at ICLR 2025
entropy minimization (Wang al. 2021 Zhang at. 2022). self self-super learning test time (Sun) et al. 2020), class- aaase feature aligmment (Jung et al. 2023), and updating batch normalization statistics using test data (Nado et al.) 2020). These methods enable models to adapt on- on-the-fly fly without requiring access to the labeled training data. In the era of foundation models, recent efforts have been made to enhance their zero-shot inference robustness under distribution shifts without modifying their internal parameters (Chuang et al. 2023 Adila et al. (024).. However, improving the robustness of the foundation model itself not the focus of our work. Instead, given any foundation model, regardless of its inherent robustness, we aim to construct an interpretable framework without sacrificing the utility, striving for performance that matches or exceeds that of the foundation model's feature-based predictions. Refer to Appendix for extended related work.
2
CONCEPT BOTTLENECK MODEL UNDER DISTRIBUTION SHIFTS
2.1 BACKGROUND; FOUNDATION MODELS WITH A CONCEPT BOTTLENECK
Consider a foundation model Φ : X' + R&d which is any pre-trained backbone model or feature extractor (Eslami et al., 2023 Jia et al. 202T Girdhar et al. 2023) that maps the input x to an intermediate feature embedding ((x)
+ (xx) is pre-trained on a large-scale, broad mixture of data for general purposes, i.e., not restricted to specific domain. For specific downstream classification task, the general practice is to either apply zero-shot prediction on Φ(x), or to train shallow label predictor gs : Rd RL, that maps (xx) to the un-normalized class predictions g. ()()))), using a supervised loss (e.g., cross-entropy).
R.
CBM (Koh et al., 2020) first projects the high-dimensional feature embedding lower m- dimensional (m < d) concept-score space (acting like a bottleneck), and follows it with a label predictor, which is simple affine or fully-connected layer that maps the concept scores into class predictions. The concept bottleneck is represented by matrix of m unit-norm concept vectors C. = [Ca1 / |||=1] 1|2 concept (e.g., "stripes", "fin". "dots"). The m concept scores are obtained via a linear projection vc. (x) C, (xx), which is followed by a fully-connected layer to obtain the CBM model as
Cem /l/came [|2] gm where each Cæi E Rd represents high-level
plobm) (x) W. vc, (x) + b, W,C, Φ(x) + b2 = (x((x)
The label predictor g. (z) is defined by the parameters W. E Lxmm b, E IR,, and C., A key advantage of the CBM is that its predictions an affine combination of the high-level concept scores, which allows for better interpretability of the model. Since the label predictor of a CBM is chosen be simple, its performance is strongly dependent the construction of the concept bank. Additional details on the preparation of concept vectors can be found in Appendix A
2.2 DISTRIBUTION SHIFTS IN THE WILD
We define the source domain D, and the target domain D.. Let H. be defined as the space of measurable concept mappings h Rd >> R from the feature representation
a concept hypothesis class,
Φ(x) to concept scores. We also define the concept set C == C2, ,Cm), where each q: R2 + IR represents high-level concept mapping (e.g., stripe pattern, grass, beach, etc.). For domain D,. j E {s, f}. we define the concept score distribution as [D. Φb h) = (h o &).P,. where (h P,,,, is the push- fosh-forward measure (Le Gall 2022) of P. under h Φ. Note that h is determined by C such that hx)((x)) [cr (Ø(()) Cm (x(x)
Let & be a classification hypothesis class, defined as set of measurable classifiers g IR" + g mapping the concept scores to prediction logits. Finally, we define the distribution of predictions as the fush-forward measure of IP con (Dj. Φ, h) under g: P Preea(D), Φ,, h, g) = g.P $ h).
Given h H. and g G. categorize the distribution shifts in the target domain, (t.) 0| T), into one of the following broad categories:
Low-level shift: This type of transformation does not change the concept score distribution across the domains. Examples include additive Gaussian noise, blurring, and pixelization, which employ
A common approach is to define ca (x)())) as the inner product of a (unit-normalized) concept vector with the feature representation (xx), which results score for concept i.
3

---

Published as a conference paper at ICLR 2025
low-level changes to the input (e.g., CIFARIO-C (Hendrycks & Dietterich 2019)):
(D. Φ h) = con (D Φ h)
(2)
Naturally, the resulting distribution of predictions based on the concept scores also remains the same across the domains, i.e., P pred (... Φ. h. g) IP prea ( Dr, Φ h, g).
Concept-level shift: This type of transformation alters the concept score distribution, but not the prediction distribution across the domains. Examples include replacing water background with
land background images (e.g., Waterbirds, Metashift (Sagawa et al. 2019 Liang & Zou 202
Poon Dr, Φ, h) * Peon D2y Φ h) Ppeea De, Φ h, = Preed Da. 2..40 Φ. h.g)
The concept set C (2,, .,..... ,Cm is complete there exists classifier g E G such that, for both low-level and concept-level shifts, the prediction distributions conditioned on the concepts identical for source and target domains:
Definition 1
PpredDD,. D,, 6, h, g) = preaa DD, Φ h.g).
(4)
This implies that there exists a mapping from concept scores to labels encompassing both the source and target domains.
2.3 FAILURE MODES OF CONCEPT BOTTLENECK FOR FOUNDATION MODELS
Based on the definitions above, we categorize the possible failure modes of the decision-making pipeline of foundation model equipped with CBM, defined by given D,, Dr, Φ, h
[c1 o Ø ... Cm Φ). and g as follows.
Non-robust concept bottleneck under low-level shift: the concept mapping h is not robust to low-level shifts, causing discrepancies in the concept-level predictions:
Poon DDs Φ h) * Foon [2.. @.
violating the requirement for a low-level shift in Eqn. [ Such discrepancies in the concept predictions can lead to degraded performance in D.. resulting from mismatched prediction
distributions, i... (D, Φ h, g) * Pre $ h,
Non-robust classifier under concept-level shift: Given that the concept score distributions differ due to concept-level shift as in Eqn. 3] the given classifier fails to produce consistent prediction distributions across the domains, violating Equ
Pprea
(Dr, (D., Φb, h, g) + Pred (D., Φ h,g)
Incomplete concept set: The concept set {c1, Cz...., Gm not complete, and there does not exist any g E g such that IP Peea( Φ h, g) = P pred (Di, , Φ, h, g). Intuitively, it fails capture all the necessary information for consistent predictions across domains, and Definition iis not achievable in the first place.
3 CONDA: CONCEPT-BASED DYNAMIC ADAPTATION
To address the failure modes of a CBM identified in the previous section, here we propose a dynamic approach for adaptation of a CBM based only on unlabeled test data. We follow the setting test-time adaptation, where the foundation model ((x) and CBM, consisting of the concept bank C, and label predictor (Wa, s. b.), trained on the source domain are given (see Eqn D. but the source (training) dataset is not available. Let D; = { Xxn ) be the unlabeled test set from the target distribution. To address the three potential failure modes in CBM pipeline identified in Section 2.3 we propose the following three-step adaptation procedure, with each step designed to tackle a specific failure mode:
Concept-Score Alignment (CSA): The goal of this step is to perform a feature alignment of the concept scores of test inputs Vx(x,) ) T such that their class-conditions distributions are close to that of the concept scores the source dataset [ By adapting the concept vectors C, this will ensure that the label predictor continues to "see" very similar class-conditional input distributions at test time, thereby maintaining accurate predictions.
We drop the subscript 's' denote that they are adaptation parameters, specific the source domain.
4

---

Published as a conference paper at ICLR 2025
Feature Representation
Concept Bettleneck
Concept Scores
Label Predictor
Source Domalnn
Prediction Logite
Foundation Model
W
Adapted: LPA
(x) ma
(x).
Adaepted: RGB
Target Domain
Foundation Model x(x)
W.b
g (xc
Yelx)
d(x)
x
we
L.
feeture dimension: classso: L
dx
main conoepts:
residuat concects: [ m))
W.b
Ye(3) ex
Lx+
Figure 2: Overview of CONDA, proposed adaptation framework. The foundation model and CBM pipeline trained source domain shown the top, while adapted CBM, consisting of main branch and residual branch, is shown at the bottom. The components that are adapted during cach stage of the proposed method (t.e, CSA, LPA, and RCB) are shown in different colors.
To further address any discrepancy or mismatch in the feature alignment CSA step (e.g., due to distribution assumptions), here we adapt the label predictor (W, b) of the CBM, with the concept vectors fixed at their updated values from the CSA step.
Linear Probing Adaptation (LPA):
3. Residual Concept Bottleneck (RCB): As discussed in Section 2.3 the concept bank from the source domain could incomplete, and new concepts may be required to bridge the distribution gap between the domains. In this step, we introduce a residual CBM with additional concept vectors and linear predictor, which are jointly optimized (with parameters fixed) to improve the test accuracy.
main CBM
Target Domain CBM. Figure 2 shows the overall architecture of CONDA. The residual con- cept bottleneck is shown as a separate branch, where we introduce additional concept vectors Č = [ [/ıll ... č /| [|2] e IRrxd The concept scores are obtained by projecting the feature representation (xx) these residual concept vectors, and the scores are passed another linear predictor W b) to obtain the on-normalized class predictions (logits) of the residual CBM: WC (x) + b. The un-normalized predictions of the target domain CBM are obtained by adding that of the main and the residual branch CBMs, giving
plubm) (x) WCp(x + b + WC + b
(WC+ WC) ((x) b b Woon Coon (xx) + boons
(5)
where W Lxr and b R. For comparison with the source domain CBM (Eqn. 1. have defined the combined parameters from the main and residual branch CBMs as Ween [W W| (xx()))) Ceon = C C| R (m=+r)xd and bas = b+ b E R., That is, adding the residual CBM equivalent to introducing r additional rows (columns) the concept (weight) matrix. For adaptation, the parameters of the main CBM {C, W, b) are initialized to their corresponding values from the source domain, while the parameters of the residual CBM (č, W, b} are initialized randomly.
E
Pseudo-labeling. Since the test samples are unlabeled, it becomes challenging to design adaptation objectives that can minimize smooth proxy of the classification error rate on the target distribution. We utilize the idea of pseudo-labeling to address this, as commonly done in the TTA and semi- supervised learning literature (Chen et al., 2022 Lee et al. 2013 Sohn et al. 2020). We leverage the fact that the feature extraction backbone Φ x) is foundation model that is pre-trained on diverse data distributions, and as a result is likely to relatively robust to distribution shifts. We take an ensemble of the commonly used pero-shot predictor (as done e.g., in Radford et al. (222 and linear probing predictor (trained on the source dataset on top of the foundation model) to get the pseudo-labell for test samples. We combine the by taking the class predicted with higher confidence across
5

---

Published as a conference paper at ICLR 2025
both predictors. We note that more advanced pseudo-labeling methods e.g., involving weak- and soong-augmenttiinns and soft nearest-neighbor voting (Chen et al. 2022 can be used to potentially
improve our method (see Appendix D for further exploration).
Following the convention in the TTA literature (Wang et al., 2021 Chen et al. 2022), we randomly
split the test data into fixed-size batches D. - U 1= Dj, and perform adaptation sequentially on each batch obtaining the adapted model's predictions on the same batch, before moving to the next one. Also, the CBM parameters (main and residual) are adapted in an online fashion (not episodically) (Wang al. 2021), > L.e., the adapted parameters learned from batch are used to initialize the next batch and so on For convenience, we define the test dataset with paired pseudo- labels as D. {xxen Vin)) ) and the corresponding pseudo-labeled test batches as Dr b E [B]. We next expand on each stage of the CBM adaptation outlined earlier, and provide a complete algorithm for the same in Algorithm D in the Appendix.
3.1 CONCEPT SCORE ALIGNMENT
From Figure 2 (top half) and Eqn. I the concept scores Vc. (x) E R" are input to the linear label predictor W.V .V by. Let (P(vc, (x,) y. y). y E y) be the class-conditional distributions of these concept scores on the source domain. At test time, if the distribution of the input changes such that x; ~ (x), then there is a corresponding change in the class-conditional distributions of concept scores {IP(v,, (x+) M/ y) ((C,,((x;) yt y), E 3/. The goal of concept-score alignment (CSA) is to adapt the source domain concept bank C, to a target comain-speciffi one C such that the class- conditional distributions after adaptation under some distributional distance (e.g., Kullback-Leebllr or Total-variation. Informally, we wish to find an adapted concept bank C₁,, starting from C,, such that
close to that of the source domain
F(C,0(x;) ye = y) ~ P(C,o((x. 1y,- y). Vy E V.
If the class priors (P(y y), Vy) do not change significantly, this can ensure that the label predictor of the main CBM continues to receive concept scores from a similar distribution as the source domain.
We model the class-conditional distributions of the concept scores in the source domain as multivariate Gaussians: IP(vc, (x.) yy. = y) = _Vcc,, (x,) My: E,,, Vy E y. Given a labeled source- domain dataset, it is strwight-forward to estimate f, and Σn using the sample mean and sample covariance of Vc, (x,) on the data subset from class (max-likelihood estimate). Although we cannot access the source domain dataset during adaptation, distribution statistics {(1y, Ey)lvey. At test time, changes to the distribution of the concept scores can be captured by a concept matrix C (to be adapted). For a test input xt, the distance of its concept scores Vc (x:) from the Gaussian distribution of class is given by the Mahalanobis metric D (x;: My Ey) (vc(x:) - m1)) My (vc(x:)) (by).
assume to have access to these
Intra-class and Inter-class Distances. Taking the pseudo-label as a proxy for the true label of x the intra-class (or classs-class) distance measures the closeness of x, to samples from its own class, while the inter-class (or between-class) distance measures the separation of x, to samples from the other classes. They are defined as follows:
D (Xe, Dt)
Dmat (x: i Mae Ege) and
(6)
1
Dimter (xe, Dt)
(Xa pe, Ee).
(7)
t=1:6:6
Motivated by class-aware feature alignment CAFA (Jung et al. 2023), we explore an adaptation loss lada that is specifically designed to achieve concept-score alignment a per-class level. This loss is based on the idea that for discriminative feature alignment, the intra-class distances should be small and the inter-class distances should be large on the test samples (Ye et al. 202 Ming et al. 2023
Dintr (xa, 91)
lada (ve(x)), Tt.) = log Dimeer (xe, Dt)
'In the episadic approach, parameters would be reset their source domain values initialize each batch.

---

Published as a conference paper at ICLR 2025
With this setup, propose the adaptation objective for CSA to minimize test batch:
(CsA(C) =
Σ
Lada (vx) De) + ABob ||C
C
D
The second term is regularization on how much the concept vectors can deviate from their source domain values in terms of the Frobenius norm.
3.2 LINEAR PROBING ADAPTATION
In this step, we focus improving the test accuracy of the label predictor of the main CBM branch (W, b), with the concept vectors C fixed at their updated values from the CSA step (the residual CBM parameters are also frozen). For this, we use the cross-entropy loss between the predictions of the target domain CBM (Eqn. 5) and the pseudo-labels of batch D In order enhance the interpretability of the label predictor, we impose sparsity and grouping effect in its weights via an Yuksekgonul et al. 2023) given by
Flastic-net penalty term (Zou Hastie 2005
L
Lspase
(W)
Σ
(a [welllll + (1 a)) [wel
(10)
L
where We E R the l-th row of W, and a = 0.99. The adaptation objective for LPA given by
(w,.
1
log og (f/chml) (xx) Axparse Leparse (W),
(11)
D
(x.
where OF (r) is the Softmax probability for class k given the logits r, and Asparse 2 0 is sparsity regularization hyper-parameter. Using this objective, the label predictor is adapted such that the CBM's predictions on test batch are consistent with their pseudo-labels.
3.3 RESIDUAL CONCEPT BOTTLENECK
We next discuss adaptation the residual branch of the CBM whose parameters are (C, W, b). The r additional concept vectors in Ć are expected to capture new concepts in the target data and compensate for the potentially incomplete coverage of the main CBM (see Section 2.3) By increasing the expressiveness of the concept subspace, we expect to improve the accuracy on the target dataset beyond the CSA and LPA steps. Therefore, first have cross-entropy loss term this adaptation objective (as in Eqn. T.. We also introduce cosine similarity based regularization in the objective to encourage the new concept vectors in C to be less redundant with each other, and to have less overlap with the existing concept vectors C (obtained from the CSA step).
Tsim C)
cos(c; Cj) r (r
2
Σ (i.))))) J>3
COs COs C.).
(12)
m.
[m) jE(r]
Finally, we include a coherency regularization term the objective (modified from Yeh al. (2020) to improve the interpretability of the learned residual concepts, given by
1 X rk
(c,, (xx)))
Looh (C)
(13)
C
where Té, is the subset of the current target batch D/ that has the k-largest concept scores for residual concept vector è; (i.e., the top-k nearest neighbors of c; among the feature representations from D?)).
The objective to be minimized for adapting the residual concept bottleneck (with the parameters of the main CBM branch frozen) is given by:
LACD Č, W,b)
Σ
log o. ((obb)(x+))) + Assim Leam C Xash Leoh (C). (14)
DDll

---

Published as a conference paper at ICLR 2025
The constants Asim 2 0 and Acoh 2 0 are hyper-parametert that control the strength of the regular- ization terms. Note that for the residual CBM, we jointly adapt C and W, b, because we have common objective of increasing the test accuracy, whereas for the main CBM, adaptation is done in two stages (CSA and LPA), with CSA focusing on distribution alignment of the concept scores based on the intra-class and inter-class distances. Additional details on our method are given in Appendix This includes 1) complexity analysis and evaluation of running times of CONDA; and 2) automatic annotation (captioning) of the adapted and residual concept vectors for interpretability analysis.
EXPERIMENTS
In this section, we conduct experiments answer the following three research questions:
RQI: How effective is CONDA in improving the test-time performance of deployed classification pipelines that use a foundation models with concept bottleneck predictor?
RQ2: How does each component of CONDA specifically address and remedy the failures caused by different types of distribution shifts?
RQ3: How do the concept-based explanations change before and after test-time adaptation?
4.1
SETUP
Datasets. We evaluate the performance of concept bottlenecks for FMs and the proposed adaptation on five real-world datasets with distribution shifts, following the setup Lee et al. (2023): (1) CIFARIO to CIFARIO-C and CIFARI00 to LIFAR100-C for low-level shift, (2) Waterbirds and Metashift for concept-level shift, and (3) Camelyon17 for natural shift.
Backbone Foundation Models. For the CIFAR datasets, we use CLIP: (FARE?) (Shhlar mann et 2024), which adversarially fine-tuned to be more robust (adversarial) low-level perturbations than standard CLIP variants. We employ CLIP:ViT-L/14 (Radford et al. 2021 for Waterbirds and Metashift. For Camelyon17, we utilize BioMedCLIP ÇZhang et al. (023), which is pre-trained on diverse medical domains to understand medical images and text jointly, making suitable for zero-shot tasks in the medical domain.
Preparing the Concept Bottleneck. We evaluate CONDA using three popular approaches for con- structing the concept bottleneck: (1) using a ceneral-purposs concept bank where natural language concept descriptions and modern vision-language models (e.g., Stable Diffusion (Rombach et al. 2022)) are leveraged to automatically generate concept examples for finding concept vectors (Yuk) sekgonul al. 2023 Wu et al. (023b);; (2) unsupervised learned concepts where concept vectors are learned via optimization to maximize the concept-based prediction accuracy (Yeh et al.] 2020); and (3) employing GPT-3 with appropriate filtering to discover a tailored set of concepts for the bottleneck (Oikarinen et al. 2023). More details can be found in Appendix [ and Appendix C.2
Metrics. We report the performance in terms of two metrics: averaged group accuracy (AVG) and worst- worst-group group accuracy (WG). AVG is the average (per-class) accuracy across the classes, and WG is the minimum (per-class) accuracy across the classes.
4.2 RQI: EFFECTIVENESS OF CONDA UNDER REAL-WORLD DISTRIBUTION SHIFTS
Table D presents our main results evaluating the effectiveness of CONDA on different real-world distribution shifts, when combined with different CBM baselines. First of all, we observe that leveraging the expressive power of the FM feature representations can enhance the performance of CBMs. For example, using the method from Oikarinen et al. (2023), their reported accuracies CIFARIO and CIFAR 100 are 86.40% and 65. respectively when using the CLIP-RN50 backbone. In experiments, by employing the adversarially fine-tuned CLIP- -.T.P--T-L/14, accuracies of 95.24% and 68.36% respectively (source domain). This demonstrates the potential for improved utility in concept-based interpretable pipelines foundation models continue improve.
achieve higher
However, this improved performance in the source domain often does not translate to robustness after meployment. Under low-level shifts, the performance of CBMs may be comparable to that of
8

---

Published as a conference paper at ICLR 2025
Yoksekgomal et all
2023
Yeh e5
2020
Olkarinen et al
2025
Thsasset
25
T.P
Uiadaped
CONDA
Unadigted
w/ CONDA
Lnocapied
CONDA
91.18
93.26
91.55
0,05
96.26 0.11
95.24
WG
71.1
88.23 + 0.08
85.64
0.55
90.89 0.97
90.11
0.26
CIRAIIO
1.65
84.56
Target
WG
55.04 2.05
71.37
3.83
68.62 + 2.93
72.09 2.49
78.78 2.43
76.09 1.66
69.03 2.47
72.88 2.01
62.73
$3.87
COF
WG
5.12
4.28 - 0.51
9.5 4: 1,14
51.0 4: 1,40
12.09 1,25
Target
WG
$1.90
1,73 4.04
55.30
2.47 = 0.49
2,80 A: 0.71
0.13
2.56 à. 0.27
72.33
30.60 4: 1.42
70.82
28.44
0.95
0.20
6.32 = 0.38
5.01 1 0.22
Source
82.61
97.43
0.16
58.80
98.80
Waterbirds
WG
67.45
05.08
0,11
96.31 + 0.38
98.21 0.08
97.03 0,26
Targer
AVG
61.06
54,10 0.55
32.03 0.58
::
45.03 1...0.333 0.34
61.11
0.09 41.86
46.18
62.71
WG
42.52
44.70 0.70
27,80 1.24
4301 0.46
38,74 1 0.68
0.25
1.52 35.29
44.01 0.60
Source
AVG
95.72
97.27 0.28
97.94
0.10
97.18 0:00
98.02
Metashif
WG
93:44
96.62 0.39
96,94 0.30
96.0 1.
97.25
Target
AVG
94.05
80.39 0.42
84.45 = 1.39
93.09 =: 0.20
90.33 4: 0.09
93.81 0.13
83.72 2.21
93.90 0.13
65.33
Source
AVG
77.71
92.14
0.01
89:07 =: 0.60
97.01 a: 0108
94.19 = 0.11
Camelyon17
69.75
1.39
0.34
AVG
84.55
93.69 0.01
89.71 =: 0.65
91.20 + 0.06
95.01 0:007
92.54 0.16
91.75 0.08
93.16 0.05
Target
93.07
Table I: Performance of CONDA on different distribution shifts when combined with different CBMs. Zero-shot (ZS) and Linear probing (LP) the non-interpretable FM baselines. Low-level shifts covered by the CIFAR datasets, concept-level shifts by Waterbirds and Metashift, and natural shifts by the Camelyon 1 benchmark. CONDA significantly improves the AVG and WG accuracy on the target domain in many scenarios.
30
17.5
25 20-
Re-ative (S8)
12.3
Sclpstation
WC
Adaptation
7.3
2.5
CIRAR1O-C
CIFARIOO.C
Waterbirds
Metashift
Camelyon17
CIRRR1OCC CIAA1100-C
Waterbinds
Motzshitt
Figure 3: Effectiveness of individual components of CONDA for the CBM method of Yuksekgonul et al. (023). We report the relative AVG and WG, which is the (acc. after adaptation) (acc. before adaptation).
the non-interpretable counterparts (ZS and LP), but they are not inherently more robust to low-level shifts. The performance drop is particularly severe under concept-level shifts when the CBM is not adapted. But with adaptation using CONDA, the test-time accuracy under different distribution shifts increases significantly most cases. The performance is on par with or even surpasses that of the non-interpretable methods, notably in terms of the WG accuracy.
4.3
RQ2: EFFECTIVENESS OF INDIVIDUAL. COMPONENTS OF CONDA
We next analyze the individual contributions of the components in CONDA, viz. CSA, LPA, and RCB. FigureBjillusraaes the relative AVG and WG (%) when adapting the CBM of Yeh al. (020)). Under low-level shifts, CSA plays crucial role in performance improvement by encouraging the high-level concept scores to remain similar. Interestingly, using CSA alone even surpasses the performance achieved when all components are combined. This trend is also observed with the Camelyon17 dataset, which resembles a low-level shift due lighting differences across hospitals. On the other hand, under concept-level shifts, IPA and RCB become the key components of adaptation. These components allow the model to adjust concept reliance to the target domain and address the incompleteness of the deployed concept set, tailoring to the target data. In this context, CSA has minimal impact, while using only LPA leads to performance gains comparable to, or even exceeding that achieved when all components are included.
This phenomenon aligns with the findings of Lee et al. 2023) that fine-tuning only a subset of layers can be more effective than min-ttnnng all layers, depending on the type of distribution shift. In our case, the concept-based based prediction pipeline can be considered special instance their framework with two-layer classifier. The concept bottleneck layer corresponds to the first layer, which is

---

Published as a conference paper at ICLR 2025
lorsi
Sone
buah
ricress
banded
larttd
faathess
banded
wings
boach
boak
wnwted
vatarted
lkko
burpy
teach
bridge
(a) Before adaptation (source)
(b) After adaptation (target)
Figure 4: CONDA adapts the concept weights to be tailored to the target data. We visualize the positive weights in the linear probing layer (width of each mapping) hefore vs. after applying CONDA to the PCBM baseline YYkkeekgonul al. 2023) on the Waterbirds dataset.
particularly important for addressing input-level shifts (following their definition), while the linear probing layer corresponds to the second layer, which is more important for handling output-level shifts (see Section of their paper). These empirical observations confirm our design motivation for CONDA, i.e., different components play key roles in adapting to different types of distribution shifts.
Additional results, including ablation experiments to understand the effect of syper-parameters. improved pscudo-labeling methods, and the choice of backbone foundation model are given in They provide additional support for the strong empirical performance of CONDA.
Appendix D
4.4
RQ3: INTERPRETABILITY OF CONDA
We investigate how the concept-based explanations change through adaptation by CONDA on the Waterbirds dataset. In Figure 4a| we present the top five most prominent concepts contributing to the predictions for each class. As expected, in the source domain, land-related concepts are most important for predicting "landbird"", and do not positively contribute to "waterbird"; and vice versa for water-related concepts. After adapting to the target domain (test dataset), we observe adjustments in the concept-to-class mappings. Notably, land-related concepts begin positively contribute to the prediction of "waterbird". This shift indicates that CONDA successfully adapts the concept- explanations to reflect the new correlations observed in the target domain. Moreover, in the original concept bottleneck constructed following Wu et al. (202b)), there were no bird-related concepts that could help make robust predictions independent of spurious background correlations. By employing RCB with five residual concepts, we identified that three of them correspond to bird-related concepts: feathers, wings, and beak This demonstrates that CONDA adapts in manner aligned with human intuition, just like human intervening in a CBM to correct its predictions would. More importantly, RCB captures concepts that may have been missed during the initial construction of the concept bottleneck, enhancing both the interpretability and robustness. Additional results and analysis of the interpretability of CONDA can be found in Appendix E
CONCLUSIONS AND FUTURE WORK
In this work, we made a first effort exploring the test-time (post-deployment) performance of CBMs combined with foundation models. We formalized potential failure modes under low-level and concept-level distribution shifts and proposed a novel test-time adaptation framework. Each component of our framework is designed to address specific failure modes, effectively improving the test-time performance of a deployed CBM. Limitations and future work are discussed in Appendix F
TTo interpret the residual concepts, we use automated concept annotations; see details in Appendix B.2
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
Jihye Choi, Jayaram Raghuram and Somesh Jha are partially supported by DARPA under agreement number 885000, NSF CCF-FMITF-18869988 and ONR N00014-21-1-2492. Yixuan Li supported by the AFOSR Young Investigator Program under award number FA9550-23-1-01844 NSF IIS IS32233737, NSF IIS-2331669. and Office of Naval Research under grant number N00014-23-1-2643.
REFERENCES
Abubakar Abid, Mert Yuksckgonul, and James Zou. Mcaningfully debugging model mistakes using conceptual counterfactual explanations. In International Conference on Machine Learning, pp. 66-88, PMLR, 2022.
Julius Adebayo, Michael Muelly, Ilaria Liccardi, and Been Kim. Dehugging tests for model explanations. In Proceedings of the 34th International Conference Neural Information Processing Systems, pp. 700 700-712, 2020.
Dyah Adila, Changho Shin, Linrong Cai, and Frederie Sala. Zero-shot robustification of zero-shot models. In The Twelfth International Conference on Learning Representations, 2024.
Ehab A AlBadawy, Ashirbani Saha, and Maciej A Mazurowski. Deep leamingg for segmentation of brain tumors:
Impact of cross institunstttl training and testing. Medical physics, 45(3):1150 111011111-1188 2018
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability deep visual representations. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 6541-6549, 2017.
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bemsseein Jeannette Bohg, Antoine Bossclut, Emma Brunskill, et al. On the opportunities and risks of foundation models. CoRR, fts/2108.00258, 2021.
Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 295-305, IEBE, 2022. doi: 10.1109/ CVPR52688.2022.00002
Jihyc Choi, Jayaram Raghuram, Ryan Feng, Jiefeng Chen, Somesh Jha, and Atul Prakash. Concept- explanations for ouitof-distribution detectors, In International Conference on Machine Learning, pp. 5817- 5837. PMLR, 2023.
Ching-Yee Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, and Stefanie Jegelka. Debiasing vision- language models via biased prompts. arXiv preprint axXiv:2302.000070, 2023.
Sedigheh Eslami, Christoph Meinel, and Gerard De Melo. PubMedCLIP: How much does CLIP benefit visual question answering in the medical domain? In Findings the Association for Computational Linguistics: EACL 2023, pp. 1151-1163, 2023.
Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In CVPR, 2023.
Marton Havasi, Sonali Parbhoo, and Finale Velei.Velez. Addressing leakage in concept bottleneck models. Advances Neural Information Processing Systems, 22:23386-23397. 2022.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. Proceedings of the International Conference on Learning Representations, 2019.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, and Tom Duerig, Scaling up visual and vision-language representation learning with noisy text supervision. In International conference machine learning, pp. 4914-4916. PMLR, 2021.
Sung, Zhen Li,
Sanghun Jung. Jungsoo Lee, Nanhee Kim, Amirreza Shaban, Byron Boots, and Jaegul Choo. CAFA: Class-aware feature alignment for test-time adaptation. In Proceedings of IEEE/CVE International Conference on Computer Vision pp. 19060-19071, 2023.
Been Kim, Martin Wattenberg, Justin Gilmer. Carrie Cai, James Wexler, Feraanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In International conference machine learning, pp. 2668-2677. PMLR, 2018.
11

---

Published as a conference paper at ICLR 2025
Pang Wei Koh, Thao Nguyen, Yew Siang Tang. Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. PMLR, 13 18 Jul 2020.
Ananya Kumar, Aditi Raghunathan. Robbie Jones, Tengyu Ma, and Percy Liang. Fine-tuning can distort pretrained features and underperform dustif-distributiin.. In International Conference on Learning Represen- tations, 2022.
Jean-Prançois Le Gall. Measure theory, Probability and Stochastic Processes. Springer, 2022.
Dong-Hyun Lee et al. Pseudo-lahel: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICMI,, volume 3, pp. 896. Atlanta, 2013.
Yoonho Lec, Annie S. Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn. Surgical fine-tuning improves adaptation to distribution shifts. In The Eleventh International Conference on Learning Representations, ICLR. OpenReview.nee 2023.
Jian Liang, He, Tieniu Tan. comprehensive survey on test-time adaptation under distribution shifts. CoRR, ahs/2303. 15361, 2023, doi: 10:48550ARRXI..233..522
Weixin Liang and James Zou. Metashift: A dataset of datasets for evaluating contextual distribution shifts and training confficts. In International Conference on Learning Representations, 2021.
Yifei Ming, Yiyou Sun, Ousmane Dia, and Yixuan Li. How to exploit hyperspherical embeddings for out- of-distribution detection? In The Eleventh International Conference Learning Representations (ICLR). OpenReview.net, 2023.
Mazda Moayeri, Keivan Rezaei. Maziar Sanjabi. Soheil Feizi. Text-to-concept (and back) via cross-model aligument. In International Conference on Machine Learning, pp. 25037-25060. PMLR, 2023.
Zachary Nado, Shreyas Padhy, D Sculley, Alexander D' Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006. 2020.
Tuomas Oikarinen and Tsui-Wei Weng. CLIP-Dissect: Automatic description of neuron representations in deep vision networks. In The Eleventh International Conference on Learning Representations, 2023.
Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, and Tsui- Wei Weng. Label-frec concept bottleneck models. In The Eleventh International Conference on Learning Representations, 2023.
Joaquin Quinonero- Caneela,, Masashi Sugiyama. Anton Schwaighofer, and Neil D Lawrence. Dataser shift in machine learming. Mit Press, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8788 8763. PMLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 10684 10695, 2022.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations. 2019.
Christian Schlarmann, Naman Deep Singh, Francesco Croce, and Matthias Hein. Rohust CLIP: Unsupervised ad- versarial fine-tuning of vision embeddings for robust large dision-language models. In Forty-first International Conference on Machine Learning (ICML). OpenReview.net, 2024.
Chenming Shang, Shiji Zhou, Hengyuan Zhang, Xinzhe Ni, Yujiu Yang, and Yuwang Wang. Incremental residual concept bottlencek models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10000-11000 2024.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. FixMatch: Simplifying semi- sumi-vupe learning with consistency and confidence. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS), 2020.
12

---

Published as a conference paper at ICLR 2025
Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
Yu Sun, Xiaolong Wang. Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts, International conference on machine learning. pp. 9229-9248 PMLR, 2020.
Bowen Wang. Liangzhi Li, Yuta Nakashima, Hajime Nagahara. Learning bottleneck concepts in image classification. In Proceedings of the ieeelcvf conference on computer vision and pattern recognition, pp. 10062,
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Brumo Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021.
Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medclip: Contrastive learning from unpaired medical images and text. arXiv preprint 1xii:22100111666 2022.
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Scbastian Gehrmann, Prabhanjan Kam- badur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303. 17564, 2023a.
Shirley Wu, Mert Yuksekgonul, Linjun Zhang, and James Zou. Discover and cure: Concept-aware mitigation of spurious correlation. arXiv preprint arXiv:2305.006650 2023b.
Haotian Ye, Chuanlong Xie, Tianle Cai, Ruichen Li. Zhenguo Li, and Liwei Wang. Towards theoretical framework of cuttof-distribtton generalization. Advances in Neural Information Processing Systems, 34: 23519-23531, 2021.
Chih-Kuan Ych, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar, On completeness-awee aware concept-base based explanations deep neural networks. Advances neural informa- tion processing systems, 33:20554 205655, 2020.
Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving dataset for heterogencous multitask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2636-2645, 2020.
Mert Yuksekgonul, Maggie Wang, and James 7ou. Post-hoe concept bottleneck models. The Eleventh International Conference on Learning Representations, 2023.
Marvin Zhang, Sergey Levine, and Chelsea Finn. MEMO: Test time robustness via adaptation and augmentation. Advances in neural information processing systems, 35:38629-38666 2022.
Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jasprect Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. Biomedelip: a multimodal biomedical foundation model pretrained from fifteen million scientifie image-text pairs, arXiv preprint 23Xx:23333.00113 2023.
Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(2):301-320, 2005.
13

---

Published as a conference paper at ICLR 2025
APPENDICES
Symbol
Description
Dimension of the feature representation from the foundation model Number classes size of the label
Number of concepts in the main CBM Number concepts the residual CBM Input and corresponding lahel in the source domain
Input and corresponding pseudo-label the target domain
((x)
Foundation model the backbone feature extractor Source and target domain
D, and De
h: R + Th with h
Concept mapping concept hypothesis class Classifier and classification hypothesis class Concept score distribution domain D J {s,t)
8:R" R with g g
Ponn(D= h)
(Pe h,g)
Prediction distribution for domain D) J {s,
Matrix of concept vectors adapted for the target domain of size m x Subscripts 's' and 't' refer the and target domain respectively.
Matrix of residual concept vectors adapted for the target domain of size d CBM linear predictor parameters. has size x m and length
W.
W, b
Residual CBM linear predictor parameters. W has size and b has length
cicbm (x)
CBM predictor in the source domain. See Equ.
pichmn (x)
CBM predictor in the target domain. See Eqn.
D.
Unlabeled test dataset
D. and D.
Test data batch. First one is unlabeled, while the second one includes the pseudo-labels. Class-specific mean and covariance matrix of concept scores from the source dataset Concept scores obtained from the concept vectors in C via the projection Cø(x)
My, Ey
va(x)
Dmua (x (x: 2gg)
Mahalanohis distance in the concept-score space Softmax probability for class given the logits Adaptation loss used for feature alignment. See Eqn.
(x(r))
ladal) (-)
Lesa
Adaptation objective CSA. Eqn. 9
Ligame
Elastic net penalty regularization used in LP. See Eqn. Adaptation objective LPA. See Eqn.
10
LIPA
Lam
Cosine similarity based regularization used RCB.
See Eqn.
12
Leah
Coherancy regularization used RCB. See Equ. 13
LRCB
Adaptation objective for RCB. See Eqn. 14
Pyrad
Number of gradient steps for cach of the CSA, LPA, and RCB adaptations.
Thatch
[D]]
Batch size of adaptation
Table 2: Symbols and notations used in the paper.
14

---

Published as a conference paper at ICLR 2025
Algorithm CONDA: CONCEPT- CONCEPT-BASED DYNAMIC ADAPTATION
Inputs:
Foundation model (xx). Source domain CBM: C., W., b,. Concept scores distribution statistics: {(y m,) yey. Unlabeled test dataset D,,
1: Set constants and hyper-parameters:
# batches B, # gradient steps Tigrad. residual concepts
Regularization constants: Afrob, Asparse Asim: Acoh
2: Initialize the main CBM branch using source domain parameters: C C, W W., b b,.
3: Initialize the residual CBM branch parameters C, W, b randomly.
4: Split the test dataset randomly into B fixed-size batches {Di
5: for batch 6 1,2, , do
6: Pseudo-labeling: Using the foundation model, take an ensemble of the zero-shot predictor and the linear-probing predictor to obtain pseudo-labed for the test batch. More advanced methods can used here, e.g., the soft nearest-neighbor voting of Chen et al. (2022).
7: CSA Step: Adapt C with the remaining parameters fixed at their current values.
8: for step i == 2, ggaad do
9:
Compute the intra-class and inter-class Mahalanobis distances for the pseudo-labeled test batch D (Eqns. and 7).
Compute the CSA adaptation objective LcsA(C) (Eqns. 8 and Perform gradient descent step to update C.
11:
12:
end for
13:
LPA Step: Adapt (W, b) with the remaining parameters fixed at their current values.
for step i 1, rggrad do
15:
Compute the Elastic-net regularization term Iyparse (W) (Eqn. 10
Compute the LPA adaptation objective LLPA (W, b) (Eqn. II Perform gradient descent step to update W, b. end for
17:
19:
RCB Step: Adapt (C, W, b) with the remaining parameters fixed at their current values.
20:
for step i 1,2, ... Rggrad do
21:
Compute the cosine similarity regularization term Lsim C)) (Eqn.
12
22:
Compute the coherency regularization term Leoh (C) (Eqn. 13
23:
Compute the RCB adaptation objective LRCB C, W, b) (Eqn. 14
24:
Perform gradient descent step to update C, W, b.
25:
end for
Using the adapted parameters, obtain the target domain CBM predictions r(chm) for the current batch (Eqn.5) 5).
Initialize parameters for the next batch using the adapted parameters from the current batch.
28: end for
Outputs: Predictions of the target domain CBM on the test dataset. Final adapted parameters of the target domain CBM: Cr, W,, by C₁ W. b
15

---

Published as a conference paper at ICLR 2025
A EXPANDED RELATED WORK
Concept Bottleneck Models (CBMs), introduced by Koh al. (2020), are interpretable neural networks that map input data to a set of coman-understandabbe concepts (the "bottleneck") before making predictions. This architecture enhances the interpretability by revealing which concepts influence the predictions and allows users to intervene by adjusting mis-predicted concepts.
Definition of Concept Bottleneck. Despite the above benefits, early variants (e.g., (Havasi et al. ) required extensive concept annotations during training, which can be costly and impractical. This reliance on predefined, annotated concepts limits their scalability and applicability to diverse domains and tasks. To address this, recent methods aim to construct CBMs without requiring explicit concept labels, and they can be placed into three main categories: (a) unsupervised learning- beaseing-based concept discovery, (b) general-purport concept bank agnostic to tasks, and (c) leveraging multi-modal foundation models. They are further discussed below.
(a) Unsupervised learning-based concept discovery: Ych et al. (2020) formulates the concept discovery as an optimization process with the objective of concept completeness, ensuring that the extracted concepts comprehensively represent the data while maintaining inter- pretability. This approach is further advanced in Wang et al. (002)), where they optimize task task-specific concepts via self- sule-superviiion techniques such as contrastive loss improve the quality of the learned concepts.
(b) General-purper purpose concept bank agnostic to tasks: Yuksekgonul et al. (023) and Wu et al. (2023b) utilize a predefined concept bank where each concept vector derived from the parameters of a Support Vector Machine (SVM) trained to distinguish between positive and negative instances in image embeddings obtained from backbone model. Here the dataset used to learn the SVMs does not have to be the same as the data for the given task.
(c) Leveraging multi-modal foundation models: Another approach leverages the rapid advance- ments in multi-modal foundation models like CLIP to align visual and textual representa- tions, enabling the mapping of each concept to a human-readable description (Moayeri et al. 2023). Yuksekgonul al. (2023) also suggests defining each concept vector with the text embeddings from the backbone, where the text serves as human--uderra understandable concept descriptions (refer Figure 2 in Shang et al. (2024) for descriptive illustration of the method). Oiiarinnn et al. (2023) relies on trained backbone like CLIP which maps images and textual descriptions into shared embedding space. They define each concept vector as the mapping of an image embedding to its corresponding text embedding.
In our paper, we consider the most representative method from each category of concept bottleneck constructions: Yeh et al. (2020) for (a), Yuksekgonul et al. (2023) for (b), and Oikarinen al. (2023) for (c) (refer to Appendix C.2 for further details on their implementations). By applying adaptation framework to various definitions of concept bottleneck, demonstrate that can effectively and flexibly enhance the post-deployment robustness of various CBM types under real-world distribution shift scenarios.
Concept-based explanations and distribution shifts. There has been growing interest in the utility of concept-based explanations under distribution shifts. The initial work by (Kim al. 2018) hinted at the potential of high-level concepts as diagnostic units against low-level perturbations, such as adversarial examples. Following this, Adebayo al. (2020) suggested that concept-based explanations could be more robust tools for debugging and anal yzing model behaviors under spurious correlations. More recently, Abid et al. (2022) and Wu et al. (2023b) have studied the utility of concept-based explanations in the context of data drift. However, these works rely on a predefined concept bank that remains static after model deployment. Our work emphasizes the need for a dynamic approach to concept bottlenecks for the optimal utility of concept-based predictions in the deployment phase where test data can have distribution shifts. To the best of our knowledge, this is the first work to present a comprehensive view of the post-deployment performance of concept-basee prediction pipelines, and to address their test-time adaptation under distribution shifts with a dynamic concept bank.
16

---

Published as a conference paper at ICLR 2025
B ADDITIONAL METHOD DETAILS
We describe the comprehensive algorithm of CONDA in Algorithm !
B.1 COMPLEXITY ANALYSIS OF CONDA
Referring to Algorithm 1 evaluate the computational complexity of the CSA, LPA, RCB, and pseudo-labeling steps for single test batch Dj. We recall some of the terms used in our notation.
m number of concepts in the main CBM.
number of concepts in the residual CBM branch.
d: dimension of the feature representation (xx).
L: number of classes.
ggad number of gradient steps for each of the CSA, LPA, and RCB adaptations.
Thatch
|
batch size
CSA step optimizes the concept matrix of the main CBM branch C, which has m d parameters. Below we breakdown the computations involved in the CSA adaptation objective and its gradient updates. We assume that the mean and inverse-covariang matrices of the class-conditional concept score distributions are pre-computed from the source dataset.
Concept score projection for a single test sample: 2 d.
Intra-class and inter-class Mahalanobis distances for single test sample: L (2m² 3m) L Lm (2m m + 3) + L.
Frobenius norm regularization term: 3 m dl
Cost stochastic gradient update step for the batch: 2 d.
The cost of optimizing the CSA objective (Eqn. 9 for nigrad gradient update steps be expressed as:
CosicsA
Nggrad YDbatchh (2md L m (2m + 3) + L) + 5md).
(15)
LPA step optimizes the linear predictor in the main CBM branch, whose parameters are (W, b). The number of parameters optimized in this step is L. (m + 1). Below breakdown the computations involved in the LPA adaptation objective and its gradient updates.
Elastic-Net regularization term: 3 m.
Cross-entropy loss term in the LPA adaptation objective (Eqn. 11 for a single test sample: 2 m d 2 mm + L 2r d 2
3 L (m + r) (d + L) L.
Cost of stochastic gradient update step for the batch: 2 T. (y + +).
The cost of optimizing the LPA objective for Nigrad gradient update steps can be expressed as:
COstLPA Ngrad Abatch (2 (m +r) (d+ (d 5.5m 21).
(16)
RCB step optimizes the concept matrix and linear predictor in the residual CBM branch, whose parameters are (C, W, b). The number of parameters optimized in this step is r d L (r+ Below we breakdown the computations involved in the RCB adaptation objective and its gradient updates.
Cosine similarity regularization term: 6d (m + (r 1)/2).
Coherancy regularization term: r Tematch d + 74batch (Pbbte)))) ) + k))
entss-yntropy loss term in the RCB adaptation objective (Eqn. 14h for single test sample: m d
2 Lm + L 2r d + Lr L +33 L = (m + +r) (d + L) + 5..
Cost of stochastic gradient update step for the batch: 2r d + 2 (r 1).
17

---

Published as a conference paper at ICLR 2025
The cost of optimizing the RCB objective for ngrad gradient update steps be expressed as:
COSTRCB
Ngradd Teatch
(2 (m + (d+ L) + 6dr (m + (r (r-1)/2)
4 Abbuchh v d + Tibatch log((aatch)) + kr + 2rd 2 (r (r++)))
(17)
Pseudo-labeling. For the seeudo-labeling method based the ensemble of the zero-shot predictor and linear-probing predictor with the foundation model backbone, the computational complexity will be dominated by the architecture and number of parameters p in the foundation model. We denote the inference cost on test batch by Co (p. Pbatch) The computation involved in the zero-shot and linear probing predictions will be negligible compared this.
The overall computational cost for single test batch is the sum of the costs of the CSA, LPA, RCB, and pseudo-labeling steps described above. From this, we select the terms that dominate the computational cost and ignore terms that depend on smaller quantities like and k. This leads us an overall complexity
OIigrd Vihatch m + Agrrad Thanch m + Nyyrad dr (m + r)). The number of concepts is usually the order of hundreds and r is much smaller than that. The embedding dimension is on the order of few hundreds to thousands, depending on the foundation model.
To quantify the computational complexity of our method, in Table report the average (per-batch) wall-clock running times of CONDA when combined with post-hoc CBM (Yuksekgonul et al. 2023). We observe that adaptation using CONDA is quite fast, taking only few seconds per batch, with the main time-come coneemonsmmnggg component being the pseudo-labeling since it involves inference on the foundation model.
Backbone
Embedding size
Target Dataset
Dataset size
+CSA
+LPA
+RCB
CLIP-VIT-L-4 (FARE
768
CIFARIO-C
10000
4.113
0.116
0.021
0.038
4.288
CLIP 14 (FARE")
768
CIFAR 100-C
10000
16.203
0.692
0.023
0.041
16.959
CLIP.VIT-L-4
768
Waterhirds
2897
0.064
0.043
0.028
0.051
0.186
CLIP.VIT-L-14
768
Metashift
541
0.077
0.051
0.022
0.039
0.188
BiomedCLIP
512
Camelyon17
85054
0.387
0.089
0.042
0.078
0.596
Table 3: Runtime of CONDA. All inputs are reshaped to the dimension of (224, 224, 3). In column PL. we report the time it takes to obtain our proposed aseudo-labeling per batch (averaged across all incoming batches). We run 20 adaptation steps for each of CSA, LPA, and RCB with five residual concepts, and report runtime (in seconds) averaged across all batches at the test time.
B.2 AUTOMATICALLY ANNOTATING CONCEPTS
We adopt and modify CLIP-DISSECT Oikarinen & Weng 2023) for automatically annotating the concepts as follows.
Suppose S is the set of possible concept annotations. We use ConceptNet. (Speer et al. 2017) to obtain texts that are relevant to the classes. ConceptNet is an open knowledge graph, where we can find concepts that have particular relationships can find relations of the form "A Cat has [whiskers, four legs, sharp claws. ...."" Similarly, we can find "parts" of given class (e.g., "bumper", "roof" for "truck" class), or the superclass of given class (e.g., "animal", "canine" for "dog"). Following the setup in Yuksekgonul et al. (2023). we restrict ourselves to five sets of relations for each class: the hasA, isA, partof, Has roperty, and Ma deOf relations in ConceptNet. We collect all the concepts that have these relations with the classes each classification task to build the concept annotation set. However, for the Waterbirds datasct, since the classes of ("waterbird"", "landbird") } are too specific in their terminology and we cannot find relevant nodes in ConceptNet, we instead use ("bird", "water", "land") as the query set. When we have the concept annotations for the main concept bottleneck from before-deploymenn (e.g.. Yuksekgonul et al. 2023 Oikarinen et al. 2023 Wu et al. 2023b). we set as the union set of those pre-detined concepts and those identified by ConceptNet.
query text. For instance, for class "cat", one
Let D, be the target domain (test) dataset. Let &' and Φ be the image encoder and text encoder (respectively) of CLIP-ViT-B/I Recall that Φ is the backbone foundation model used in
18

---

Published as a conference paper at ICLR 2025
our framework. To determine the annotation for concept vector Ca C₁2 our goal is to assign it the most relevant caption to E S as follows:
Compute the normalized text embedding of the concepts in S using Φ let Tj be the normalized text embedding of the j-th concept in S. Also, compute the image embedding of all images in D; using ΦCLp: let I, be the image embedding of the i-th image in Dr. We then compute the inner product of the all pairs of image-text embeddings via the image-text matrix P = IT E IR [P|x||] where E [R(,:|xd and T E RRS x and d' is the dimension of the CLIP embeddings. That is, P.j the inner product of the normalized embeddings of the i-th target image and the j-th candidate annotation.
2. For all images in the target dataset, we compute and collect their concept scores as Vo.. [(((x)), Ca), (d(()).)) c.)]] E R RPl
3. The annotation for ca is determined by calculating the most similar concept label in S based on its concept scores Ve., The similarity with respect to a concept tę E is defined as
(Ve. P.e) ||Pel
sim(te, Vc. : P)
(18)
|[ve,
which the cosine similarity between the concept scores and the corresponding column of the image-text matrix p.e.- Then, the annotation for c, becomes the concept in S with the maximum similarity, given by ia where b = arg max; sim(te, Ve,; P). To reduce noise in the annotations, we only accept &s as the annotation for C2 only when s sim(ts, Ve, P) > 0.8.
To annotate the concepts in the residual concept bottleneck C, we repeat the same process.
C EXPERIMENTAL DETAILS
All the experiments are run on a server with thirty-two AMD EPYC 7313P 883 16-core processors, 528 GB of memory, and four 884 Nvidia A100 GPUs. Each GPU has 80 GB of 885 memory. For each setup, we repeated each experiment for 10 trials (using seed 40-49 for the random number generation) and report the mean and standard error.
C.I DATASETS
CIFAR1O. It consists of 60k RGB images of size 32x32 (50k images for the train set, and 10k images for the test set), equally balanced over 10 different classes (e.g., airplane, car, dog, cat, etc.). We follow the given train/test split to report the performance in the source domain.
CIFAR100. It is similar CIFARIO, but in a larger-scale; there are 100 classes, and each class has 500 32x32 RGB training images and 100 test images, making the classification more challenging.
CIFAR1O-C and CIFAR100-C. To report the accuracies, we take the average over 15 different types of corruptions with the severity level of two (out of the scale from one to five); Gaussian Noise, Shot Noise, Impulse Noise, Defocus Blur, Frosted Glass Blur, Motion Blur, Zoom Blur, Snow, Frost, Fog, Brightness, Contrast, Elastic, Pixelate, JPEG Compression. Conventionally, studies in disti--distiibution generalization literature, severity level five used, but we observe that it severely hurts the performance of the foundation model, making it impossible to be used as a decent oracle for the pseudo labeling. Hence, we chose the severity level two that still causes the performance drop due to the distribution shift, but against which, the backbone model still presents decent performance compared to the CBMs.
Waterbirds. Waterbirds dataset is for two-class classification task ("landbird" vs. "waterbird"", In the source domain, landbird (waterbird) images are always associated with the land (water) background, while in the target domain, the correlation with the background is flipped, i.e., landbird (waterbird) images are always on the water (land) background.
Metashift. Metashift has two classes of "cat" and "dog", and it simulates the disparate correlation to the backgrounds in a similar way. Source cat images are always correlated with sofa or bed the background, while dog images always correlated with bench bike in the background. For
19

---

Published as a conference paper at ICLR 2025
evaluation, randomly split 90:10 equally across the correlation types, i.e., 10% of dog images with sofa, 10% of dog images with bed, 10% of cat images with bench, and 10% of cat images with bike. In the target domain, both cat and dog images always the shelf background.
Camelyon17. This dataset is collection of histopathology whole-slide images used for the detection of metastases in lymph nodes; classifying the given slide into benign tissue vs cancerous tissue. It includes images from five medical centers, each with different staining protocols, equipment, and imaging settings. These differences simulate natural real-world distribution shifts. We use the train set (hospital 1-3) for source, and the test set (hospital 5) for the target.
basic text template: "A photo of {class_name)" for CIFARIO. CIFARI00, Waterbirds, and Metashift datasets. For the Camelyon17 dataset, we use the ensemble of prompts: for class benign, {"A histopathelogy image no normal lymph node tissue stained with hematoxylin and eosin.",
For zero-shot prediction, we
"An H&E stained slide showing healthy 1ymph node without cancer cells.", ", "Mi "Microocopic image of cancceuus lymph node tissue.". "A pathology image of benign lymph node with normal histology, "Hematoxylic and eosin stained section of normal lymph node.'M for class malicious, {"A histopathooogy image of 1ymph node with metastatic
breast cancer stained with hematoxylin and eosin.", "An H&E stained slide showing lymph node tissue infiltrated by cancer cells.", "Microscopic image of 1ymph node containing metastatic carcinoma.", ", "A pathology image of malignant lymph node with cancer metastasis." ", "Hematoxylin and eosin stained section of lymph node with breast cancer metastases.")
C.2 PREPARING THE CONCEPT BOTTLENECK
There are various ways of defining the concept vectors {c in the concept prediction layer Vc, (x). Early works on CBM required the training dataset to have concept annotations from domain experts in addition to the class labels for training the concept predictor (Koh et al. (2220). Subsequent works have also explored learning the concept vectors in an unsupervised manner (without any concept annotations) (Yeh et al., 2020 Choi et al. 2023). More recently, natural language concept descriptions and modern vision- -angun- models (e.g.. Stable Diffusion (Rombach et al. 2022) are being leveraged to automatically generate concept examples (Yuksekgonul et 2203 Wu et al.) 2023bb for finding the Concept Activation Vectors (CAVs) (Kim et al. (2018) (each CAV corresponds to Csi),
directly guide the construction of concept bank C, (Oikarinen al., 2023). We highlight that in all prior works (to our knowledge) the concept bank remains static, i.e., once the of concept vectors defined and the CBM is deployed, its predictions are made based on these predefined concepts, regardless of any distribution shift at test time.
Yuksekgonul et al. (2023). For CIFARIO and CIFARIO0, we use the BRODEN visual concepts datasets Bau et al. (2017) to learn concept activation vectors, which are used to initialize the weights and bias parameters the concept bottleneck layer, as described Yuksekgonul al. (2023). For Waterbirds and Metashift, we use the images belonging to the concept categories as follows; nature, color, and textures for Waterbirds, and nature, color, texture, city, household, and others for Metashift. For Camelyon17, we color and textures categories, following the setting in Wu et al. (023b).
Yeh et al. (022)). For fair comparison, we set the number of the concepts to be the same as the size of concept bottleneck by Yuksekgonul et al. (2023) except with Metashift where we use 100 concepts instead, since with over 100 concepts, we found there are much unnecessary redundancy between them.
Oikarinen et al. (2023). Following their instructions, we create the initial concept set using GPT-3, tollowed by concept filtering. For the sparsity of the linear probing layer, we set À 0.001 and a = 0.5.
Table shows summary of the major hyper-pan parammerrss used in our experiments. As for the hyper- hyper-paramedt parameter & in Equation [3] we set l equal to batch size / (2 x number of classes), which is a heuristic that works well in practice.
20

---

Published as a conference paper at ICLR 2025
Dataset
Backbone
Barch Size
# Epochs
Ir (CSA, LPA, RCB)
Adaptation stops
(Aueb. Aquns Avin Ave)
CIFARIO
CLIP:VIT-L-4 (FARE")
128
Adam, 0.01
CIFAR100
CLIPNITT 14 (FARE")
512
Adam, 0.01
1.0,,.0,..1,2...
Waterbirds
CLIP-VIT-L-H
SGD,
Metashift
CLIP-VII-L-1
32
20
SGD, 0.1
15.0, 100001]
Camelyon17
BionsdCLIP
SGD, 0.01
10.5, 1.0)
Table Summary of the hyper-parameters used our experiments.
Eeher Semported
(Beime Atmaalee
C10T1 2W0
NGreed
(RO)
(a) Atroh in CSA. CIFARIO-C
(b) Atco in CSA. Waterbirds
(c) Axpase in LPA
OHASSOC 4M0
2000
(007
(d) in RCB
(e) Asim in RCB
(f) Aeah in RCB
in CONDA. We ablate on the individual hyper- parameters in CONDA for each type of distribution shift: (1) CIFARIO-C (impulse noise) simulating low-level shift, and (2) Waterbirds simulating concept-level shift.
Figure 5: Ablations the
D ABLATION EXPERIMENTS
Ablation Study Hyperparameters.
In Figure 5, we present comprehensive ablation study illustrating how different hyperparameter choices affect the performance of our proposed method.
Most notably, in Figures 5a and 5b we observe that Xinh influences the adaptation performance differently depending the type of distribution shift (i.e., CIFARIO-C for low-level shifts and Waterbirds for concept-level shifts). Recall that Anob controls how much the concept vectors are allowed deviate from their original construction during adaptation. When Atob is very low (e.g., 0.001), the weights in the concept bottleneck layer deviate excessively, leading to instability.
In the case of low-level shifts, as shown in Figure 5a over-regularizing the Frobenius norm term (e.g., setting Anu as high 10) prevents the method from addressing the non-robust of the concept bottleneck under such shifts (i.e., the first failure mode in Section 2.3 Selecting a suitable moderate value such as Arab 0.1 leads to optimal performance.
contrast, under concept-level shifts depicted in Figure 5b allowing deviation of the concept vectors harms performance. By strongly regularizing with high Afrob value (e.g., Atrb 10), preserve the original pre-adaptation performance (note that WG drops to almost zero when Afreb < 1). This occurs because failures of CBMs under concept shifts need to be addressed by adapting the linear probing layer rather than the concept bottleneck layer (the second failure mode in Section 2.3
nearly
However, when all components of CONDA activated, the adaptation performance becomes quite insensitive to the choice of X regardless of the type of distribution shift, since all the components collaboratively combine handle all possible failure modes.
21

---

Published as a conference paper at ICLR 2025
Regarding the hyperparameters for the other regularization terms namely, Aspasee Asim and Aeoh (Figures 5c|| 5e and we find that the performance is relatively insensitive to their values unless they are set too high, which could override the main optimization objectives.
As for the number of residual concepts r, shown in Figure |d we observe that increasing r helps improve the performance up to certain point (specifically, r 5), after which the performance saturates and additional concepts become redundant. We recommend that a criterion like this be used select r in practice. Choose a high cosine similarity threshold (e.g.. 0.9), and stop adding new residual concepts once a new concept vector starts to have maximum cosine similarity (with the existing set of residual concept vectors) larger than the set threshold.
Pseudo-labeling variants. We acknowledge that the performance of CONDA relies on the quality of pseudo-labels. In Section B we introduced simple yet effective approach that ensembles the foundation model's zero-shot predictions and linear probing predictions, with focus on matching the CBMs' post-deployment performance with that of the feature-based predictions. However, more advanced pseudo-labeling techniques could further improve our method's is performance.
Importantly, the pseudo-labeling technique should operate on batch basis to run alongside CONDA, which performs adaptation with an incoming batch of test data in an online fashion. For this, we employ a recent method from the TTA literature Chen et al. (2022). They employ an online pscudo-labeling refinement scheme that generates significantly more accurate pscudo-labels by using soft k-nearest neighbors voting in the target domain's s feature space for each target sample. The neighboring samples are generated by applying weak augmentation to each incoming target sample. The core intuition of their method that the model should make consistent predictions for these nearest neighbors. We apply this method to the feature-based finear-probing predictions ("Refined LP") and ensemble it with ZS predictions.
Table 5 compares the performance of CONDA with Post-hoc CBM (Yuksekgonul al. 2023 when using i) our simple sscudo-labeling approach, ii) pscudo-label refinement by Chen et al. 2022). and iii) perfect pseudo labeling (using the ground-truie labels of the target dataset to provide an empirical upper bound on the performance). We observe that the refined pseudo-labeling of Chen al. (022) helps further improve the adaptation performance of CONDA. is particularly effective with low-level shifts (CIFAR10-C and CIFAR CIFAR100-C), as the method by Chen et al. (2022) enforces consistent predictions among weakly-augmented instances, which correspond to low-level shifts (e.g., cropping, color jittering, flipping, etc.). However, compared to the performance with perfect pseudo-labeling, there remains a significant performance gap (cspecially CIFAR-100) 100). Reducing this gap with more advanced pscudo-labeling that can handle both distribution shift types is important direction for future work.
Dataset
Metric
LP)
(ZS. Refined
Perfect PL
AVG
84,38 + 1.52
90.06 + 1.94
96.37 0.37
CIFARLO-C
WG
72.69 2.49
76.31 3.01
92.65 0.56
AVG
53.88 : 0.23
61.25 1 0.29
97.31 0.35
CIFARI00-C
WG
2.56 + 0.27
10.28 +: 0.27
79.13 + 1.73
AVG
60.69 + 0.23
62.77 + 0.16
95.39 + 0.21
Waterbirds
WG
43.01
0.46
44.30 + 0.11
92.02 0.42
AVG
93.69
0.20
94.07 0.11
100.0
Metashift
WG
92.02 : 0.12
93.56 0.13
100.0
AVG
91.20 1 0.06
93.19 1: 0.10
94.82 : 0.08
Camelyon17
WG
88.96 0.16
90.88 + 0.15
93.50 + 0.17
Table 5: Performance of CONDA with different pseudo-labeling techniques. Here, ZS and LP refer to zero-shot and linear probing methods used for prediction based on the foundation model. Refined IP refers to the pseudo-labeling method Chen et al. (2022)..
Choice of foundation model. Another factor that inherently affects the performance of CONDA is the choice of the backbone foundation model. While foundation models are usually designed for general purpoos tasks (e.g., BiomedCLIP Zhang al. 2023), pretrained on diverse medical domains), they are sometimes fine-tuned for specific domains (e.g., MedCLIP (Wang et al. 2022). specifically pretrained chest X-rays).
22

---

Published as a conference paper at ICLR 2025
In Table we compare the performance of our proposed adaptation with different CBM baselines on the Camelyon17 dataset, while varying the backbone foundation model. Intuitively, MedCLIP may not be well-suited for pathology data such as Camelyon17, and observe significant drops in zero-shot (7S) and linear probing (I.P) accuracies in both source and target domains. Consequently, the performance of the CBM based on its emheddings is much worse when mis-matched foundation model is used. On the other hand, with BioMedCLII as the foundation model, the source domain performance well as the adaptation performance of CONDA on the target domain are much better. This confirms that selecting an appropriate backbone leads to better representative embeddings and higher-quality pseudo labels, which in-turn leads more accurate test-time adaptation.
This suggests another avenue for further improving the adaptation performance beyond advanced pseudo-labeling techniques for example, zero-shot robustification of the foundation model embed- dings (Adila et al. 2024). Such approaches could be employed when the chosen foundation model is not specifically tailored to the given task. We leave this as an important direction for future work.
Backbone
Z8
LP
Yaksekgonul at 2023
Yeb et al
2020
Oikarinen et
2023
Unudapted
w/CONDA
Unosupted
CONNA
Unsdapted
CONDA
Source
AVG
77.71
92.14
= 0.01
89.07 = 0.60
97.01 = 0.05
94.19 : 0.11
BiomedCLIP
WG
69.75
88.89
0.02
84.34 1.39
96.31 0.24
91.23 + 0.12
AVG
84.55
93.69
= 0.01
89.71 = 0.85
91.20 = 0.06
95.01 0.07
92.54 = 0.16
91.75 =: 0.08
93.16
Target
WG
76.08
89.19
= 0.02
85,96 = 0.88
88.96
0.16
93.07 0.37
91.07 = 0.32
87.21 :: 0.09
89.00
Source
AVG
53,09
79.89
0.06
94.58
0.08
McdCLIP
WG
11.75
79.28
0.01
76,21 20,16
92.20 = 0.44
78.01 :: 0.15
AVG
48.87
68.37
0.07
67.35 - 0.12
67.56
0.11
88.72 0.28
86.04 - 0.19
66.29 + 0.18
67.05 0.08
Target
14.66
68.32
0.05
0.19
65.36
0.14
81.42
81.01 1.65
0.21
65.17 0.14
Table 6: Performance of CONDA varying backbone foundation model. The dataset is Camelyon17, simulating a natural shift between the source and target domains.
E ADDITIONAL INTERPRETABILITY ANALYSIS
In this section, we include additional experiments and analysis to better understand the interpretability of CONDA well as the utility of the RCB component.
E. RESIDUAL CONCEPT BOTTLENECK COMPENSATES FOR PREDICTION ERRORS
Here we aim to understand how including the RCB component in CONDA impacts the predictions of the adapted classifier. We conduct an analysis similar to the one in Appendix B of Yukkekkonuul al. 2023), where they evaluate the impact of the residual predictor PCBM-h and when it alters the predictions of the main predictor PCBM. In Figure 6 we compare the predictions made by (i) PCBM + CSA + LPA (i.e., excluding RCB) with that of (n1) PCBM CSA LPA + RCB (i.e., including RCB) on the CIFAR 10-C (with Gaussian Noise, Shot Noise, and Impulse Noise) and Metashift datasets. The x-axis shows the confidence of predictions, which are binned into 5 intervals; and y-axis shows both the accuracy of (i) within each confidence bin (blue curve), and the consistency of predictions between (i) and (ii) within each confidence bin (red curve). Consistency is defined as the fraction of samples where the predictions of two models are the same.
Figures ba and show the accuracylconsistency plots for the CIFARIO-C and Metashift datasets respectively. We observe that in both cases, when the confidence is high, the accuracy and consistency are high. As the confidence of predictions decreases, the accuracy and consistency within the confidence bins also decrease sharply. From this, we can infer that the residual component (RCB) modifies the predictions of PCBM + CSA + IPA mostly when they are incorrect and have low confidence. This is readily apparent in the case of Metashift which addresses binary classification, since all the inconsistent predictions where PCBM + CSA + LPA is incorrect have to be correct when RCB is included. Thus, we hypothesize that RCB has the effect of intervening to compensate mainly when the prior adaptation components (CSA + LPA) have prediction errors or low confidence.
We also summarize the test accuracies of the CONDA variants (i) and (ii) on Metashift and CIFAR10 1-- in Table 7 We observe that including RCB (variant ii) leads to a small increase in both the AVG and WG accuracies.
23

---

Published as a conference paper at ICLR 2025
1.0
1.0
0.9
Acouracy Cors sterry
Conssterrc
0.8
0.8
0.7
0.6
0.6
0.5
0.4
Accuracy L+
0.4
Corantency
Corandency
0.1-0.3
0.3-0.5
0.5-0.7
0.7-0.9
0.9-1.0
0.5-0.6
0.6.0.7
0.7-0.8
0.8-0.9
0.9-1.0
Confidence
Confidence
(a) CIFARIO-C
(b) Metashift
Figure 6: RCB intervenes and compensates for mis-classifications of PCBM CSA + LPA, particu- larly in the lower confidence prediction bands. For Metashift, with two classes, a mis-classification and inconsistent prediction imply that RCB corrects the prediction.
Dataset
CONDA variant
Accuracy (AVG)
Accuracy (WG)
PCBM CSA LPA
93.96
91.61
Metashift
PCBM CSA LPA RCB
94,57
93,25
PCBM CSA
81.89
64.63
CIFARIO-C
PCBM CSA LPA RCB
82.27
66.54
Table 7: Accuracy comparison of the CONDA variants where (i) RCB is excluded and (ii) RCB is included on Metashift and CIFARIO-C (with Gaussian Noise, Shot Noise, and Impulse Noise).
E.2 ACCURACY-INTERPRETAAII TRADEOFF IN RESIDUAL CONCEPT BOTTLENECK
In this sub-section, we aim to answer to the following question: while the residual concept bottleneck improves the adaptability of CONDA, does it potentially affect the interpretability by introducing additional model complexity? An analysis of the trade-off between model complexity and inter- pretability, particularly as new residual concepts are added, would be valuable for practitioners seeking interpretable yet robust models.
Here we apply our adaptation to PCBM (CLIP), where each concept vector is constructed using CLIP text embeddings of concept captions, deployed to the Waterbirds dataset. As discussed in Appendix[B22 B.22, for concept annotation, we leveraged the ConceptNet hierarchy following the setup in Yuksekgonul et al. 2023). We searched ConceptNet for the words "Bird", "Water", "Land" and obtained concepts that have the following relationship with the query concept: hasA, isA, partof, HasP PasPPoperty, and MadeOf.
We compare the two CONDA variants (i) PCBM + CSA + LPA and (ii) PCBM + CSA + LPA + RCB by varying the number of residual concepts (r) and evaluating the following metrics:
Relative accuracy of method (ii) minus (i), both for AVG and WG. Similarity score output in Eqn. 18 from the automatic concept annotation method described in Appendix B.2
The similarity score is used as quantitative metric to measure the interpretability of RCB. To be more specific, the score Eqn. tells us how aligned the assigned concept caption is with each residual concept vector. A low score implies that the assigned caption is not good description for the concept.
Figure shows the relative accuracy and similarity score as function of the number of residual concepts (as they are added incrementally). We observe that choosing r = 5 would result the best relative accuracy, but it leads drop in the similarity score which peaks at 4 (implying drop in interpretability the residual concept). A practitioner can choose a suitable stopping point for the residual concepts by monitoring these criteria as shown in the figure.
24

---

Published as a conference paper at ICLR 2025
1.00
0.75
Anmotation score
0.50
20
accurscy (%)
0.25
0.00
0.25
10
-0.50
181
-0.75
W6
Number residual concepts (1)
Figure 7: Accuracy vs. Interpretability of the Residual Concept Bottleneck. with PCBM (CLIP) deployed on the Waterbirds dataset (target domain).
pRW
chair
mouse
Cal
bodelothas
mouse
badijthhes
deek
keybowr
koyboard
bookcase
bookcasww
cabret
cheat of drowars
desk
chain, wheel
book
lake
dog
muzzle
0og
bench
house
wholn
ground
ground
house
motorblke
motorbike
(a) Before adaptation (source)
(b) After adaptation (target)
Figure 8: CONDA adapts the concept weights to be tailored to the target data. We visualize the linear probing layer weights (width of each mapping) before vs after applying CONDA to PCBM Yuksekkonull al. 2023) on the Metashift dataset. We only show the mappings with positive weights.
E.3 ADDITIONAL INTERPRETABILITY RESULTS
present another example demonstrating how CONDA adapts interpretations on the MetaShift dataset, similar to Figure 4 In the source domain, cat images are exclusively correlated with sofa or bed objects, whereas dog images are always associated with bench or bike objects. In the target domain, however, both cat and dog images appear with a shelf background.
In Figure g
Without any adaptation, the deployed CBM indicates that the most contributing concepts to the "cat" class are mainly household-related objects (see Figure (Ba), and these concepts do not positively contribute to the "dog" class at all. After applying our adaptation (Figure 8b). the influence of the bed-related concepts is diminished, while shelf-related concepts (highlighted in bright green) begin to contribute to the prediction of both the "cat" and "dog" classes.
F LIMITATIONS AND FUTURE WORK
This work was motivated by our observation that recent CBM variants atop a backbone foundation model may close the performance gap with feature-based predictions in the source domain, but they
25

---

Published as a conference paper at ICLR 2025
often unable to do under distribution shifts test time (after deployment). Hence, for an interpretable and robust decision-maken pipeline under distribution shifts, while fully leveraging the representative power of foundation models, adaptive test-time approach is required. To the best of our knowledge, we have proposed the first effort to tackle this problem setting for CBMs. We formalized potential failure modes under low-level and concept-level distribution shifts and proposed a novel test-time adaptation framework, named CONDA. Each component of CONDA is designed to address specific failure modes, effectively improving the test-time performance of a deployed CBM using only unlabeled test data.
We acknowledge that the effectiveness of our framework is limited by the inherent robustness of the backbone foundation model, especially due to its reliance on pseudo-labeling, Specifically, when the backbone foundation model remains robust (e.g., against low-level shifts with lower severity level or concept-level shifts), concept-based predictions can be adjusted to be more robust than feature-based predictions through adaptation (e.g., see Metashift results in T.. However, when the backbone foundation model is not robust (e.g., against low-level shifts with higher severity level), the CBM adaptation, which relies on the pseudo-labels of the foundation model (feature representations), cannot be guided to successful solution and could lead to reduced performance; see results in Table g Moreover, in Table [
we note that there are instances where our adaptation did not yield
Dusaci
23
I.P
adcgration
Nakeeigonnl it al 2023
CSA LBA
weo adaptation
ot 2000
LPA
CSA LPA
Sounz:
AVG
0.957
0.972
0.079 1: 0.001
0.972 2 0.001
Metaakff
WG
0.934
0.950
0,909
+: 0.003
0.960 : 0,00)
0,705
03855
0:350
0.006
0.20 UIE
0,713 0,05
0.676 0/099
0.540 0.009
0.34
0.009
0,740
0,06
0.460
0.720
0.850
0.013
0.27 0.I0
0.76
0017
0,398 0.015
0.712 0.018
0.000
0.020
0.512 0006
Table 8: Negative results of our test-time adaptation. In the target domain, the model faces Metashift images with random Gaussian noise (severity level five), following the implementation of Hendrycks & Dietterich (011)). When the performance of zero-shot and linear-probing inference is poor on the target domain, the pseudo- adaptation. Therefore, the performance of CONDA with different components the target domain is worse than that of the model without any adaptation.
cannot serve as a reliable reference for the test-time
improvements with the CBM method of Yeh et al. (2020). In cases such as the CIFAR datasets and Camelyon17, the unadapted CBM already outperforms ZS LP in the target domain, and adaptation using pseudo-labels produced by these methods can negatively impact the performance. This likely because the concept learning algorithm in Yeh et al. (2020) is designed to optimize accuracy, with the concept bottleneck layer serving as an additional layer that can be optimized along with the subsequent IP layer. However, a caveat of this approach is that the interpretability of the concept bottleneck is not guaranteed, whereas methods such as Yukkekgonul et al. (202)) and Oikarinen et al. (2023) provide clear textual annotations for concepts, enhancing the interpretability.
Despite these limitations, we believe our work is an important first step toward leveraging off-the-shelff foundation models in an interpretable decision-making process, while preserving the post-deployment utility. We highlight that our framework can continue to benefit from ongoing improvements in the robustness of foundation models and the development of more advanced lseudo-labeling techniques (as hinted in Appendix [D), both of which represent promising avenues for future work. Another promising direction for future research is to develop a deeper theoretical understanding of concept bottlenecks under distribution shifts. For instance, it would be valuable to i) characterize the sufficiency of given concept set from training (source domain) for robust test-time accuracy under different distribution shifts; and ii) to quantify or bound the extent to which test-time adaptation can bridge the accuracy gap between the source and target distributions. Such theoretical insights would complement the algorithmic and empirical advancements, guiding both the design of more effective residual concept bottleneck and the development of improved adaptation strategies.
26


Reviews:
[{"review_id": "3Dgx63JbNS", "reviewer": "Reviewer_uv21", "summary": "This paper investigates an adaptive concept bottleneck approach for converting non-interpretable foundation models into interpretable decision-making pipelines using high-level concept vectors. This method is particularly useful for handling distribution shifts, especially when there is unlabeled data from the target domain. The key ideas behind this work are inspired by Concept Bottleneck Models (CBM) and Test-Time Adaptation (TTA).\n\nThis paper provides several insightful observations: (1) a naive application of CBMs is insufficient for fully leveraging the robustness and expressiveness of foundation model (FM) features under test-time shifts, as illustrated in Figure 1; (2) the identification of failure modes in concept bottlenecks for foundation models, based on a definition of distribution shifts “in the wild.” Based on these observations, the paper proposes a three-stage approach to align the concept bottleneck, label predictor, and Residual Concept Bottleneck, allowing for the extension of additional concept vectors. This approach is combined with a new architectural design and regularization strategies to enhance test-time adaptation, coherency, and interpretability.\n\nExperimental testing on five datasets and three baselines demonstrates that the proposed approach effectively improves the test-time performance of deployed CBMs in most cases.", "questions": "The main question concerns the definition of the concept bottleneck, as outlined in the “weakness” section. I suggest that the authors provide a concrete explanation or illustrative figures to clarify this concept within the context of the paper.\n\nMy primary concern regarding the acceptance of this paper is the necessity of introducing the residual concept bottleneck, as it does not appear to offer significant benefits in modeling. Although a case study shows adjustments in the concept-to-class mappings (Figure 4), this is based on a single example. I recommend providing a more comprehensive analysis, ideally with quantified interpretation results, to better explain this phenomenon.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This work is the first to study the post-deployment performance of concept bottlenecks for foundation models. The research question is both interesting and practical.\n\n2. The experimental results are reasonable, demonstrating that the proposed method improves performance in most cases.\n\n3. The writing is well-organized, clearly outlining the scope, motivation, and insights regarding shift types and failure modes of concept bottlenecks during test-time adaptation.", "weaknesses": "1. The definition of a Concept Bottleneck should be clarified. From my reading of the paper, it seems that the concept bottleneck is a learnable mapping function that transforms hidden features from foundation models into lower-dimensional concept features. However, it’s unclear whether this bottleneck acts as a dictionary or codebook that links concepts (text) to embeddings. Additionally, a more thorough introduction to the three baselines used for constructing the concept bottleneck is needed. The current explanations are somewhat abstract and too brief, making it difficult for readers to grasp the details.\n\n2. The ablation study reveals that the influence of individual components is inconsistent. In different scenarios, different components demonstrate varying levels of importance. In some cases, individual components even outperform hybrid strategies, or all components have a negative influence in the worst cases. (Additionally, please correct the left plot in Figure 3.) Rather than focusing solely on individual contributions, it would be more informative to evaluate combinations of components. Specifically, removing components one by one to assess the impact on performance using the remaining components would provide a clearer understanding of their collective contributions.\n\n3. The poor performance on the Camelyon17 dataset is largely due to the inappropriate choice of foundation model. MedCLIP, which is pretrained on chest X-rays, is not well-suited for pathology data such as Camelyon17. To fairly demonstrate the effectiveness of the proposed method, the authors should use data from the chest X-ray domain, as the foundation model significantly impacts pseudo-labeling quality. Alternatively, models like BiomedGPT (Zhang, Kai, et al. “A generalist vision–language foundation model for diverse biomedical tasks.” Nature Medicine (2024): 1-13) or BiomedCLIP (Zhang, Sheng, et al. “BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs.” arXiv preprint arXiv:2303.00915 (2023)) could be more appropriate due to their pretraining on diverse medical domains. Additionally, the results highlight the significant influence of foundation models on performance. From my perspective, this is a crucial analysis point that should be explored in more depth in this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vNvL5gphmn", "reviewer": "Reviewer_VnZY", "summary": "This paper investigates the potential of transforming complex and non-interpretable Foundation Models (FMs) into interpretable models using Concept Bottleneck Models (CBMs). Specifically, it focuses on building robust models that maintain strong performance under distribution shifts through test-time adaptation. The authors categorize three types of failure modes where CBMs may struggle under distribution shifts—low-level shift, concept-level shift, and incomplete concept set—and propose a framework called CONDA to address each. CONDA comprises three modules: Concept-Score Alignment, Linear Probing Adaptation, and Residual Concept Bottleneck. Experimental results demonstrate that these modules effectively mitigate failure modes, thereby enhancing model robustness and interpretability in various challenging scenarios.", "questions": "see weaknesses", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "•  This paper addresses a significant gap by focusing on test-time adaptation for Concept Bottleneck Models (CBMs), a rarely explored area. By tackling the robustness of interpretable models under distribution shifts, the paper contributes valuable insights into making foundational model-based pipelines more practical and trustworthy in real-world scenarios.\n\n• The proposed CONDA framework is well-structured with three distinct modules—Concept-Score Alignment, Linear Probing Adaptation, and Residual Concept Bottleneck. Each module addresses a specific type of distribution shift failure mode, allowing for a comprehensive approach to adapting CBMs dynamically without compromising interpretability.\n\n•  The authors conduct experiments across a diverse range of datasets, including CIFAR, Waterbirds, and Camelyon17, to validate CONDA’s effectiveness under various distribution shifts. This diversity strengthens the claim that CONDA generalizes well across different domains and types of shifts, reinforcing its potential applicability to real-world challenges.\n\n•  The paper demonstrates up to a 28% improvement in test-time accuracy over standard CBM approaches, which is a substantial gain. This improvement, particularly in challenging distribution-shifted settings, highlights the framework’s robustness and establishes its effectiveness in bridging the performance gap between interpretable and non-interpretable models.", "weaknesses": "•  Lack of Depth in the Related Work Section:\nThe related work section could be expanded to enhance readability and provide a more comprehensive overview of relevant literature. It would be beneficial to include a discussion of label-free CBMs and related methods. The authors should consider incorporating additional references to enrich the context, particularly those exploring label-free CBMs.\n\n(1) Oikarinen, T., Das, S., Nguyen, L. M., & Weng, T. W. Label-free Concept Bottleneck Models. In The Eleventh International Conference on Learning Representations.\n\n(2) Wang, B., Li, L., Nakashima, Y., & Nagahara, H. (2023). Learning bottleneck concepts in image classification. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition (pp. 10962-10971).\n\n(3) Shang, C., Zhou, S., Zhang, H., Ni, X., Yang, Y., & Wang, Y. (2024). Incremental residual concept bottleneck models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11030-11040).\n\n\n•  Unclear Articulation of Contributions in the Introduction:\nThe introduction section lacks clarity regarding the specific contributions of the paper. The authors should refine this section to explicitly outline their contributions to improve the reader’s understanding of the paper’s unique impact.\n\n•  Need for More Comparative Experiments with Baseline CBMs:\nThe experimental results could be strengthened by adding comparisons to traditional CBM models. Including analyses of concept and task accuracy would also provide valuable insights into the framework's improvements over standard CBMs.\n\n•  Limited Dataset Diversity in Experiments:\nThe study would benefit from additional experiments on commonly used CBM datasets such as AwA2, CelebA, CUB (Caltech-UCSD Birds) and TravelingBirds. Expanding to these datasets, along with comparisons to other state-of-the-art CBM models, could further validate the generalizability and competitiveness of the proposed method.\n\n•  Reference Error on Line 875:\nThere is a referencing error on line 875, which should be corrected to improve the document's accuracy and professionalism.\n\n•  Reliance on Foundation Model Robustness Assumptions:\nThe pseudo labeing effectiveness assumes that the feature extraction from foundation models remains robust under distribution shifts, which might not hold in all real-world scenarios. Evaluating the performance with varied foundation models or explicitly testing robustness assumptions could help clarify these dependencies.\n\n•  Limited Analysis of Interpretability-Complexity Trade-off for Residual Concepts:\nWhile the residual concept bottleneck improves adaptability, it potentially introduces complexity that may affect interpretability. An analysis of the trade-off between model complexity and interpretability, particularly as new residual concepts are added, would be valuable for practitioners seeking interpretable yet robust models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sNe6lngdqC", "reviewer": "Reviewer_wVbS", "summary": "The paper introduces CONDA, a framework that dynamically adapts Concept Bottleneck Models (CBMs) to handle distribution shifts using only unlabeled target domain data, thereby improving interpretability and performance.", "questions": "1) Could you explore more advanced pseudo-labeling methods, such as weak and strong augmentations or soft nearest-neighbor voting, to see if they improve the robustness and accuracy of the adaptation process?\n2) Besides accuracy, what quantitative metrics did you use to evaluate the interpretability of the adapted models?  Could you provide more detailed results on interpretability?\n3) Have you tested CONDA with different types of foundation models and concept bottleneck constructions?  If so, what were the results, and if not, what are your thoughts on its generalizability?\n4) Could you perform a sensitivity analysis to understand how different hyperparameters affect the performance of CONDA, and provide guidelines for tuning these parameters?\n5) It makes me feel that the overall workload/experiment/contribution is limited. In addition, I would also like to see some case studies and visualizations to better understand your problems and methods. Overall, I find this paper a bit like an unfinished version, but I have to admit that the problem setting is actually quite interesting and meaningful.\n\nI look forward to talking to the authors during rebuttal to get a better understanding of the paper. My initial score is only based on my current understanding of this paper. I really hope that I can better understand the whole paper and improve my score during the rebuttal.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The research is well-designed and comprehensive, with robust experimental results across multiple datasets, demonstrating significant improvements in accuracy and interpretability. The paper is well-written and clear, making the technical aspects and findings accessible. Its significance lies in enhancing the robustness and interpretability of foundation models in real-world applications, particularly in safety-critical domains where distribution shifts are common.", "weaknesses": "1) The paper does not thoroughly explore the sensitivity of the results to different hyperparameters. \n2) The computational efficiency of CONDA is not extensively discussed.  Evaluating the runtime and resource requirements, especially for large-scale datasets, would be valuable for practical applications.\n3) The paper focuses on accuracy improvements but could benefit from more quantitative metrics to evaluate the interpretability of the adapted models.  This would provide a more comprehensive assessment of the framework's impact on interpretability. I'm actually a researcher in this area, and I've been wondering about interpretable metrics for this kind of unsupervised/unlabled CBM task, so I didn't mean to be hard on you, I just wanted to hear what you have to say about the evaluation of interpretable credibility in this area.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cWebCK79LD", "reviewer": "Reviewer_yYWh", "summary": "This paper studies an interesting problem, i.e., how to transform non-interpretable foundation models into concept-based interpretable decision-making pipelines under different types of distribution shifts. To address this problem, this paper proposes an adaptive concept bottleneck framework (CONDA) to address these distribution shifts, that dynamically adapts the concept-vector bank and the prediction layer based on unlabeled data from the target domain. The authors also conduct experiments to evaluate the performance of the proposed method.", "questions": "[1] This paper introduces $r$ additional concept vectors. Could you clarify the potential criteria to select these additional vectors? What are the connections between these different criteria? How would different criteria influence the performance of your proposed method? \n\n[2] This paper mentions that the parameters of the residual CBM are randomly initialized. What are the potential effective randomization techniques for the residual CBM? Do these different randomization techniques have difference influence degrees on the performance of the proposed method? \n\n[3] How to show that the adopted cosine similarity-based regularization in the objective can ensure that the new concept vectors in are minimally redundant with each other, and also have minimal overlap with the existing concept vectors? It would be better if the authors could provide the experiments and high-level analysis for this.\n\n[4] How to determine the effective top-$k$ nearest neighbors for different $\\tilde{c}_{i}$? How would different choices influence of the performance of the proposed method?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "[+] This paper lists several possible failure modes of the decision-making pipeline of a foundation model equipped with a Concept Bottleneck Model (CBM)\n\n[+] To handle the potential incomplete and new concepts to bridge the distribution gap between the source and target domains, this paper introduces a residual CBM with additional concept vectors and a linear predictor.", "weaknesses": "[-] More explanations should be provided to clarify the rationale behind the techniques adopted in the proposed method. For example, this paper introduces $r$ additional concept vectors. Could you clarify the potential criteria to select these additional vectors? What are the connections between these different criteria? How would different criteria influence the performance of your proposed method? \n\n[-] The authors fail to provide the complexity analysis of the proposed method. The proposed method consists of some components, which could involve several different steps (e.g., using an ensemble of zero-shot predictor and linear probing predictor to get pseudo-labels). It would be better if the authors could provide the detailed complexity analysis of the proposed method.\n\n[-] The experiments to support the proposed method are not sufficient. For example, it is unclear about how the cosine similarity-based regularization in the objective can ensure that the new concept vectors are \"minimally\" redundant with each other and have \"minimal\"  overlap with existing concept vectors? It would be helpful if they could include experiments and a high-level analysis to support this \"minimal\" overlap.\n\n[-] Some typos, e.g., Appendix ?? in line 875.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)