Concept-Based Unsupervised Domain Adaptation
Authors: Xinyue Xu
Yueying Hu
Hui Tang
Yi Qin
Lu Mi
Hao Wang
Xiaomeng Li
Venue: ICML 2025 poster (2025)
Keywords: Interpretability
Concepts
Domain Adaptation
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: We propose Concept-based Unsupervised Domain Adaptation (CUDA), a framework that improves interpretability and robustness of Concept Bottleneck Models under domain shifts through adversarial training.

Abstract: Concept Bottleneck Models (CBMs) enhance interpretability by explaining predictions through human-understandable concepts but typically assume that training and test data share the same distribution. This assumption often fails under domain shifts, leading to degraded performance and poor generalization. To address these limitations and improve the robustness of CBMs, we propose the Concept-based Unsupervised Domain Adaptation (CUDA) framework. CUDA is designed to: (1) align concept representations across domains using adversarial training, (2) introduce a relaxation threshold to allow minor domain-specific differences in concept distributions, thereby preventing performance drop due to over-constraints of these distributions, (3) infer concepts directly in the target domain without requiring labeled concept data, enabling CBMs to adapt to diverse domains, and (4) integrate concept learning into conventional domain adaptation (DA) with theoretical guarantees, improving interpretability and establishing new benchmarks for DA. Experiments demonstrate that our approach significantly outperforms the state-of-the-art CBM and DA methods on real-world datasets.

Concept-Based Unsupervised Domain Adaptation
Xinyue Xu
Yueying Hu"
Hui Tang'
Yi Qin'
Lu Mi Hao Wang? Xiaomeng Li'
Abstract
Before Addgtaation
After Adaptation Cenceee Embriding Dhariaatien
Grousd- Truil (GT)
Coocegt Entlediing
Conncee Prddictas Dooributien
Concept Bottleneck Models (CBMs) enhance in- terpretability by explaining predictions through human-undenstandable concepts but typically as- sume that training and test data share the same distribution. This assumption often fails under domain shifts, leading to degraded performance and poor generalization. To address these limita- tions and improve the robustness of CBMs, we propose the Concept-band main Adaptation (CUDA) framework. CUDA is designed to: (1) align concept representations across domains using adversarial training, (2) introduce a relaxation threshold to allow minor domain-specific differences in concept distribu- tions, thereby preventing performance drop due to over-constraints of these distributions, (3) infer concepts directly in the target domain without re- quiring labeled concept data, enabling CBMs to adapt to diverse domains, and (4) integrate con- cept learning into conventional domain adaptation (DA) with theoretical guarantees, improving in- terpretability and establishing new benchmarks for DA. Experiments demonstrate that our ap- proach significantly outperforms the state-of-the art CBM and DA methods on real-world datasets.
Consrpt Dacribalie
G:
by Sald Wiee
+
du Balf Nape
And 20%
Vey Dificaat
49%
Unsupervised Do-
Figure I. Illustration of our key idca. Left: Ground-truth (GT) concept distributions (for each concept) (top) and data distributions (bottom). Right: Uniform alignment (top) and relaxed alignment (bottom) after adaptation. Our relaxed alignment allows for greater differences between source and target concept distributions; such flexibility leads to predicted concept distributions closer to the ground truth and therefore higher final classification accuracy.
allows experts to understand misclassifications and make necessary interventions when needed (Abid et al., 2022). However, existing CBMs typically assume that the training and test data share the same distribution, which limits their effectiveness in real-world applications where domain shifts between training and test sets are common. For example, methods such as CBMs (Koh et al., 2020) and Concept Embedding Models (Zarlenga al., 2022) demonstrate a significant drop in performance when tested under domain shift conditions. These models achieve only around 66% accuracy under background shifts, a notable drop compared to their 80% accuracy on test sets that align with the training distribution, as observed on the CUB dataset (Wah et al., 2011) 1) (Sec. 5). Despite these findings, the challenge of de- signing interpretable models capable of handling real-world domain shifts remains largely underexplored.
1. Introduction
Black-box models often lack interpretability, making them difficult to trust in high-stakes scenarios. Concept Bottle- neck Models (CBMs) (Koh et al., 2020; Ghorbani et al., 2019) tackle this interpretability issue by using human- understandable concepts. These models first predict con- cepts from the imput data and then use concepts to predict the final label, thereby improving their interpretability, e.g., pre- dicting concepts "black eyes" and "solid belly" to classify and interpret the bird species "Sooty Albatross". This also
A straightforward approach is to combine CBMs with Do- main Adaptation (DA) (Ben-Davis) et al., 2010; Ganin et al., 2016), which tackles domain shifts by utilizing labeled data from source domains alongside unlabeled (or sparsely la- beled) data from target domains. Specifically, a naive com- bination of CBMs and DA would simply add concept learn- ing into DA models. Unfortunately, this method performs poorly (more results in Appendix C.2) for two reasons. First, it enforces separate class-wise and concept-wise alignment, failing to unify them into a single feature space, limiting both interpretability and generalization. Second, existing DA methods assume uniform (perfect) alignment between
Equal contribution 'The Hong Kong University of Science and Technology GGeorgia Institute of Technology 'Rutgers University. Correspondence to: Xiaomeng Li <eexmli@ust.in ust.
Proceedings of the 42 International Conference on Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Concept-Basce Unsupervised Domain Adaptation
source and target concepts, overlooking domain-specific variations that are essential for CBMs to capture meaningful and interpretable concepts. As shown in Fig. I (upper-right), while uniform (perfect) alignment can strictly align source and target concepts, it overlooks the inherent differences between concepts across domains. Such over-constraints lead to significant performance drops.
ual fitting, storing Concept Activation Vectors (CAVs) (Kim et al., 2018) in concept bank, which eliminates the need for retraining on target domains. DISC (Wu et al., 2023) com- plements this by building comprehensive concept bank that covers potential spurious concept candidates. CONDA (Choi et al., 2024) further extends PCBMs by performing test-time adaptation using pseudo-labels generated by foun- dation models. Our approach combines the advantages of these methods: it requires neither retraining nor concept labels in the target domain, while retaining the complete interpretability of the original concepts. Unlike PCBMs and CONDA, our method supports direct evaluation of con- cept learning performance, ensuring both interpretability and strong performance in the target domain. Note that our work is orthogonal unsupervised concept interpretation of foundation models (Wang et al., 2024a;b; Wang & Yeung, 2016; 2020).
One of our key ideas is therefore to introduce a degree of relaxed
relaxation. As shown in Fig. (bottom--ight) alignment allows for greater differences between source and target concept distributions, c.g., allowing the propor- tion of the concept "Primary Color: Brown" to be 19% in the source domain and 17% in the targe domain for bird classification; such flexibility leads to predicted concept distributions closer to the ground truth and therefore higher final classification accuracy. Specifically, we propose novel Concept-based Unsupervised Domain Adaptation (CUDA) framework, simple yet effective approach with strong generalization capabilities. To achieve this, we in- troduce a novel relaxed uniform alignment loss that adapts more flexibly across domains. This approach enables the learning of domain-invariant concept embeddings while effectively preserving domain-specific variations. We sum- marize our contributions as follows:
Domain Adaptation. In domain adaptation, the task re- mains the same across source and target domains, while the data distributions differ across domains (Pan & Yang, 2009). Our work assumes unlabeled data in the target do- main, falling under the category of unsupervised domain adaptation (UDA) (Beijbom, 2012). Existing UDA methods primarily focus learning domain-invarte abling a classifier trained on source to be applied to target data. These methods can be broadly categorized into three adaptation paradigms: input-level (Sankaramarayanan et al., 2018; Hoffman et al., 2018), feature-level (Ganin et al., 2016; Saito et al., 2018; Xu al., 2022; Liu et al., 2023; Xu et al., 2023; Huang al., 2024), and output-level (Zhang et al., 2019b; Tang al., 2020; Hu et al., 2022). Input-level adaptation stylizes data (e.g.. images) from one domain to match the style of another. This involves generating source- like target data as regularization (Sankaranarayanan et al., 2018) or target-like source data as training data (Hoffman et al., 2018), often using GANs (Goodfellow et al., 2014). Feature-level adaptation minimizes feature distribution dis- crepancies between domains (Long al., 2015) employs adversarial training at the domain (Ganin et al., 2016; Xu et al., 2023) or class levels (Saito et al., 2018; Huang et al., 2024). Output-level adaptation focuses on learning target- discriminative features through self- training with pseudo- labels (Zhang et al., 2019b; Tang et al., 2020; Hu et al., 2022). None of the methods above provide concept-level interpretability. In contrast, our approach, for the first time, introduces the concept-level perspective for adaptation. By leveraging concept learning, we bridge domain discrepan- cies while achieving concept-based interpretable UDA.
features, en-
We provide the first generalization error bound for CBMs, with theoretical analysis on how concept em- beddings can be utilized to align source and target dis- tributions in DA.
Inspired by the theoretical analysis, we propose the first general framework for concept-based DA, providing both cross-domain generalization and concept-based interpretability.
We improve generalization of CBMs and eliminate the need for labeled concept data and retraining on the target domain, enabling adaptation to diverse domains. Experiments on real-world datasets show that our method significantly outperforms state-of-the-art CBM and DA models, establishing new benchmarks for concept-based domain adaptation.
2. Related Work
Concept Bottleneck Models (CBMs) (Koh et al., 2020) use bottleneck models to map inputs into the concept space and make predictions based on the extracted concepts. Concept Embedding Models (CEMs) (Zarlenga et al., 2022) improve performance by using a weighted mixture of positive and negative embeddings for each concept. Energy-based Con- cept Bottleneck Models (ECBMs) (Xu et al., 2024) unify prediction, concept correction, and interpretation as condi- tional probabilities under a joint energy formulation. Post- hoe Concept Bottleneck Models (PCBMs) (Yuksekgonul et al., 2022) employ post-hoc explanation model with resid-
3. Methodology
In this section, we begin by analyzing the generalization er- ror bound for CBMs and then discuss our proposed method
2

---

Concept-Basce Unsupervised Domain Adaptation
inspired by the analysis. A detailed theoretical analysis is provided in Sec. 4.
For any h H with H. as the hypothesis space, Ben-David et al. (2006; 2010) present a theoretical upper bound on the target error eT()):
Problem Setting and Notations. We consider the concept- based UDA setting with Q classes and K concepts. The input, label, and concepts are denoted as x E X, y E y C [0,1]8, and с E C = {0, 10, respectively; note that y represents the space of Q R-dimensional one-hot vectors while doos not. We use discrete domain indices (Wang et al., 2020) 0 and u 1 to denote source and target domains, respectively. Given the labeled data {ar] vf, c() from source domain (u = 0), and unlabeled data {=]}== from target domain (u = 1), the goal is to accurately predict both the classification labels {u/) and the unlabeled concepts {c0)= in the target domain.
(I) where = minnew (Es(h) + <r denotes the error of a joint ideal hypothesis on both source and target domains, and the HAH divergence dHAH (Ds, Dr) represents the corst-case source-target domain discrepancy over concept embedding space (different from Ben-David et al. (2010), which is in the input space).
eT(h) S cs(h) + Hdnan Dr) +1,
Concept Embeddings v,. Given that using scalar repre- sentations for concepts can significantly degrade predictive performance in realistic settings (Mahinpei et al., 2021; Dominici et al., 2024), we choose to use more robust approach that constructs positive and negative semantic em- beddings for each concept (Zarlenga et al., 2022; Xu et al., 2024). Specifically, the concept embedding v is represented as a concatenation of sub-embeddings for K concepts, i.e. v - [o.] K E IR', where each sub-embedding v, is a com- bination of its positive and negative embeddings weighted by the predicted concept probability 24
3.1. Generalization Error Bound for CBMs
Previous works on CBMs have primarily been evaluated on background shift tasks (Koh et al., 2020), but they lack theoretical analysis of the generalization error bound. To address this limitation and provide deeper insights into our proposed method, we begin by analyzing the generalization error bound for CBMs. Although our primary focus is on binary classification, our framework can extend to multi- class classification following Zhang et al. (2019a; 2020), which we leave for future work.
U. C v.
(+) + (1 2) -))
(2)
where = =2] E RRK.
Generalization Bound without Concept Terms. Building on the framework established in Ben- David et al. (2006; 2010), we formalize the data generation process for both source domain and target domain using marginal (data) dis- tribution and underlying labeling function pairs, denoted as (Ds. fs) for the source domain and (Dr, fr) for the target domain. Here, Ds and Dr denote the marginal distribu- tions over the input space X, while fs X' - [0, 1] and fr X' [0, 1] represent the labeling functions that assign the probability of an instance being classified as label 1 in the source and target domains, respectively. We adopt concept embedding encoder E X' + V C IR', a function which maps inputs to concept embeddings. This induces distributions Ds and Dr over the concept embedding space v, as well as corresponding labeling functions:
Ideal Concept Embeddings e.f. Note that ground-truth concepts
[ci]/_ {0, 1j are only accessible in the source domain. which allows us to define an idcalized scenario for analyzing the source error. In this scenario, we replace the predicted concept probabilities č with the ground-truth concepts e to construct the ideal concept em- beddings of = [u E RR, with each of defined as:
of - c d. (+) + (1- o
where G; denotes the ground truth of the i-th concept. This eliminates the noise introduced by the prediction, providing minimal-error baseline that isolates the inherent limitations of the model itself.
Source Error with Ideal Concept Embeddings. To quan- tify performance under this noise-free baseline, we define the source error for e.:
fs( e EDWDD [fs( E(x) v).
Fr()) 2 E (x) F((x)) = v).
(b) - (h, (9) = E FeewDg
- M
We define hypothesis h: - 00, 1] as a predictor operat- ing over the concept embedding space V. For any embed- ding v E V, h(v) outputs the predicted probability that the classification label is 1. The error of h on the source and target domains is then defined as:
where DE denotes the marginal distribution over oe and ig is the corresponding induced labeling function, defined as:
F5(09) A (fsD)lfs(x) E(z) = 09].
Generalization Bound with Concept Terms. With this setup, we are ready to perform a generalization error anal- ysis of concept- based models for the binary classification task. A complete proof can be found in Appendix B.
(h) a Es (h, Fs) = E
[is(o) h(w) [ir(o) - M(O)
(h( a (h, Fr) = B Br
3

---

Concept-Basee Unsupervised Domain Adaptation
Source image X
Feature Eimbeddding
Concept Predictions = [AI
Concept Emboedings == [Muli-
Label Predictor
Class Label
y
94
Feature Extractor +
Noural Network Conncepe
1
:
Ex
Domain
Domain Label
Classifier D
Target Image Xx
Concept Embedding Encoder E
P2 1 + (1 60)0
Figure 2. Overview of our CUDA framework. The framework takes source and target domain images inputs to first learn feature embeddings. Positive embeddings of and negative embeddings w!" are then derived from these feature embeddings. These are passed through the neural network Geoncept to obtain concept predictions 2, which are subsequently combined to construct the final concept embeddings During training, adversarial training is employed: the domain classifier (discriminator) is trained first, followed by the concept embedding encoder and label predictor. These two steps are alternated throughout the training process.
where R > 0 is a finite constant, 17f = min <g(h)) cT())) By comparing the noise-free source error eg (which serves as the theoretical baseline for evaluating performance un- der ideal conditions) with the actual source error eş that incorporates noisy predicted probabilities, we can directly quantify the additional error introduced by prediction noise. This relationship is formalized in the following lemma.
NEH
and ( DT) denotes the H.H divergence between distribution D3 and distribution Dr-
Theorem 3.1 implies that the target error ET can be mini- mized by reducing the source error with ground-truth con- cepts eg. the HAH divergence dHAH (DS, Dr), and the discrepancy Es [l|è - c||2] simultaneously, thereby achiev- ing high classification accuracy on the target domain.
Lemma 3.1 (Source Error with Predicted Concept Em- beddings). Let H. be a hypothesis space where all hypothe- ses h E H are L-Lipschitz continuous under the Euclidean norm || ||2 for some constant L > O. Assume that for all E V, ||0/| is bounded. Then, for any hy, h2 E H, there such that
3.2. Concept-Based Unsupervised Domain Adaptation
exists a finite constant
Inspired by Theorem 3.1, we propose a framework, dubbed Concept-based Unsupervised Domain Adaptation (CUDA). Fig. provides an overview of CUDA, which involves four players:
<s(1n, S (11 h2) + r. Es [1@--cl22) c||2):
where cg(h1. 122 = E. ( h2(will and h2) = Eaenag [1h](v9) - h2( are the dis- agreement between hypotheses h and h2 w.r.t distributions Ds and D% respectively, and Es denotes the expectation taken over the source distribution.
concept embedding encoder E which generates the concept embedding = E(x) given the input æ, concept probability encoder Foprob which predicts concepts = Epros (x) (though Eprob is part of E, we treat them separately for analysis purposes),
Lemma 3.1 quantitatively connects the concept prediction performance to the source error. Specifically, IEs ||| c| quantifies the discrepancy between the predicted concepts and ground-trute -tuthh concepts c, serving as a measure of the accuracy of concept prediction. We defer the discussion of the validity of the L-Lipsehitz continuity assumption to Appendix C.2. With this foundation, we are now ready to models.
discrininator D which identifies the domain ū using the concept embedding e, i.e. û D(v), and predictor F which predicts the classification label 9 based on the concept embedding ÿ = F(v).
The Need for Relaxed Alignment. Before introducing the game, note that the adversarial interaction between E and D forces E to strip all domain-specific information from the concept embedding v at the optimal point, mak- ing v effectively concept probability é is part of v, ē should also become domain-invariane achieving perfect (uniform) alignment across domains. However, the concepts in the source and tar- get domains are often inconsistent due to differences in data distributions in practice (Xu et al., 2022; Liu al., 2023);
derive bound the target error for concept-
Theorem 3.1 (Target-Domaal Error Bound for Concep- t-Based Models), Under the assumption of Lemma 3.1, for any h E H. we have:
-ivariann Intuitively, since the
(b)) < <g(h) + Idnax (Dg, Dy) + 1
+ R. Es ||ē - -cllz),

---

Concept-Basse Unsupervised Domain Adaptation
such discrepancies make the uniform alignment overly re- strictive, it may impose unnecessary constraints c, therefore harming performance in the target domain. To address this gap, we draw inspiration from (Xu et al.. 2022: Liu et al., 2023) and propose a relaxed alignment mecha- nism on v, which naturally translates to tolerating smaller discrepancies in é between the source and target domains.
Es |l| ellg) ensuring accurate concept probability esti- mation. Meanwhile, E collaborates with the predictor F to reduce the source error eg, and "fools" the discrimina- tor D to minimize the HAH divergence d (Dg. Dr). Together, they jointly optimize the upper bound of the target- domain error, i.e., Eqn. 3 of Theorem 3.1.
Concept Loss C.. In Equ. 5, the concept loss is defined as:
Overall Objective Function. Formally, CUDA solves the following optimization problem:
La (Ferob) - Ess [he
c)],
min
(4)
where Le is the binary cross-entropy loss, Epros E R where each dimension (#)r is the predicted concept probability for concept i; the corresponding ground-truth concept is o (note that C = [G]]K E RR).
min E, E F) + Aooo(Eprot) - D), (5)
where Lp is the prediction loss, La and La are the discrimi- nator loss with and without relaxation, respectively (more details below), and C. is the concept loss. The hyperparame- ters Ay and de balance Cp(E, F)), (Eprob) and (), Below, we discuss each term in detail.
Discriminator Loss without Relaxation La and Discrimi- nator D. The discriminator D identifies the domain u from the concept embedding v. Given E, the discriminator loss
La D) E (DL((D())))) u]],
(8)
Prediction Loss Lp and Predictor F. The prediction loss F) in Eqn. 5 is defined as:
where La is the binary cross-entropy loss, u is the domain label which indicates whether æ comes from the source (u 0) or target (u 1) domain, E denotes the expectation taken over the entire data distribution p(zz u). and D(E(x)) denotes the probability of æ belonging to the target domain.
- Es (FF(((((()))
(6)
where Lp is the cross-entropy loss, F(E(x)) e Nº and each element F( F(E(w)); the predicted probability for class i, and Es denotes the expectation taken over the source data distribution (ar, y, c); note that the label y and ground- truth concepts e are only accessible in the source domain.
Relaxed Discriminator Loss Ca. La is only used to learn the discriminator D (Eqn. 4). To learn the encoder E in Eqn. 5, we introduce relaxed discriminator loss:
Ca(FE, D) A min (Ca(E, D),
(9)
Concept Embedding Encoder E. The concept embedding encoder generates both concept predictions and concept embeddings v. As presented in Fig. 2. positive and negative embeddings for the i-th concept firstly constructed as: (o(t) uu = yp (D(x))) where (-)) is a pretrained back- bone and (-) is the linear layer. Then the concatenated embeddings (o]+)), of{>) are passed through Geoncept to pre- dict the concept probability: 2 = Geoncepe (lo) (+) o "1). Thus, we have:
where 0 < S max (F, D) a relaxation threshold, ef- fectively controlling the tolerance for domain discrepancies in the concept embedding v.
Relaxed Discriminator Loss for Relaxed Alignment. By capping the domain classification loss at T, this relaxation intentionally sacrifices a small amount of domain align- ment, corresponding to the second term (dHAH(DS Dr) of Eqn. 3, to reduce the concept prediction error in the fourth term Es ||ē ||اا of Eqn. This trade-off enables more flexible optimization of the concept embedding en- coder E, balancing domain alignment and concept predic- tion accuracy. Besides, it allows the encoder E to retain domain-specific information stemming from intrinsic dif- ferences between source and target concepts, crucial for downstream tasks (see Sec. 4 for a comprehensive analysis). We summarize CUDA's training procedure in Algorithm I of Appendix C.3. Essentially, it alternates between Eqn. 4 and 5 with adversarial training using Eqn. 6~9.
è= Exrob( (((()))))GGoooooa [Gooncept (10)
(t)) 0 D
where (-ros(-) is the concept probability encoder compos- ing (.), (p-) and Geancepe(+) (-).
As mentioned in Eqn. 2, we then use the full concept em- bedding encoder E to compute the concept embedding v:
v- = E(x) = [ k (x)))o((x))
[2: v. + (1 24) 0. |)
((t + (1 (Eprob())),) D K
|
Note that the concept probability encoder Epros is part of the full concept embedding encoder E. We separate con- cept probability encoder Eprob out to facilitate theoreti- cal analysis. Specifically, Exrob is optimized minimize
4. Theoretical Analysis for CUDA
In this section, we provide the theoretical guarantees for CUDA. All proofs are provided in Appendix B.2.
5

---

Concept-Based -ased Unsupervised Domain Adaptation
Simplified Game We start by analyzing a simplified game which does not involve the concept probability encoder Epros and the predictor F. Specifically, we focus on
When T = log 2, CUDA achieves uniform alignment, which is defined in Definition 4.1 below.
Definition 4.1 (Uniform Alignment). A concept-based DA
model achieves uniform alignment if its encoder satisfies
D Ca(E, D), max L, e min(Ca(E,
(10)
(v) = (T))
Ps é ( pf))),
(11)
or equivalently, 1. u and é T. u.
where the discriminator loss without relaxation La is defined in Eqn. 8, and 0 < I max Cx(E, D) is a relaxation threshold that quantifies the allowed deviation from uniform alignment of v. Solving this game ensures that D learns to distinguish domain representations, while E can "fool" the discriminator with the relaxation threshold т, thereby flexibly aligning concept embeddings across domains.
Relaxed alignment ensures that CUDA is robust to concept differences across domains while maintaining alignment (more empirical results in Sec. 5).
Full Game. For any given E, we then derive the property of the optimal predictor F and establish tight lower bound for the prediction loss.
Lemma 4.1 below analyzes the optimal discriminator in Eqn. 10 with the concept embedding encoder E fixed.
Lemma 4.2 (Optimal Predictor). Given the concept em- bedding encoder E, the prediction loss (p F) has light lower bound
Lemma 4.1 (Optimal Discriminator). For E fixed, the optimal discriminator D is
Cp F) A Es [L,(F(E())). y)] H E(x));
(v) where (v) and (v) are the probability density function of v in source and target domains, respectively.
PT (p
where (.).)) denotes the conditional entropy. The optimal predictor F that minimizes the prediction loss
F*((())) (M = 1 E(x)))
Analyzing the Relaxed Discriminator Loss. Given the optimal discriminator D' in Lemma 4.1, we define the relaxed diseriminator objective in Eqn. 11 as:
where yi denotes the i-th element of y.
Assuming the discriminator D and the predictor F are trained to achieve their optimum by Lemma 4.1 and Lemma 4.2, Eqn. 4 and Eqn. 5 can then be rewritten as:
(.( e [E, = =min(Ca(E) Db), min,C.(E),7), (12)
min Le Le(EEprab) F
(15)
where Ca(E) & La(E, Dz).. Theorem 4.1 below shows that the global optimum of the game in Eqn. 10~11 corre- sponds relaxed alignment of concept embeddings and concept predictions è between source and target domains.
min H(y E(x)) da Ca(),,
(16)
E
where Ca(F)) is defined Eqn. 12. With Eqn. 15~16 above,
Theorem 4.2 below analyzes our optimal concept probability and embedding encoders E and Eproh-
Theorem 4.1 (Relaxed Alignment). If the discriminator D have enough capacity to be trained to reach optimum, the relaxed optimization objective Ea( defined in Eqn. 12 achieves its global maximum if and only if the concept em- bedding encoder satisfies the following conditions:
Theorem 4.2 (Optimal Concept Embedding Encoder). Assuming u L y, if the concept embedding encoder E, concept probability encoder Eprob, the predictor F and the discriminator have enough capacity and are trained to reach optimum, any global optimal concept embedding encoder F. and its corresponding global optimal concept probability encoder Epros have the following properties:
LS)( (u)) log 2 -
(13)
ISD log 2 - I(V, u
(14)
where [(,,1+)) is the conditional mutual information, (C) and ( (2) are the probability function of è in source and target domains, respectively.
Eprob (x) [P(c: 1<x))
(17)
H ( Ex)x))) H( xx)
(18)
Ca(F') max Ca (F
(19)
Theorem 4.1 links the relaxation threshold in CUDA to the alignment of concept embedding v's distributions and concept prediction Es distributions across domains:
Theorem 4.2 shows that, at equilibrium, (1) the optimal concept probability encoder Bir recovers the conditional distribution of the ground-truth concepts, and (2) the optimal concept embedding encoder E preserves all the informa- tion about label y contained in the data x.
When f E (0, log 2), CUDA achieves relaxed align- ment, and the degree of relaxation for is guaranteed be no greater than that of v.
6

---

Concept-Basce Unsupervised Domain Adaptation
Table 1. Performance of concept-hasse methods on both concept learning and classification across different datasets. CEM (w/o R.) indicates "without RandInt". I-II, II-IV and V-VT indicate different skin tone scale in the Fitzpatrick dataset. We mark the best result with bold face and the second best results with underline. Average accuracy is calculated over every three datasets of the same type images.
Datasets
Waterbirds-2
Waterbirds-200
Waterbinds-CUB
AVG
Metries
Concept
Concept F1
Class
Concept
Concept F1
Class
Concept
Concept F1
Class
ACC
CEM
91.1140.13
81,71=0.30
70.27+1.70
93.68
81 222222
62 02.26:1:11
93.64+...5
80.08
66.1840.91
66.31
CEM (wlo R.)
94.17+0.11
81.9622030 50
69.45:2.15
933 76..62
81 ..:220082
63.5611.25 25
93.66+0.14
79.8020.30
65.8
66.30
CBM
93.60
83.89 a
22.1
93. 03.50
83.14 23.
63.89
93.40
82. 20.1100..00
63.89a1.00
67.53
CUDA (Ours)
94.63:0005
92.90 04.97
95.15.10.0
85. 05.06
3..37
94.58:0.or
10.
74.66=0.19
81.15
Datasets
MNIST MNIST-M
SWHN
MNIST
MNIST USPS
AVG
Metries
Concept
Concept F1
Class
Concept
Concept F1
Class
Concept
Concept F1
Class
ACC
CEM
86.55+1.011 1.01
72.97+1.48
+1.8
89.20
78.99
67.5822.01
93.08
85.27
64.03
CEM (wlo R.)
16.10+1.01
72.58=1.01
2.330
89.89+2.20
80.22+4.91
69.76+5.20
92. 1.98
83.75
72.92as.05
61.01
CBM
86.2840.22
72.8620..2
49.66
89.63.0.83
79.51
90.67.2.78
79.34
61.79414.2
58.82
CUDA (Ours)
98.51
97.20=0.02
95.24:0.1
95. 35.22::0.4
90.95:0024
02.49
98. 78..8800
97.46:0000
96.01=0.18
91.25
Datasets
I-11 + I11-IV
III-IV - V-VI
I1I-IV - 1-11
AVG
Metries
Concept
Concept F1
Class
Concept
Concept F1
Class
Concept
Concept F1
Class
ACC
CEM
93.81+0.16
52.01=0.20 25
+3.41
93.05+0.02
66.1620.19
76.27
93.85+a.16
51.32+0..2
71.31
73.67
CEM (wlo R.)
93.78200.7
51.9 31.9820.2 27
73.1319965 63
93.0552..02
66.47:0015
76.86.1.11 19
93.8020.13
54.2620.19
71.72g0.58
73.91
CBM
94.1140.48
52. 17.0.8
72.37=0..0
92.27
56.21
78.82+0.00
94.16+0.34
64.27
70.49+0.00
73.89
CUDA (Ours)
95.37=0.07
79.91=0.16
78.85
04.62.0.01
70.57=0.25
80. 50.58:0.72
95.45:00.0
80. 80.17:00.2
76. 00.43
78.65
Classification accuracy across different datasets. Zero-shot predictor is one of the baselines and components of CONDA. We mark the best result with bold face and the second best results with underline. Average accuracy is calculated over every three datasets of the same type images. Note that these baselines do not have concept accuracy and F1 because they cannot prodict concepts directly.
Table 2.
Dataset
WB-2
WB-200
WB-CUB
AVG
M
..M
8 .M
M U
AVG
I-II
-IIII
I-VIV
V.VI
III
AVG
Model
Zero-shot
59.27as0.00
1.93e0.00
2,11
21.10
11.60
13.16
13. 15ea.co
12.64
69. 89.8
72.5029.00
72.50sma
71.01
PCBM
53.0841.9
28. 85
31.6050.44
38.89
29 6.
21.32=2.12
15.55eouu
22.18
12.134
72.6420..4
72.642004
72:47
CONDA
70.2340
0.70200
0.4
23.82
9.75g
9 9.802a.00 0o
17.80
12.48
13. 13.122000
14.58.0 00
14.581005
14.09
DANN
48.08110.4
07.1920.00 so
64.5220 na
30.93
37 1-11
78.07 sn
73.96
63.19
75. 31
79.16.0
73 73.29
76.07
MCD
55.961244
10.37
64.31
61.71
51 18.08
30.20
a3.90
75.06
15.02
78.14.0.13
12.34
75.20
SRDC
45. 19.0.9.
78.29
12..27
63.73
30 35
78.99
90.71es.s4
67.68
73.70sm
78.09220.0
72.91amas
75.10
UTEP
13.50+o.as
68.0920.42
35.284022
65 98
06.35-s..1
85.04
75.79
70.34+0.44
80.344o
76.66202
77.11
GHH++
45. 45.6521111 is
79 29.87
79 41.46i.... sa
68.333
50 10-400aav so
79:12 so
77.29
75.9845 57
78.76:00 de
75.04400s
76.59
CODA (Ours)
923
10.87
74.0610.15
81.15
95.24.0.13
82.49.0.3r
96.01.10.1
91.25
78.85_0031
80.58
76.53.00.4
78.65
5. Experiments
(Netzer et al., 2011), and USPS (Hull, 1994), as different source and target domains. Since the target labels represent the digits 0-9, we design 11 topology concepts based on these datasets. Besides, we use SkinCON (Daneshjou et al., 2022b) to evaluate our approach in the medical domain. SkinCON includes 48 concepts selected by two dermatolo- gists, annotated the Fitzpatrick 17k dataset (Groh et al., 2021). For our experiments, we use one skin tone as the source domain and another as the target domain. Additional details are provided in Appendix C.I.
We evaluate CUDA across eight real-world datasets.
5.1. Evaluation Setup
Datasets. The original Waterbirds dataset (Sagawa et al., 2019) is split into source domain and a target domain (Waterbirds-shift), by selecting images with opposite label and background; it only includes binary labels and does not have any concept information. To evaluate concept- concept-based based DA, we augment the Waterbirds dataset by incorporating concepts from the CUB dataset (Wah et al., 201 1), leading to three datasets:
Baselines and Implementation Details. For concept-based baselines, we include CBMs (Koh et al., 2020), CEMs (Zarlenga et al., 2022), and PCBMs (Yuksekgonul et al., 2022). Additionally, we use state-of-the-ar unsupervised domain adaption methods baselines, including DANN (Ganin et al., 2016), MCD (Saito et al., 2018) 8). SRDC (Tang et al., 2020), UTEP (Hu et al., 2022), and GH++ (Huang et al., 2024). We also include CONDA (Choi et al., 2024), which performs test-time adaptation on PCBMs. Collec- tively, these methods define a comprehensive benchmark for domain adaptation in the context of concept learning. We summarize the implementation details in Appendix C.2.
Waterbirds-2 is similar to the original Waterbirds with binary classification, i.e., landbirds/waterbin Waterhirds-200 is the augmented version of Water- birds with 200-class labels from CUB, and Waterbirds-CUE contains CUB training data as the source domain and Waterbirds-shift as the target.
We also use digit image datasets, including MNIST (Le- Cun et al., 1998), MNIST-M (Ganin et al., 2016), SVHN
7

---

Concept-Basse Unsupervised Domain Adaptation
Waterbinds-
Waserbicds-20 200
Minerhinb-CUUR
Ciesepp Indox 54
Concept Indes - 97
<2B of So Concepts Source Domain
KDE of Selected Concepts in Source Domrann canree banc Dlwird wly te
Class Aocuncc
Sease Oue cull wis folce
Source Domain
ected Connepts Thrpet Demala
K5E Selteed Corcagen Thrpet Dernen
Conneep Ascuncy
Taysi car vn Tdue Twget contrnt or
Sugac Croupt Piiice
Topp Decamo
Sigett Earr
CUDA CEM
Ratio
Intervention Ratio
Figure 3. Concept intervention performance with different ratios of intervened concepts on Watebirds datasets. The intervention ratio denotes the proportion of provided correct concepts.
Figure 4. The kernel density estimation (KDE) plots compare the distributions of two selected concept indices under three different scenarios: Ground-trutt -ouutd-ruthh (GT), without relaxation (w/o Relax), and with relaxation (w/ Relax).
Evaluation Metrics. We calculate concept accuracy and the related concept F1 score to assess the concept learning process. Note that only concept-based methods, i.e., CEM, CBM, and CUDA, have concept accuracy and concept F1. We also use class accuracy to evaluate the model's s prediction accuracy, All metrics are computed on the target domain.
tervene on (modify) specific predicted concepts to correct model predictions. Our CUDA is also capable of concept intervention while traditional DA is not. Similar to CBMs and CEMs (Koh et al., 2020; Zarlenga et al., 2022), ground-truth concepts with varying proportions at test-time to conduct interventions. Fig. 3 shows the performance of different methods after intervening on (correcting) varying proportions of concepts, referred to as intervention ratios. Our CUDA significantly outperforms the baselines across all intervention ratios in terms of both concept accuracy and classification accuracy.
5.2. Results
Prediction. Tables and 2 summarize the results. Table shows that our CUDA performs exceptionally well within the CBM category, achieving state-of-the-art performance across all metrics. Notably, it outperforms other CBMs by significant margin on the Waterbirds and MNIST datasets, while demonstrating consistent improvements on SkinCON. These results highlight the effectiveness of our method in learning concepts and adapting to domain shifts.
A Alignment Relaxation. In Theorem 1, we discussed the relaxation on the discriminator loss to account for concept differences. Fig. 4 illustrates the distributions of two se- lected concept indices under three scenarios: ground-truth (GT), without relaxation (w/o Relax), and with relaxation (w/ Relax). The GT distribution serves as a reference to evaluate the impact of relaxation concept representations. The curves demonstrate how the relaxation process influ- ences the density distribution of the concepts. Specifically, our relaxed alignment allows for greater differences between source and target concept distributions; such flexibility leads to predicted concept distributions closer to the ground truth and therefore higher final classification accuracy.
The upper section of Table 2 shows results for PCBM meth- ods. Although PCBMs utilize concept banks to improve the efficiency of concept learning, their applicability to real- world domain adaptation tasks is limited, with performance falling short of standard CBMs. While CONDA incorpo- rates test-time adaptation, its effectiveness is inconsistent, and its robustness is inferior to that of vanilla PCBMs. This underscores the importance of learning meaningful concept embeddings merely compressing concepts does not work well for domain adaptation tasks.
6. Conclusion
The lower section of Table 2 shows results for DA methods and our concept-based CUDA. While DA models outperform some concept-based baselines, CUDA remains competitive, achieving the highest average accuracy across each type of the datasets. Note that existing DA methods cannot learn interpretable concepts, making them challenging to apply in high-risk scenarios. Our CUDA addresses this limitation, en- suring interpretability without compromising performance. Limitations and future works are discussed in Appendix D.
In this work, we proposed the Concept-basee Unsuper- vised Domain Adaptation (CUDA) framework to address the challenges of generalization problem in Concept Bot- tleneck Models (CBMs). By aligning concept embeddings across domains through adversarial training and relaxing strict uniform alignment assumptions, CUDA enables CBMs to generalize effectively without requiring labeled concept data in the target domain. Our approach establishes new benchmarks for cancept-based domain adaptation, signifi- cantly outperforming ofate-of-the-arr CBM and DA methods while enhancing both interpretability and robustness.
Concept Intervention. Concept intervention is a key task to evaluate concept- based interpretability, where users in-
8

---

Concept-Based Unsupervised Domain Adaptation
Impact Statement
Ghorbani, A., Wexler, J., Zou, J. Y., and Kim, B. Towards automatic concept-basee explanations. Advances in neu- ral information processing systems, 32, 2019.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. Advances in neural informa- tion processing systems, 27, 2014.
References
Abid, A., Yuksekgonul, M., and Zou, J. Meaningfully de- bugging model mistakes using conceptual counterfactual explanations. In International Conference on Machine Learning, pp. 66-88. PMLR, 2022.
Groh, M., Harris, C., Soenksen, L., Lau, F., Han, R., Kim, A., Koochek, A., and Badri, O. Evaluating deep neu- ral networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1820-1828, 2021.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. 2023.
He, K. Zhang, X., Ren, S., and Sun, Deep residual learning for image recognition. In 2016 IEEE Conference Computer Vision and Pattern Recognition (CVPR), Jun 2016. doi: 10. 19011000022001..00.. URL. http:// ww..ooi.o. www.olggg//ooooo 1 1109 hwt111///vp/.... 2016. 90.
Beijbom, O. Domain adaptations for computer vision appli- cations. arXiv preprint 1xXiv:2211.4860 2012.
Ben-David, S., Blitzer, J., Crammer, K., and Pereira, F. Anal- ysis of representations for domain adaptation. Advances in neural information processing systems, 19, 2006.
Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., Efros, A., and Darrell, T. Cycada: Cycle-consistent adversarial domain adaptation. In International confer- ence machine learning, pp. 1989-19999999 1998. Pmlr, 2018.
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W. A theory of learning from different domains. Machine Learming, pp. 151-175, May 2010. doi: 10. 00.100077/100944-09955555-----42 URL http:: dx. doi. (adi..orgeoo orriiorg/lie 11 100 /00000////110/ 110.1/110000001111000000-----000
Hu, J., Zhong, H., Yang, F., Gong, S., Wu, G., and Yan, J. Learning unbiased transferability for domain adaptation by uncertainty modeling. In European Conference on Computer Vision, pp. 223-241. Springer, 2022.
Choi, J., Raghuram, J., Li, Y., and Jha, Adaptive concept bottleneck for foundation models under distribution shifts. arXiv preprint ar arXiv:2412.44097, 2024.
Huang, F., Song, S., and Zhang, L. Gradient harmonization in unsupervised domain adaptation. IEEE Transactions Pattern Analysis and Machine Intelligence, 2024.
Daneshjou, R., Vodrahalli, K., Novoa, R. A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S. M., Bailey, E. E., Gevaert, O., et al. Disparities in dermatology ai performance on a diverse, curated clinical image set. Science advances, reabbb677144 2022a.
Hull, J. J. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550-554, 1994.
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F,, et al. Interpretability beyond feature attribu- tion: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pp. 2668 3668-2677. PMLR, 2018.
Daneshjou, R., Yuksekgonul, M., Cai, Z. R., Novoa, R., and Zou, J. Y. Skincon: A skin disease dataset densely debugging
annotated by domain experts for fine- and analysis. Advances in Neural Information Processing Systems, 35:18157-18167, 2022b.
Koh, P. W., Nguyen, T., Tang. Y. Y.S., Mussmann, S., Pierson, E., Kim, B., and Liang, Concept bottleneck models. International Conference on Machine Learning, pp. PMLR, 2020.
Dominici, G., Barbiero, P., Zarlenga, M. E., Termine, A., Gjoreski, M., and Langheinrich, M. Causal concept em- bedding models: Beyond causal opacity in deep learning. arXiv preprint ar XrXiv:2405.1 165540055667...... 2024.
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient- based learning applied to document recognition. Proceed- ings of the IEEE, 86(1 1)22222223 1998.
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., March, M., and Lempitsky, V. Domain- adversarial training of neural networks. Journal of ma- chine learning research, 1155)):1-35, 2016.
Liu, T., Xu, Z., He, H., Hao, G., Lee, G.-H., and Wang. H. Taxonomy-struu structured domain adaptation. ICMI., 2023.

---

Concept-Basee Unsupervised Domain Adaptation
Long, M., Cao, Y., Wang, J., and Jordan, M. Learning transferable features with deep adaptation networks. In International conference on machine learning, pp. 97- 105. PMLR, 2015.
Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The caltech-ucsd birds-200-2011 dataset. 2011.
Wang. H. and Yeung, D.-Y. Towards bayesian deep learning: A framework and some existing methods. TDKE, 28(12): 3395-3408, 2016.
Mahinpei, A., Clark, J., Lage, I., Doshi-Velez, F., and Pan, W. Promises and pitfalls of black-box box concept learning models. arXiv preprint arXiv:2106. 2021.
Wang. H. and Yeung. D.-Y. A survey on bayesian deep learning. CSUR, 53(5):1-37, 2020.
Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng. A. Y., et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised learning, volume 2011,
Wang, H., He, H., and Katabi, D. Continuously indexed
domain adaptation. In ICML, 2020.
pp. 4. Granada, 2011.
Wang. H., Tan, S., Hong, Z., Zhang, D., and Wang. H. Variational language concepts for interpreting pretrained language models. arXiv preprint, 2024a.
Pan, S. J. and Yang, Q. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10): 1345-1359, 2009.
Wang, H., Tan, S., and Wang, H. Probabilistic conceptual explainers: Towards trustworthy conceptual explanations for vision foundation models. In ICML, 2024b.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Wu, S., Yuksekgonul, M., Zhang, L., and Zou, J. Discover and cure: Concept-aware mitigation of spurious correla- tion. In International Conference on Machine Learning, pp. 37765-37786. PMLR, 2023.
Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case gener- 2019.
Xu, X., Qin, Y., Mi, L.. Wang, H., and Li, X. Energy-based concept bottleneck models: unifying prediction, con- cept intervention, and conditional interpretations. arXiv preprint arXiv:2401.14444 2024.
alization. arXiv preprint
Saito, K., Watanabe, K., Ushiku, Y., and Harada, T. Max- imum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference computer vision and pultern recognition, pp. 3723-3732, 2018.
Xu, Z., Lee, G.-H., Wang, Y., Wang, H., al. Graph- relational domain adaptation. In ICLR, 2022.
Xu, Z., Hao, G., He, H., and Wang, H. Domain indexing variational bayes: Interpretable domain index for domain adaptation. In ICLR, 2023.
Sankaranarayanan, S., Balaji, Y., Castillo, C. D., and Chel- lappa, R. Generate to adapt: Aligning domains using generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pp. 8503 8512, 2018.
Yuksekgonul, M., Wang, M., and Zou, J. Post-hoc concept bottleneck models. arXiv preprint arXhv:2205115480. 2022.
Shen, J., Qu, Y., Zhang, W., and Yu, Y. Wasserstein distance guided representation learning for domain adaptation. In Proceedings of the AAAI conference artificial intelli- gence, volume 32, 2018.
Zarlenga, M. E., Barbiero, P., Ciravegna, G., Marra, G., Gi- annini. F., Diligenti, M., Shams, Z., Precioso. F., Melacci, S., Weller, A., et al. Concept embedding models. arXiv preprint MoXiv:2209.0005: 2022.
Speer, R., Chin, J., and Havasi, C. Conceptnet 5.5: An open multilingual graph of general knowledge. In Pro- ceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
Zhang, Y., Liu, T. Long, M., and Jordan, M. Bridging theory and algorithm for domain adaptation. In International conference on machine learning, pp. 7404-7413. PMLR, 2019a.
Tang, H., Chen, K., and Jia, K. Unsupervised domain adaptation via structurally regularized deep clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8725-8735, 2020.
Zhang, Y., Tang, Ji, K., and Tan, M. Domain-symmetric networks for adversarial domain adaptation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5031-5044 2019b.
10

---

Concept-Based Unsupervised Domain Adaptation
Zhang, Y., Deng, B., Tang, H., Zhang, L.,, and Jia, K. Un- supervised multi-class domain adaptation: Theory, algo- rithms, and practice. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 44(5):2775-277 27992, 2020.
Zhao, M., Yue, S., Katabi, D., Jaakkola, S., and Bianchi, M. T. Learning sleep stages from radio signals: A con- ditional adversarial architecture. In International confer- ence on machine learning. pp. 4100 PMLR, 2017.

---

Koncept-Based Unsupervised Domain Adaptation
A. Notation Table
Table 3. Main notations used in the method section. Click here to return to the main paper.
Notation
Meaning
X
Input space
y
Label space
C
Concept space
V
Concept emhedding space
Hypothesis space
Number of source domain data
m
Number target domain data
K
Number of concepts
Q
Number classes
J
Dimension of concept embedding Ground-truth concepts
c
Concept predictions
The positive/ negative concept emhedding of the i-th concept ez Concept embedding with predicted concepts Concept embedding with ground-trun concepts
ve
Concept emhedding encoder
Eprob
Concept probability encoder
F
Label predictor Domain discriminator
Del Dr
Source/Target domain distribution over X' Source/Tareet domain labeling function over X' Source/Target domain distribution over y
fal fr
Del DT
fsl fr
Source/Target domain labeling function over V Source domain distribution over with ground concepts Source domain labeling function over V with ground-truth concepts Hypothesis function
DE
fg
h
6s
Source error
Target error
cg
Source with ground-truth concepts
B. Proof
B.1. Proof of Generalization Error Bound for CBMs
Lemma 3.1 (Source Error with Predicted Concept Embeddings). Let H. be hypothesis space where all hypotheses h E H are L-Lipschitz continuous under the Euclidean norm || 1l2 some constant L >> 0. Assume that for all v E V, I 22 is bounded. Then, for any ha, h2 E H. there exists a finite constant r > O such that
es(hi, h2) S (g(hn, + Es [|| cl12].
h2 (v) and tween hypotheses hı and h2 w.r.. distributions Ds and Dg. respectively, and Eg denotes the expectation taken over the source distribution.
where es(hi, h2) EEIDD
E Ben [1/ (bg))) are the disagreement be-
Proof. Note that the concept embedding with the ground-truth concepts of and the concept embedding with the predicted concepts are defined follows:
(1 -ilaaaoo-)
(ck (t + (1--cx)0f
c
- -0)o{(-) ..... (Exe + - -xx)o T
12

---

Concept-Based Unsupervised Domain Adaptation
(+)
(+)
and v(>) = 09 10 ..... uk
[ T
where v and ve share the same v(+)
vi K
Then, Es(h1, h2) with
respect to arbitrary concept embedding can be upper bounded by
(s( h2) unDD (lb1(0) h2(m))]
E D. Dg. De (1) (v) - -1((0)) + h1(00) - h2((0) + - h2(v)]l
-
B on De Den [1 - hi h1(b0)) I1 + (11(0)) - -22(001 +
(20)
(1)
2 2L
I en D. r Dg I1lv -112] + E Den [1 h2(09)l
2L. E N Ds
D u Dg |||0 - 0112] + ( h2),
where (i) is due to the Lipschitz continuity of h1, h2 H with constant ) and of cæ (1 c c:)u (-) Bowwng [ h2(v")]]. Note that for the i-th concept, 8, ट. (+) (1 6)p
0, and cg h2) -
Thus, v; v.. = (2: c4) (o!t)-o! ) Because we assume for all = | E V. a|la la is bounded. There exists
-
sufficiently large M, such that max oo! - of o! M. Then, the difference between the concept embedding with the ground-truti concepts and that with the predicted concepts under the Euclidean norm has the following upper bound:
(0) uf) ..... (UK - (wK- "7
10 v 0
2
(61 c1) (1ft))-o-)) o1-1)= ...... (Cx ck) (b/t) )
(21)
< M ||ē - c|l2
Plugging Eqn. 21 into Eqn. 20 and then we can get
Es(h1, 2L NowDs.wewitg _| v112] e h2)
2LM Es [ll cll2ع + eg h2),
where e only available in the source domain. Letting r = 2LM, we complete the proof.
Theorem 3.1 (Target-Domain Error Bound for Concept-Based Models). Under the assumption of Lemma 3.1.for any h E H. we have:
eT(h) S (h) + HHHAN (Dg, (Dg. 1
(3)
+ R Es ||| cl|2],
where > O is finite constant, 1f min (h) et (h), and dRan (Ds, Dr) denotes the HAH divergence between distribution D' and distribution Dr.
heH
Proof. Let h² = arg min (b) + er (h) and 1f = min (h) + er(h) = c3(h") + (h").. By the triangle inequality for keH classification error, i.c. h12) S (h1, ha) + (h2, ha), we have
keR
ci (h) < cr (h') + cr (h, h²) cr (h*) cc h²) + Ics (h, cr (h, h-))
(22)
We define the source error for concept embedding constructed using ground-truth concepts as:
((h)) A (g,,, = E BenDD [īS(0 n(09)]
13

---

Concept-Based Unsupervised Domain Adaptation
where D is the marginal distribution over of and (5(())) - EE-Ds (ss(w)) E(x)) o0] is the corresponding induced
labeling function. Note that f can also be a hypothesis. Then for the second term es (h, h²), we can bound it by the source error with ground-truth concepts:
Es (h,h") S Es
h, [8]
+ Es
hh, [5)
cB (h. fs) c3 h,, (n.P8) + (Les hh', - (h', (h*,F8) t (h'' (8)
1 Es h, f
(23)
(n Es |||e c|ا2 + c (h)) (r2 Es 1| c|اا + c (h^));
where eg hh, /3 = c (h) and og hh hh:J's [3 3 (h*). and (i) is due to Lemma 3.1: there exists finite constant ri, T2
such that Es 1t, J3S s h1 Js) 2 ri Es | - اوااع and és hht [T] 5 (h' r2 IEs Il|è cl|2]. By the
definition of H H. divergence (Ben- (Ben-David et al., 2010):
duam (Dg, Dr) 2 sup h.N'EH
|P Po- (h(v) +((('] P. Pe [h(09) + N((09)|
the last term of Eqn. 22 bounded by
lcs (h, er hh) s Ics (h. ---)))))-- c (h,
ler (h, - (h, 11,
(8))
S r3 Es [l|ē cc12 + ler (h, h²) - Es (h, h*)
r Es è c||2l + sup
le (h, c3 (h,
A.N'EH
(24)
S rr Es |||ē - c1اا + sup
P urDr (h(w) + h'(v)] P DewDg * [h((
+
AAY'H
r3 Es è c1|اع + dHan (Dg.
where (ii) is also due to Lemma 3.1 with the constant r = r3. Plugging Eqn. 23 and Eqn. 24 into Eqn. 22, then we can obtain the final upper bound of target error for CBMs:
(h) S (h') + (h, h2)
(h,
ee (h') (m Es ||| c122 e (h)) + (r2 Es [1| cl122 + +gg + Es ||| c|12] + JHHaH (Dg. Dr)
er (h*) + R Es ||| - c|12l j_NAN (Dg, Dr))
<s (h) + (h²) (h")
c (h) "f R Es |||- c1|اa 1 dHAH (Dg, Dr).
where Rt = Ti + T2 + r3 and 1ff = = (h1*) + eT (h*), completing the proof.
B.2. Proof of Theoretical Analysis for CUDA
Lemma 4.1 (Optimal Discriminator). For E fixed, the optimal discriminator D is
DE(U)=
where ()) and (w) are the probability density function of source and target domains, respectively.
14

---

Concept-Basce Unsupervised Domain Adaptation
Proof. With E fixed, the optimal D should be
D arg min E (p))))
LL(D(E(x)), u)]
D
arg min E (mppa))
u log D(E(#)) (1 log 1- (-D(E()))
D
arg min (vp
B Bunplule)
ulog D(v) (1 u) log [-D(o]]
D
arg min (ppp)))
B [[//o] log D(v) (1 E (u/e]) log T-D(
D
arg max E Bepp(e)
[R [u/v] log (vv) B [ulee] log (1 D(v))].
D
where v E(x). Note that for any (a, b) R2 (0, 0), the function y - a log((1 y) + 6 log(y) achieves its maximum in
[0, at a+bb Note that Plu 0) = P(u 1) 2. thus have
= E [ulv -
= Pu 1aP(ul11p 1/0)
p( Hell=11)(u=s
p(p/u=1)
p(oju_l)pp(ojus) (
where (1) is due to the Bayes rule, and the discriminator does not need to be defined outside of (upp(p (upp(pt(v))
Theorem 4.1 (Relaxed Alignment). If the discriminator D have enough capacity to be trained to reach optimum, the relaxed optimization objective (E( defined Eqn. 12 achieves global maximum if and only if the concept embedding encoder satisfies the following conditions:
(SD( log T, 7 -log2 T -T,,( ule),
(13)
(14)
where I())..)) is the conditional mutual information, y (c) and (c) are the probability density function of è in source and target domains, respectively.
Proof. If D always achieves its optimum w.r.t E during the training, have
Ca(E) A min Ca(E, D) La La, D
= E (La())(E(())) u)]
B(D.N)~((o.))
u log Di(0) + (1 - u) log (0)
(25)
Ep-p(e) Eupp(ulu) [u]] log 1 Bumnpull) + (1 E Eu-pp(ll)) [u]) log 1
Eurp(u) [Puu 1/v) log न(=ाlo] + IP (u 0/ log F(u=Ule] Hullv) H(u) ((v.u).
Note that P(u 1) P(u 0) 1, then we have
H (u)) P(u 1) log F(u=1) + Plu 0) log P(u=0) log 2,
15

---

Concept- Unsupervised Domain Adaptation
and
[(v, u) & (((aa)
log p(a.v) p(n)
pump(u)
E Be-p(olu)
log
p(p|u) p(w)
Buup(u)
[KL(p( [)ll
KL(p(1
1) [((])) PPppp))))Ppuulll)l (L u= 0)(pp())) P((u 0)
1 KKL (p(v) = 1> ( + KL pp(v) 0) |
JSD (p(u)u 1) 0))
JSD (w)
where p(v) = p(ol u = 1) P(u 1) + pl = 0) P(u = 0) pl (( +) and JSD is short for Jensen- densen-Shannon divergence, which is both non-negative and zero if and only if the two distributions are equal. Then Eqn. 25 can be rewritten
2
Ca(E) log JSD (p7(v))lp(())
To obtain the maximum of Ca(E), E should satisfy
(5( (v),
and the corresponding maximum value equals log 2. Thus, the relaxed objective CE(E) defined in Eqn. 12:
Ca(E) 2 (E, D = (E, min(C)(B),7} = min(log - JSD (p)(() .T}
achieves its global maximum if and only the concept embedding encoder satisfies:
JSD (v))l(()))) = log
(26)
Similarly, we can also obtain 1(2, u) = JSD (L)(())) For the i-th concept, of and of are first mapped 2 which is then used combine them into v, as follows:
0. = ao (+) + (1- 2
This indicates that v contains all the information of é, and Hwulv) = H (ulv,,) Thus, we have
T(u, a) H(u) H(ulv)
H(u) H(ulv,
= H(u) - H(u(2) + -
=I(C, u) + +
which is equivalent
JSD (pT (p) =
u(v,u()).
(27)
Plugging Eqn. 26 into Eqn. 27, we finally obtain
JSD(p(())lBE()) = log2 - I(v, ul2),
completing the proof.
As for the special case for the theorem above, T= log 2, it follows that JSD (07 (v) (v)) = 0, which implies I u. Thus, in Eqn. 27 the last term I(v, u/ē) and
JSD ()) log [(v, w
which is equivalent to
16

---

Concept- Lancept-Based Unsupervised Domain Adaptation
Lemma 4.2 (Optimal Predictor). Given the concepi embedding encoder E, the prediction loss C. F) has a tight lower bound
E,(E, F) & Es (L)(F((())) u) H(y) [(x)),
where H(-)-) denotes the conditional entropy. The optimal predictor F that minimizes the prediction loss is
(-(()))) [P (u E(2)))2
where y, denotes the i-th element of y.
Proof.
With fixed, the prediction loss Cp(E, can be rewritten as
(p E ((,p(F(E(x)) v)l
Yi log (F(B(=))
B(v.y)~p(v.s)
Yi log [F(e)]]
Evw((())
Byr ys log (F(
Ev
E [ log (F(e)):
1
where F(B())) = F()) E R and we denote the i-th component of F(v) (F(v)), Note that F(v) must satisfy the following constraints: (1) (F(()), 2 for all i {I...., Q), (F() Thus, minimizing the prediction loss F), w.r.t F is equivalent to solve the following constrained optimization problem:
Σ E [ys v) log(F(v));
max F(n)
s.t.
(P(u)):= (F(u)),2 0, i (1....,Q).
To solve this constrained problem, we first define the Lagrangian function:
(1
((F()),A,
Σ E [mll]] Mog(F(())) + A
E(P())))))
+ Σ pa (F(D)))
j=1
k=1
where À 2 0 and 2 0 for i E (l...., Q}. By the first-order Karush-Kunn KuuhhhKKhn--aaaa TucherTucker (KKT) conditions:
Q Σ (F(v)): = 0,
at Elydel
X + 14 = 0, i (1,...,Q),
aF, (F(D))
81 = (F(v)); 2 0 (1...... .......... dp.
P: 2 0, (I.....)),
#4 (F(u)); 0, ie (1....,)),
we can derive the optimal (F(v)); fori (1..... Q} as:
(F*())): = E. [ya [p] = P (v. = 1/0),
17

---

Concept-Basee Unsupervised Domain Adaptation
and
(+) = [P(s 1|0)| R
At that point, Lp(E, F) achieves its minimum value:
C, 18 (L y)]
Q
())p(())
E yoloo log (F-{
Q
Rump(v)
P(ys = 1/0) log Flu==1))
-==1
H(ylv) = Hy E()),
completing the proof.
Theorem 4.2 (Optimal Concept Embedding Encoder). Assuming u I y, if the concept embedding encoder E, concept probability encoder Eproto the predictor and the discriminator D have enough capacity and are trained reach optimum, any global optimal concept embedding encoder E and its corresponding global optimal concept probability encoder E goo have the following properties:
Eprra (2)) = P(c, 1|x)
(17)
H (y E (x)) = H(y z),
(18)
(F maxg' C2 (F').
(19)
Proof. We first prove the optimal concept probability encoder in Eqn. 17. Because Ex)ob(x)) (Epros(2))) R and Le is the average binary cross entropy:
Le (c(Eprob(z)..)
1 K
Σ a log (Epron(<e)) + (1 - c.) log i-((prov(2e));
then we have
Es (Lo(Eoro c)] = K
Σ
Es cs log (Eigro((a)), + (1 - c;) log (E(pproo())),
i=1
((anc))p(x,c) c log (Ferro((r)), (1 ci) log ((pprot(
K
EgnDs
E Eeppplasae)
[clog (Flres + - c;) log (- (x))
K
EnnDs E(c;(z) log (Epros(a)), + (1 - B(c(l=) log 1-(Eprob(20)
R
Thus, the optimal concept probability encoder (Ferot);; for E fI,..., K} should be
(Eprot)s = arg min Es [Le (Eproo (Epret)s
K
= arg min (Epret)i
EanDs
B(cy (r) log (Eprot(()) + (1 - Elcy (x)) log I- i-(Eproo
K
5=1
=arg En-Ds (Eproshi
[B(c((2)) log [Eras(o]], (1 B(ca(ar)) log I-(Epros()))
18

---

Concept-Based Unsupervised Domain Adaptation
For any (a, b) E (0, 0), the function y a log(1 y) + b log(y) achieves its maximum in [0, at _ Applying this result, we derive the optimal value of (Ferob (x))); for i E (1,...: : K} as:
(proo(()))) (c/aa)) = P(c, 1/x).
and the optimal (x) is given by
E (x)a [( (2)) (.))....... (Epros (x)) []] k|
(P(c = 1|x),... P(ck = 1
completing the proof for Eqn. 17.
Since (xx) a function of æ, by the data processing inequality, we have
HlylE(z)) 2 Hgy/z)
The objective function mentioned in Eqn. 16 has the following lower bound:
C(E) A H(y E(æ)) - XaCa(E)
2 (g x) A max Oo (E').
E
This equality holds if and only if H (y) E(x)) H (v x)) and (x( maxe Ōa (E'). Therefore, we only need to prove that the optimal value of C(B) is equal to H (y x) Aa maxg Co (E") in order to prove that any global encoder E" satisfies Eqn. 18, and Eqn. 19.
We show that C(F) can achieve its lower bound by considering the following encoder Fgg: Fa( (-a())r) (Zhao et al., 2017; Wang et al., 2020). It can be checked that H (y Eo(x)) H (y x)) and Eo(x) 1 u which leads to (Eo) maxg C2 (E"), completing the proof.
C. Experiments
C.1. Dataset Details
Waterbirds Datasets (Sagawa et al., 2019). First, we incorporate the concepts from the CUB (Wah et al., 2011) dataset into the original Waterbirds dataset to make it compatible with concept-based models. Since the original Waterbirds dataset a binary classification task (landbirds are always associated with land and waterbirds with water as source domain), we construct the target domain, Waterbirds- 2024) Waterbirds dataset), by selecting images with opposite attributes (e.g., landbirds in water and waterbirds on land). This results in 2 a binary classification domain adaptation dataset. Additionally, because the CUB dataset is inherently a multi-class classification task, we construct Waterbirds-200 by replacing the labels in the Waterbirds-2 dataset with the multi-class labels from CUB without modifying the data itself. Finally, as the CUB dataset represents a natural domain shift relative to Waterbirds-22 200, we use the CUB training data as the source domain and retain the Waterbirds-shift images as the target domain to construct Waterbirds-CUB.
(backgroud shift data, the same construct method as CONDA (Choi et al.,
MNIST Concepts. We selected 11 topology concepts [Ring, Line, Are, Corner, Top-Curve, Semicircles, Triangle, Bottom- Curve, Top-Line, Wedge, Bottom-Line] (initially generated by GPT-4 (Achiam al., 2023) and refined through manual screening) for the MNIST (LeCun et al., 1998), MNNST-M(Ganin et al., 2016). SVHN (Netzer et al., 2011), and USPS (Hull, 1994) digit datasets to evaluate the performance of our method. In addition, PCBMs (Yuksekgonul et al., 2022) can utilize the CLIP model (we tested with CLIP:RN50) (Radford et al., 2021) to automatically generate concepts. To evaluate its effectiveness, we compare the concepts generated by CLIP with our predefined set of concepts. However, since the PCBM --ggerrreet concepts are stored in a concept bank and lack explicit relationships between classes and concepts, they cannot be directly used to evaluate our model.
19

---

Concept-Based Unsupervised Domain Adaptation
Table Performance comparison across MNIST datasets using different concepts. The numbers 11 and represent the concepts generated by PCBMs through once and twice recursive exploration of the ConceptNet (Speer et al., 2017) graph.
Dataset MNIST MNIST-M
SVHN MINST
MNIST USPS
Concepts
11
13
Ours
11
13
Ours
11
13
Ours
PCBM
13
29.660211 18.
oa
11.35020.000
11. 11.350
21 21.32312.10
116
13.337=00188
14.11720.963
15.3420.15
CONDA
00754100000
9.754
9.754 oo0
9. 0.80010.000
9.80020.00
9.800
17.887=0.000
17.88740.000
17.88710.00
CUDA (Ours)
95.2 0.
12.49
96. 00.11
SKinCON Datasets (Daneshjou et al., 2022b). The SkinCON dataset is constructed using two existing datasets: Fitzpatrick 17k (Groh et al., 2021) and Diverse Dermatology Images (DDI) (Daneshjou et al., 2022a). Both datasets are publicly available for scientific, con-commercial use. Fitzpatrick 17k, which was scraped from online atlases, contains a higher level of noise compared to DDI, making domain adaptation on Fitzpatrick 17k more challenging. However, due to the small size of the DDI dataset, we exclusively use Fitzpatrick 17k while excluding non-skin images (those with unknown skin tone types labels not consider by SkinCON).
C.2. Experimental Details
Model and Optimization Details. We use ResNet-50 (He et al., 2016) for the Waterbirds dataset and ResNet-18 for the MNIST and SkinCON datasets. The hyperparameters are summarized in Table 5. All DA baselines, CBMs (Koh et al., 2020), and CEMs (Zarlenga et al., 2022) share the same backbone as approach for fair comparison. Zero-shot serves as the naive baseline for CONDA (Choi et al., 2024), where it uses the prompt "an image of [class]" to generate predictions. CONDA improves upon this by combining the zero-shot predictor with a linear-probing predictor to obtain pseudo-labels for the test batch, followed by test-time adaptation. Both PCBM and CONDA require pretrained models to construct the concept bank. Therefore, we utilize CLIP:ViT-L-IE (Radford et al., 2021) for the Waterbirds dataset consistent with CONDA, and CLIP:RN50 for the MNIST and SkinCON datasets. Our code will be available at https//gtthub.com/mmellaaaaat
Table 5. Haper-parameters of CUDA during training.
Leaning Rate
Weight Decay
A
Xa
Relax Threshold
Waterbirds-2
1e-3
4e-5
5
0.3
0.5
Waterbirds-200/CUB
1e-3
4e-5
5
0.3
0.7
MNIST + MMNISTMMUSSPS
1e-3
le-5
5
0.1
0.6
SVHN + MNIST
le-3
le-5
5
0.1
0.7
I-II + III-IV
1e-3
4e-5
10
0.1
0.3
IIIIIV + V-VI/IIII
1e-3
4e-5
10
0.1
0.7
Naive Baseline. DA methods and the concept-driven paradigm of CBMs cannot be naively combined. In our naive baseline, we extend the DA model by adding a linear layer to its feature output layer to predict concepts, incorporating the concept loss into the original DA loss. However, as shown in Table 6, this approach fails to effectively capture concept information and performs worse than the original CBMs method. This highlights that the standard DA structure is not inherently suited for learning concepts and fails to leverage the benefits of concepts improve domain alignment.
Lipschitz Continuity. Lemma 3.1 assumes that all hypotheses are I.-Lipschitz continuous for some constant J, > 0. While this assumption might seem restrictive at first glance, it is actually quite reasonable. In practice, hypotheses are often implemented using neural networks (e.g., our label predictor), where the fundamental components such as linear layers and activation functions are naturally Lipschitz continuous. Therefore, this assumption is not overly strong and is typically satisfied (Shen et al., 2018).
C.3. Framework Details and Training Algorithm
Our adversarial training process consists of two main steps. First, we optimize the domain discriminator using the original discriminator loss (Eqn. 8) and then calculate the relaxed discriminator loss (Eqn. 9). Second, we optimize the concept
20

---

Concept-Basee Unsupervised Domain Adaptation
Table Performance concept-hased methods both concept learning and classification indicates without RandIut. Naive refers our naive combination baseline. We mark the best result with bold face and the second best results with underline. Average accuracy calculated over every three datasets of the same type images.
different datasets. CEM (w/o R.)
Datasets
Waterbirds-2
Waterbirds-200
Waterbirds-CUn
AVG
Metrics
Concept
Concept
Class
Concept
Concept
Concept
Concept
Class
ACC
CEM
94. 94.14
81.74:0.39
1.0
93.68
81 21.222..0c
62:26
93.64420.06
80.08
66.48
66.34
CEM (w/o R.)
94.17+0.14
$1.9640.30
69.15+2.15
93. 76+0.20
$1.04
63.56+1.25
93, 6040.0
79.80
65.89=0.51
66.30
CBM
93.60
83.89+0.40
74.81+2.16
93.50+0.16
83, $3.14
63.89
93, 03..14
82. 10=0.0.
63.89
Naive
85.41.0.11
71.8620.19
66.8332.00
88, 20...0
73.96
63.51
88.1140.05
73.56.20.0
60.72s0.27
63.69
CUDA (Ours)
94.63:20.0
84.97
9..90
95.15100.0
85.06-0.19
10.31
94.58:0.07
82.81 20.19
74.66
81.15
embedding encoder and label predictor. The overall framework is illustrated in Fig. 5, and the detailed training process is outlined in Algorithm The objective to learn both labels and concepts in the target domain, given source and target domain images as input. The training procedure alternates between Equ. 4 and 5 with adversarial training using Eqn. 6~0,, During inference, predict the target domain class label F ((E)))) and concepts é Eppob((:)). The code will be released upon the acceptance of this work.
Scunce Image
Festure Embeddiing
Cencept Predictions [Ri]-a
limbeskinge [My]e
Label Predictor
Feature Extractor
Nevral Network Gronogg
Domain Classifier
Domain Label
Discriminator
Tirget Image X Image
Embeeding Encodar
du1 (1 b)p
Concapt Predictions é= [Ki]ea
Concapt libeeklngs - [M,]feu
Embeeddings
Label Prodictor F
Class Label
Neural Network Geonogg
Extractor
Domiin Classifier D
Domain Label
Relaxed Discriminator Loss Z,
Tirget Image x,
Concept Embolking Enoader
dal"") +11 B))
Figure 5. The full CUDA framework. It processes source and target domain images to learn feature embeddings, from which positive ef and negative wf embeddings are derived. These embeddings are passed through Geoncege to compute concept predictions ē and construct final concept embeddings v. Adversarial training alternates between optimizing the domain classifier (discriminator) with Eqn. and optimizing the concept embedding encoder and label predictor with Eqn. 5, guided by adversarial training using Eqn. 6~9.
D. Limitations and Future Works
Our approach falls short of the state-of-the-art UDA method GH++ (Huang et al., 2024) on the Waterbirds 200 classification task. This may be attributed to GH++'s use of gradient harmonization, which balances the classification and domain alignment tasks, particularly benefiting scenarios with a large number of categories. Exploring how to leverage gradient harmonization to balance concept learning in our framework an interesting direction for future work. We plan to
21

---

Concept-Basee Unsupervised Domain Adaptation
investigate the related theoretical foundations and explore how it be effectively integrated into method in future work. Additionally, our approach achieves competitive results and stable performance without using the most advanced DA backbone. We believe that plugging our method into more sophisticated backbone could lead to more remarkable performance, which we leave future work. Lastly, the domain shift studied in this work primarily involves shifts related to background or other label-agnostic factors. In the future, we aim to extend our method to address other types of domain shifts, broadening its applicability to other scenarios.
Algorithm Pseudocode CUDA Training
Input: Source domain data S ((r), yi, <))==. target domain data T
{æf مززز { feature extractor 4 concept
embedding generator concept probability network Geoncopt. label predictor F, domain discriminator D, learning rates 01 02, concept loss weight λe, domain discriminator loss weight Ad, concept number K" relaxation threshold т. Predicted target labels and concepts (Dr, C.
Output:
1: while not converged do
2: Sample minibatches X₂ C S and Xg C T.
3: for each domain d E {s, 1) (source or targel) do
4: Extract feature embeddings: 2d - ((xa).
5: for k = 1 to do
6: Generate positive and negative concept embeddings: (o 0/22l
+ px((z).
7:
Predict concept probabilities:
Cd.
+ Gooncept ((v d,k 1)
(+)
8: Combine positive and negative embeddings:
Ud.k + Ed, dd (1 Car) di
(-)
9:
end for
10:
end for
Predict source class labels: Ø. F(Va).
Predict domain labels: ü, f D(v.),, Ue D(v.).
13:
Compute Cp(Us: Us) based Eqn. 6.
14:
Compute (C,,., C.) based on Eqn. 7.
15:
Compute Ca(ue, uo). E {s, based on Eqn. 8.
16:
Relax the domain discriminator loss to get La based on Eqn. 9.
17:
Crotal Lp Ach.- XaCa
18:
Update D to minimize La with learning rate a1-
19:
Update &, Geoncept, and F to minimize Liotal with learning rate a2.
20:
end while
return (ge, }
22


Reviews:
[{"review_id": "xiK7vSGLV0", "reviewer": "Reviewer_tPTT", "summary": "This paper aims to tackle the problem of limited generalization of concept bottleneck models in cross-domain scenarios. It utilizes adversarial training to align the cross-domain concept embeddings and introduces a relaxed uniform alignment technique to alleviate the influence of over-restricted domain alignment assumptions. Experiments on diverse datasets validate its effectiveness.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. We are glad that you found our theory ``\"extensive\"``/``\"correct\"``, our claim ``\"convincing\"``, our experimental designs/analyses ``\"valid\"``, and the paper ``\"well-written\"``. Below, we address your questions one by one. \n\n\n**W1 & Q1. Provide more concrete examples to demonstrate concept distribution differences between source and target domains.**\n\n+ One example is the concept \"Primary Color: Brown\", where the positive rates are 19% and 17% for the source and target domains, respectively. Additional examples include \"Black Bill\" (48% in source / 51% in target) and \"All-purpose Bill\" (40% in source / 41% in target).\n\n+ Concept distribution differences are indeed a common problem in domain adaptation (DA) datasets with concept annotations. By directly counting the dataset statistics, we observe such discrepancies frequently arise across diverse DA scenarios.\n\n\n**W2 & Q2. Include experimental results on more datasets and compare with additional UDA methods (e.g., [1-3]).**\n\nThis is a good question. \n\n**Common Datasets for Concept Bottleneck Models:** Since our CUDA belongs to the category of concept bottleneck models (CBMs), we follow the literature (e.g., CBMs [1], CEMs [2], and PCBMs [3]) to use common datasets such as CUB, MNIST, and SkinCON in our experiments, as they provide rich concept annotations essential for evaluating concept-based methods.\n\n**Mainstream Datasets for DA Are Not Applicable:** Note that mainstream datasets for DA such as Office-31, Office-home, VisDA-2017 are not applicable because they do not contain concept annotations, which are needed to evaluate the correctness of concept prediction in all methods. \n\n\n\n[1] Koh et al., ICML'20. [2] Zarlenga et al., NIPS'22. [3] Yuksekgonul et al., ICLR'22.\n\n**Additional Datasets:** Inspired by your comments, we add a new dataset: AwA2-ImageNet [4], where AwA2 is the source domain containing 85 concepts for animals and ImageNet is the target domain. We match the AwA2 classes with corresponding subsets of ImageNet and filter overlapping data.  Furthermore, we perform style transfer on the target domain (ImageNet) to induce domain shift, making the task more challenging. The table below summarizes the results, which demonstrate our CUDA can improve performance even in the case of larger domain gaps and more diverse data. \n\n\n| Metrics | Concept Accuracy |    Concept F1    |   Class Accuracy   |\n| :-----: | :--------: | :--------: | :--------: |\n|   CBM   | 80.15±0.09 | 59.25±0.64 | 33.89±1.12 |\n|   CEM   | 80.63±0.37 | 59.17±0.19 | 33.73±2.81 |\n|  GH++   |     -      |     -      | 54.83±0.23 |\n|  CUDA   | 86.84±0.30 | 65.60±0.26 | 55.30±0.11 |\n\n\n[4] Deep learning face attributes in the wild, CVPR'15.\n\n**Comparison with UDA Methods [1-3]:** Following your suggestion, we tested [5] and [6] on our nine datasets, and the results are summarized in the table below. However, [7] relies on a ViT backbone, making it unsuitable for fair comparison with our results, which use ResNet50. We agree that integrating transformer-based methods into our framework is a promising direction for future work. We will include these references in our paper. Thank you for this insightful suggestion!\n\n| Datasets |    WB-2    |   WB-200   |   WB-CUB   |  AVG  |   M → M-M  |    S → M   |    M → U   |  AVG  | I-II → III-IV | III-IV → V-VI | III-IV → I-II |  AVG  |\n|:------:|:----------:|:----------:|:----------:|:-----:|:----------:|:----------:|:----------:|:-----:|:-------------:|:-------------:|:-------------:|:-----:|\n|   GVB  | 40.97±3.78 | 74.89±0.69 | 73.66±0.35 | 63.17 | 41.28±3.55 | 81.21±0.36 | 94.27±1.64 | 72.25 |   76.35±0.73  |   78.3±1.43   |   74.32±0.1   | 76.32 |\n|   DWL  | 54.86±2.07 | 73.64±0.38 | 72.31±0.52 | 66.94 | 50.69±3.25 | 83.97±2.19 | 96.29±0.13 | 76.98 |   74.98±0.72  |   79.24±0.83  |   73.97±0.97  | 76.06 |\n|  CUDA  | 92.90±0.31 | 75.87±0.31 | 74.66±0.19 | 81.15 | 95.24±0.13 | 82.49±0.27 | 96.01±0.13 | 91.25 |   78.85±0.31  |   80.58±0.72  |   76.53±0.49  | 78.65 |\n\n[5] Gradually vanishing bridge for adversarial domain adaptation, CVPR20.\n\n[6] Dynamic weighted learning for unsupervised domain adaptation, CVPR21. \n\n[7] Safe self-refinement for transformer-based domain adaptation, CVPR22.", "claims_and_evidence": "Yes, the author has validated his hypotheses and claims through clear and convincing evidence, including extensive theoretical derivation and proof, as well as experimental results.", "methods_and_evaluation_criteria": "Yes, the methods CUDA and evaluation criteria for concept and class prediction make sense for the problem.", "theoretical_claims": "I check the correctness of the proof of Optimal Discriminator(Lemma 4.1, B.2 in supplementary material), I believe this proof is correct and there are no issues with it.", "experimental_designs_or_analyses": "The experimental section shows the comparison with other CBM and DA methods, by which I check the validity of the designs and analyses.", "relation_to_broader_scientific_literature": "The introduced adversarial training and relaxed uniform alignment technique may be extended toward multi-class classification scenarios, but the author leaves it for future research.", "essential_references_not_discussed": "To my knowledge, the paper includes all the necessary related works essential for its understanding with discussion. But there are several UDA methods are not included in the experiment section for quantitative comparison, such as [1-3] listed below.", "other_strengths_and_weaknesses": "1.\tFor the over-constraints for domain distributions alignment, can the author provide some concrete examples for that, source and target concepts indeed have minor distribution differences?\n2.\tIn experiments, the author includes several datasets including bird, digital and medical scenarios. But several mainstream datasets for DA are not considered, such as Office-31, Office-home, VisDA-2017, etc.", "other_comments_or_suggestions": "The paper is well-written and easy to follow, I have no further comments here.", "questions_for_authors": "1.\tThe author is suggested to provide more concrete examples to convince me the existence of concept distribution differences between source and target domains, and such discrepancy is a common problem in diverse DA scenarios, to make the questions this paper claimed for DA task more valuable.\n2.\tThe author should include the experimental results on more datasets as mentioned in weakness, and compare with more UDA methods such as [1-3].\n\n\n[1] Gradually vanishing bridge for adversarial domain adaptation, CVPR20\n[2] Dynamic weighted learning for unsupervised domain adaptation, CVPR21\n[3] Safe self-refinement for transformer-based domain adaptation, CVPR22", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V4Gb8tVjGp", "reviewer": "Reviewer_1Zod", "summary": "This paper introduces a framework to handle domain shifts effectively while maintaining interpretability. It leverages adversarial training and a relaxed alignment mechanism to align concept embeddings across source and target domains, allowing for flexibility in capturing domain-specific variations. It also eliminates the need for labeled concept data in the target domain, enabling it to operate in unsupervised settings. The framework is supported by theoretical guarantees, including a generalization error bound and analysis of relaxed alignment, and achieves state-of-the-art performance in improving both concept accuracy and classification accuracy. Experiments on benchmark datasets demonstrate its significant improvements.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. We are glad that you found our method ``\"novel and impactful\"``, our theoretical guarantees ``\"rigorous and mathematically sound\"``, and the experiments ``\"comprehensive\"`` and ``\"applicable across a wide range of scenarios\"``. Below, we address your comments one by one.\n\n\n**W1. The paper lacks visual or qualitative examples to illustrate how relaxed alignment impacts interpretability.**\n\nThanks for mentioning this. Following your suggestion, we provide a more concrete example to illustrate how relaxed alignment impacts interpretability. We will also include this example into our revised version.\n\nWe consider the task of making predictions for a target-domain image of a Black-Footed Albatross. The table below summarizes the predictions of different methods for both concept predictions and class label predictions; note that \"CUDA w/o Relax\" means \"CUDA w/o Relaxed Alignment\". Each row corresponds to one method, while the columns represent the predictions for specific concepts and class labels.\n\n\n+ **Concept Predictions:** The first three columns show predictions for specific concepts (e.g., Seabird Bill, Black Nepe, Solid Breast). The values \"0\" and \"1\" indicate whether the prediction is incorrect or correct, respectively. The ground-truth distribution (GT) of each concept (in terms of GT positive rates) in the source and target domains is provided in parentheses (\"GT Ratio: Source / Target\").\n\n+ **Class Label Predictions:** The last two columns show the predicted and ground-truth labels indices for each method.\n\n\n|      Method      | Seabird Bill (6.10% / 5.90%) | Black Nepe (48.59% / 50.69%) | Solid Breast (64.72% / 64.55%) | Predicted Label Index | Ground-Truth Label Index |\n| :--------------: | :--------------------------: | :--------------------------: | :----------------------------: | :-------------------: | :----------------------: |\n|       CBM        |              0               |              0               |               1                |          24           |            0             |\n|       CEM        |              1               |              0               |               1                |           0           |            0             |\n| CUDA (w/o Relax) |              0               |              1               |               1                |           0           |            0             |\n|     **CUDA**     |              1               |              1               |               1                |           0           |            0             |\n\n\n\nWe can see that:\n\n+ CBM predicts incorrect concepts, resulting in an incorrect class label prediction.\n+ CEM and CUDA (w/o Relax.) predict the correct class label but fail to predict some concepts correctly, which harms interpretability.\n+ Our full method (CUDA with Relaxed Alignment) predicts both the concepts and the class label correctly in the target domain, demonstrating that it achieves both interpretability and performance simultaneously.\n+ Interestingly, the table shows that concepts with clear distribution differences (e.g., \"Seabird Bill\" and \"Black Nepe\") lead to incorrect concept predictions in models without the relaxed alignment mechanism (e.g., CUDA w/o Relax. and CBM predict \"Seabird Bill\" incorrectly). This further supports the importance of our relaxed alignment.\n\n\n**Q1. Can the proposed method scale to larger datasets or more complex domains? Have you tested this?**\n\nYes, our method can scale to larger datasets and more complex domains. To demonstrate this, we followed your suggestion to conduct additional experiments on ImageNet. In particular, we include a new dataset: AwA2-ImageNet [4], where AwA2 is the source domain containing 85 concepts for animals, and ImageNet is the target domain. We match the AwA2 classes with corresponding subsets of ImageNet and filter overlapping data. Furthermore, we perform style transfer on the target domain (ImageNet) to induce domain shift, making the task more challenging. The table below summarizes the results, which \n+ demonstrate our CUDA can improve performance even in the case of larger domain gaps and more diverse data and\n+ highlight the scalability of our approach while maintaining its performance and interpretability in larger and more complex domains.\n\n\n| Metrics | Concept Accuracy |    Concept F1    |   Class Accuracy   |\n| :-----: | :--------: | :--------: | :--------: |\n|   CBM   | 80.15±0.09 | 59.25±0.64 | 33.89±1.12 |\n|   CEM   | 80.63±0.37 | 59.17±0.19 | 33.73±2.81 |\n|  GH++   |     -      |     -      | 54.83±0.23 |\n|  CUDA   | 86.84±0.30 | 65.60±0.26 | 55.30±0.11 |\n\n\n[4] Deep learning face attributes in the wild, CVPR15.", "claims_and_evidence": "1. Theoretical analysis and experimental results show that relaxed alignment achieves better concept distribution matching and higher target accuracy.\n2. Self-supervised learning and adversarial training are used to infer concepts in the target domain without explicit labels, validated by experiments across multiple datasets.\n3. Figure 3 shows that the proposed method achieves better performance with higher intervention ratios compared to baseline methods.", "methods_and_evaluation_criteria": "1. The proposed methods are well-tailored to address domain adaptation and concept learning challenges.\n2. The use of diverse datasets strengthens the evaluation’s comprehensiveness.", "theoretical_claims": "The proofs for Theorem 3.1 and Theorem 4.1 were reviewed. They are rigorous and mathematically sound.", "experimental_designs_or_analyses": "1. The experiments cover a wide range of datasets and scenarios, including background shifts and medical imaging, which enhance the framework’s applicability.", "relation_to_broader_scientific_literature": "The paper builds upon works like CBMs, CEMs and DANN. It extends these by integrating interpretability into domain adaptation.", "essential_references_not_discussed": "No critical omissions", "other_strengths_and_weaknesses": "Strengths:\n1. The introduction of relaxed alignment for concept-based domain adaptation is novel and impactful.\n2. The method addresses a critical gap in interpretable domain adaptation, enhancing both accuracy and interpretability.\n3. The paper is well-organized.\n\nWeakness:\n1. The paper lacks visual or qualitative examples to illustrate how relaxed alignment impacts interpretability.", "other_comments_or_suggestions": "None", "questions_for_authors": "Can the proposed method scale to larger datasets or more complex domains? Have you tested this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wFdao6CLa5", "reviewer": "Reviewer_sn2y", "summary": "This paper aims to improve the generalizability and transferability of Concept Bottleneck Models (CBMs) by proposing a novel Concept-based Unsupervised Adaptation (CUDA) framework. The CUDA framework is designed to align concepts across domains in an adversarial manner while introducing a relaxation threshold. The authors evaluated the proposed framework across three sets of domain adaptation datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable comments. We are glad that you found our framework ``\"novel\"``, our idea ``\"interesting\"``, and our proofs ``\"correct\"``. We address your comments in turn below.\n\n**Q1. ... relationship between the target domain error ... CBM (Formula 3) and the original target domain error in feature space ...?**\n\nTo see the relationship between Formula 3 and the original target domain error in feature space, we start with the target error defined in the feature space (denoted as $\\\\mathcal{Z}$):\n\n**Without concept learning**, we have the standard domain adaptation bound as (similar to Formula 1):\n\n$$\n\\\\epsilon_{T}^{\\\\boldsymbol{z}}(h) \\\\leq \\\\epsilon^{\\\\boldsymbol{z}}_{S}(h) + \\\\frac{1}{2} d _{\\\\mathcal{H} \\Delta \\\\mathcal{H}}(\\\\mathcal{D}^{\\\\boldsymbol{z}} _S, \\\\mathcal{D}^{\\\\boldsymbol{z}} _T) + \\\\eta^{\\\\boldsymbol{z}},\n$$\n\nwhere $\\mathcal{D}_S^{\\boldsymbol{z}}$ and  $\\mathcal{D}_T^{\\boldsymbol{z}}$ are denoted as the induced marginalized distributions in $\\mathcal{Z}$, and all terms are defined in this space.\n\n**With concept learning**, the bound above can be refined to include complex transformation error terms (resembling Formula 3):\n\n$$\n\\\\epsilon\\_T^{\\\\boldsymbol{z}}(h) \\\\leq \\\\epsilon^{\\\\boldsymbol{z}} _S(h) + \\\\frac{1}{2} d _{\\\\mathcal{H}\\\\Delta\\\\mathcal{H}}(\\\\mathcal{D}^{\\\\boldsymbol{z}} _S, \\\\mathcal{D}^{\\\\boldsymbol{z}} _T) + ( const \\times \\\\mathbb{E} _S [ || \\\\hat{\\\\boldsymbol{c}} - \\\\boldsymbol{c}|| ] + error _{other}) + \\\\eta^{\\\\boldsymbol{z}},\n$$\n\nwhere $error _{other}$ contains additional losses during the transformation from $\\\\mathcal{Z}$ to $\\\\mathcal{V}$ (e.g., projection errors). If $error _{other}$ is spread and absorbed into $\\\\epsilon _S^{\\\\boldsymbol{z}}(h)$, $d _{\\\\mathcal{H} \\\\Delta \\\\mathcal{H}}( \\\\mathcal{D} _S^{\\\\boldsymbol{z}}, \\\\mathcal{D} _T^{\\\\boldsymbol{z}})$, and $\\\\eta^{\\\\boldsymbol{z}}$, the resulting bound becomes structurally equivalent to Formula 3 (copied below):\n\n$$\n\\\\epsilon _{T}(h)  \\\\leq \\\\epsilon _{S}^{\\\\boldsymbol{c}}(h) + \\\\frac{1}{2} d _{\\\\mathcal{H} \\\\Delta \\\\mathcal{H}}(\\\\tilde{\\\\mathcal{D}}^{\\\\boldsymbol{c}}_S, \\\\tilde{\\\\mathcal{D}} _T)  + R \\\\cdot \\\\mathbb{E} _{S} [ || \\\\hat{\\\\boldsymbol{c}} - \\\\boldsymbol{c} ||] + \\\\eta^{\\\\boldsymbol{c}}.\n$$\n\nThus, Formula 3 can be viewed as a refinement of the feature-space bound, making concept prediction error explicit and improving interpretability.\n\n**Q2. ... widely used DA benchmarks ... against more recent UDA methods ...**\n\nGood question.\n\n**Mainstream DA Datasets Are Not Applicable:** Mainstream DA datasets for DA such as Office-31, Office-home, VisDA-2017 are not applicable to our setting because they do not contain concept annotations, which are needed to evaluate the correctness of concept prediction (and interpretation) in all methods.\n\n**Additional Datasets:** As you suggested, we add a new dataset: AwA2-ImageNet, where AwA2 [Liu et al. 2015] is the source domain containing 85 concepts for animals, and ImageNet is the target domain. We match the AwA2 classes with corresponding subsets of ImageNet and filter overlapping data. We further perform style transfer on ImageNet to induce domain shift, making the task more challenging. The table below summarizes the results, which demonstrate our CUDA outperforms existing CBM variants and DA methods even in the case of larger domain gaps and more diverse data.\n\n|Metrics|Concept Accuracy|Concept F1|Class Accuracy|\n| :-----: | :--------: | :--------: | :--------: |\n|CBM|80.15±0.09|59.25±0.64|33.89±1.12|\n|CEM|80.63±0.37|59.17±0.19|33.73±2.81|\n|GH++|-|-|54.83±0.23|\n|CUDA|86.84±0.30| 65.60±0.26 |55.30±0.11|\n\n**More Recent UDA methods.** We did compare CUDA with GH++, a more recent, state-of-the-art UDA method (PAMI '24) that outperforms DANN and MCD. Our results show CUDA\n+ enables concept interpretation while GH++ cannot, and\n+ achieves competitive prediction performance.\nWe also follow your suggestion to include more DA baselines, GVB (CVPR '20) and DWL (CVPR '21); see our **response to Q2&W2 for Reviewer tPTT**.\n\n**Q3. ... why CUDA stood out much more in WB-2 than other WB related datasets?**\n\n**Poor Performance of DA Methods on WB-2.** DA methods, such as GH++, rely heavily on class label information during training, without access to concept annotation. WB-2 has only binary labels, which contain limited information; it therefore leads to worse performance for DA methods.\n\n**Large Improvement from CUDA in WB-2.** In contrast, CUDA utilizes concept learning and relaxed alignment, effectively leveraging the 112 concepts for binary classification in WB-2. \n\n**Smaller Improvement in WB-200.** However, when applied to WB-200 or WB-CUB, it is more challenging for 112 concepts to represent 200 classes, leading to CUDA's smaller relative advantage against baselines. \n\n**Q4. Code reproducibility.**\n\nThank you for your interest. We have finished cleaning up the source code and will release it for reproducibility.", "claims_and_evidence": "While the authors have put in good effort in proving the generalization error bound for CBMs with the concept terms and also the optimality of the designed CUDA framework with the relaxed discriminator loss, it can be observed that the proves are discussed within the concept space which is assumed to be a separate space than that of the original feature space. This allows Theorem 3.1 over the target-domain error to be defined directly for the source and target concept spaces. However, the ceoncept embeddings originates from the feature embeddings which is part of the concept embedding encoder. Could the authors explain the relationship between the target domain error that is bound for the CBM (Formula 3) and the original target domain error (changing $\\tilde{D}_S$ and $\\tilde{D}_T$ in Formula 1 with $D_S$ and $D_T$)? That would help to understand how different the target error will be in the context of feature embedding and concept embedding.", "methods_and_evaluation_criteria": "The authors evaluate the methds against a number of CBMs and UDA methods on three sets of benchmarks. However, some widely used DA benchmarks, including DomainNet, VisDA-2017, and Office-31 (or Office-Home/Office-Caltech) have not been leveraged. Therefore while the efficacy of CUDA is indeed proven against prior CBMs and some very prior UDA methods (DANN, MCD), its efficacy in more comprehensive benchmarks against more recent UDA methods are not sufficiently proven.", "theoretical_claims": "The proofs provided are mostly detailed and correct.", "experimental_designs_or_analyses": "One outstanding result that can be observed is the efficacy of CUDA on WB-2, where the second best result is an astonishingly 20+ percent lower than that of CUDA while the majority methods (even methods such as GH++ which performs well on WB-200 and WB-CUB) scores lower than 50, which indicates the performance to be worse than even a wild guess. Could the authors explain the huge gap between the results and why CUDA stood out much more in WB-2 than other WB related datasets?", "relation_to_broader_scientific_literature": "The paper touches upon domain adaptation in the context of CBM, which relates to the domain adaptation in the context of feature spaces.", "essential_references_not_discussed": "Not available (to the best of my knowledge)", "other_strengths_and_weaknesses": "Overall, the paper is well written with comprehensive proofs and sufficient results. The idea of combining DA with CBM is interesting and could potentially be leveraged for improving both the generalizability and interpretability of current methods. However, there are some concerns over the assumptions that leads to the computation of the error upper bound with the concept embedding, and the evaluation benchmarks, highlighted as above.", "other_comments_or_suggestions": "Not available, see above for details.", "questions_for_authors": "Not available, see above for details.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper makes a valuable and timely contribution by connecting concept-based interpretability with domain adaptation—a direction that has been underexplored. The relaxed alignment mechanism is well-articulated, and both theoretical and empirical sections are solid. Reviewers raised valid concerns about experimental breadth and interpretability demonstrations, but these were mostly addressed during the rebuttal. On balance, this work appears to be a step forward in interpretable domain adaptation, with sound methodology and promising results. While some extensions could enhance the paper further, its core ideas are clear, technically correct, and of interest to the ICML community. I recommend acceptance.
Decision: Accept (poster)