Bayesian Domain Adaptation with Gaussian Mixture Domain-Indexing
Authors: Yanfang Ling
Jiyong Li
Lingbo Li
Shangsong Liang
Venue: NeurIPS 2024 poster (2024)
Keywords: domain adaptation
dynamic Gaussian mixture model
structural variational inference
Primary area: probabilistic_methods
TL;DR: A Bayesian Domain Adaptation with Gaussian Mixture Domain-Indexing algorithm aims to generating domain indexes with stronger interpretability.

Abstract: Recent methods are proposed to improve  performance of domain adaptation by inferring domain index under an adversarial variational bayesian framework, where domain index is unavailable. 
However, existing methods typically assume that the global domain indices are sampled from a vanilla gaussian prior, overlooking the inherent structures among different domains.
To address this challenge, we propose a Bayesian Domain Adaptation with Gaussian Mixture Domain-Indexing(GMDI) algorithm. 
GMDI employs a Gaussian Mixture Model for domain indices, with the number of component distributions in the "domain-themes'' space adaptively determined by a Chinese Restaurant Process. 
By dynamically adjusting the mixtures at the domain indices level, GMDI significantly improves domain adaptation performance. 
Our theoretical analysis demonstrates that GMDI achieves a more stringent evidence lower bound, closer to the log-likelihood. 
For classification, GMDI outperforms all approaches, and surpasses the state-of-the-art method, VDI, by up to 3.4%, reaching 99.3%. 
For regression, GMDI reduces MSE by up to 21% (from 3.160 to 2.493), achieving the lowest errors among all methods.

Bayesian Domain Adaptation with Gaussian Mixture Domain-Indexing
Yanfang Ling Sun Yan-sen University
Jiyong Li Sun Yat-sen University 11jy3730mai12, sysu. edu. en
lingyf3@mail2.. sysu. edu.
Lingbo
Shangsong Liang"
InfMind Technology Ltd lingbo@infmind.w. ai
Sun Yat-sen University
1 llangshangsong@@ail.ccomm com
Abstract
Recent methods are proposed to improve performance of domain adaptation by inferring domain index under an adversarial variational bayesian framework, where domain index is unavailable. However, existing methods typically assume that the global domain indices are sampled from a vanilla gaussian prior, overlooking the inherent structures among different domains. To address this challenge, propose a Bayesian Domain Adaptation with Gaussian Mixture Domain- Pomain-IIndexin(( Indexing(GMDD) algorithm. GMDI employs Gaussian Mixture Model for domain indices, with the number of component distributions in the "domain- -hemmes" space adaptively determined by Chinese Restaurant Process. By dynamically adjusting the mix- tures at the domain indices level, GMDI significantly improves domain adaptation performance. Our theoretical analysis demonstrates that GMDI achieves stringent evidence lower bound, closer to the log-likelihood. For classification, GMDI outperforms all approaches, and surpasses the state-of-the-an method, VDI, by to 3.4%, reaching 99.3%. For regression, GMDI reduces MSE by up to 21 % (from 3. 360 to 2.493), achieving the lowest errors among all methods. Source code is publicly available from //gtts:///ittee thub. com wwwccommmiiii commmingyf3/GMDI
Introduction
Machine learning models often suffer from performance degradation when applied to new domains
that differ from their training domains, phenomenon known as domain shift [21 8 28 15) Domain Adaptation (DA) invariant features, thereby enhancing generalization from source to target domains (3 12 33 45
6 37 32 38 [3] seeks to mitigate this issue by producing domain-
Recent research has explored the use of domain identity and domain index improve domain- invariant data encoding and enhance domain adaptation performance [36 40 Domain iden- rity 41 one-hot discrete variable vector, differentiates between domains, whereas domain index 41. real-valued continuous variable vector, captures domain semantics. Due to the limited information the discrete domain identity vector, research has increasingly focused on the domain index. Current approaches to incorporating domain index in domain adapta- tion include: (1) Directly using existing additional information the dataset as the domain in- dex [36] 40), which is impractical for datasets lacking such indices (22 29), and (2) Treating the domain index as a latent variable to be inferred 226 41]. However, these methods typically
*Coreesonningg author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

model the domain indices with a simple Gaussian distribution, limiting the domain indices space and thus hindering adaptation diverse target domains, resulting in suboptimal performance.
To address the aforementioned issues, we propose Bayesian Domain Adaptation with Gaussian Mixture lndxxig (GMDI) algorithm. The proposed ad- versarial Bayesian algorithm assumes that domain indices follow mixture of Gaussian distributions, with the num- ber of mixture components dynamically determined by Chinese Restaurant Process. As shown in Figure [ a single Gaussian distribution struggles to adequately fit the domain indices, neglecting the inherent structures among different domains. This observation motivates us to model domain indices from different domains collectively as a Gaussian mixture distribution. To the best of our knowl- edge, we are the first to model domain indices as a mixture of Gaussian distributions to address the aforementioned challenges. Inspired by [3, the latent space the mixture is defined as the "domain theiiintteees space. The mixtures of distributions provide higher level of flexibility in larger latent space, thereby increasing the capability to adapt to various target domains with domain shift. Our theoretical analysis demonstrates that GMDI achieves more rigorous evidence lower bound, and that maximizing this bound along with adversarial loss effectively infers optimal domain indices. Extensive experimental results validate the significant effectiveness of GMDI.
p(oy
aingle Caustion
meatue Coursoos
-.- groand-truth daman inciusa
Illustration of domain indices modeled by different distributions.
Figure 1:
Our key contributions are summarized as: (1) Our proposed GMDI is the first one to consider the entire distributions of domain indices in the "domain-themes space following mixture of Gaussian distributions, and dynamically determining the number of components the mixture with the Chinese Restaurant Process. (2) Our detailed theoretical analysis demonstrates that training with GMDI's superior evidence lower bound together with adversarial loss can yield optimal and more interpretable domain indices.
(3) Extensive experiments on classification and regression tasks showcase the strong domain index modeling capability of GMDI, significantly outperforming the state-of-the-ane
2 Related Work
Adversarial domain adaptation. There exists a substantial body of work on domain adapta- tion [4] 44 49 42
32 38). They focus generating domain-invariant data encoding by aligning the distributions of source and target domains to adapt to target domains. This alignment
37
achieved by directly matching the statistics of distributions [25] 24 or by employing adversarial loss [1]. which encourages domain confusion through adversarial objective with a discriminator. Adversarial domain adaptation is widely used due to its integration with deep learning, strong theo- retical foundation [7]. and superior performance. Various different types of ad adversarial losses have been explored: (35 uses an inverted label GAN loss, (5) utilizes a minimax loss, and 34 employs a cross-entropy loss against the uniform distribution. Typically, the discriminators in these models rely on domain identity, which contains limited information, to align data encoding distributions. 20 and
10 also pay attention to domain identity. Our work, however, focuses on domain index, providing more detailed representation of domains.
Domain adaptation related to domain indices. Recently, there has been growing interest in using continuous domain index, which contain richer and more interpretable information, to enhance domain adaptation performance. (36) use the rotation angle of images as the domain index for the Rotating MNIST dataset and patients' ages as the domain index for Healthcare Datasets. Their theoretical analysis demonstrates the value of utilizing domain indices to generate domain-invariant features. (40) employ graph node embeddings as domain indices to achieve domain adaptation in graph-relational domains. These methods assume that domain indices are available. However, in practice, domain indices are not always accessible 22 29 26 generates features representing the similarity between different domains but do not formally define the domain index. [1] formally
define the domain index and treat as a latent variable to be inferred. Although 41 takes steps
towards Bayesian approximation to parameter distributions, it only assumes single domain index

---

distribution, limiting its capability to adapt to diverse target domains effectively. In contrast, we address this issue by representing the domain index with a dynamically updated mixture model.
3 Background
3.1 Problem setup
We aim at unsupervised domain adaptation: given N domains with different domain shifts, each domain has a domain identity w E W = [N] & [I...,NN] N. and each domain contains Dw data points. Similar to the conventional unsupervised domain adaptation setting, the .V domains are divided into source domains with labeled data D> {( V², }=)) and target domains with unlabeled data DT { ) A foundational element that builds up our research problem is the diverse domain shifts [14] between different target domains and source domains. For source domains, the complexity of each target domain varies, which motivates us to dynamically infer domain indices the "domain-themes" space and model them with dynamic Gaussian Mixture Model. We aim to (1) predict the label J of target domain data, and (2) infer local domain index uw R.B and global domain index Ow E IR in the dynamic "domain-themes" space. The summary of the notations is presented Appendix J
3.2 Preliminary
Domain index. The domain index, distinct from domain identity w, represents domain semantics, thus empowering it to significantly enhance domain adaptation performance. As per its definition [36 41 the domain index satisfies the following
(1) To acquire domain-invarian data encoding 2, global domain index O must remain independent of data encoding i.c., 0 IL z equivalently p(z 0) = p(z). (2) Effectively representing data point æ while averting the occurrence of collapsing. (3) Ensuring optimal performance of downstream tasks utilizing the data encoding learned by the encoder under the aforemenntoned constraints, and necessitating the maintenance of sensitivity to labels.
Variational domain index. In circumstances where the domain index may not be readily accessible, the Variational Domain Index (VDI) [41] is a Bayesian approach to infer the domain index 0 and u as latent variables. VDI factorizes the generative model p(æ, y. u, 0, z e) as:
p y, u. 0, E) p(0 <)p(u 0)p(z u)p(= x. 0)p(v) 2)
(1)
where E denotes the parameters for the prior probability distribution of the domain index 0. As shown in Equation [ VDI stands capable of significantly enhancing domain adaptation proficiency by leveraging the inferred domain index for generating data encoding z. Note that the independence
between the domain index 0 and data encoding i.e., p(= 0) p(z), does not contradict p(z x u, 0), given the existence of multiple pathways between the domain index A and data encoding Compared to VDI, which treats the distribution of domain index as single Gaussian, we focus on dynamic mixture of Gaussian distributions.
Chinese Restaurant Process(CRPP) (CPP) The Dirichlet Process (DP) is a classical method used for clus- tering. However, DP difficult construct directly, we apply the Chinese Restaurant Process(CRP) to implement it. Since similar domains have similar domain indices, the clusters formed by the domain indices correspond one-to-one with the components in mixture of Gaussian distributions. CRP can be employed to determine which cluster domain belongs to (i.e., which component distribution domaín index corresponds to). Specially, CRP is able dynamically and adaptively determine the number of mixture components. CRP operates as follows:
16
17
18
46
mi
if the cluster exists,
N
P = k)
(2)
if cluster is a new cluster,
where ni is the number of domain contained in cluster k. and parameter a the concentration parameter of the CRP. A larger implies a tendency to generate more domain clusters.
3

---

"Domain-thenne twwetteememsseeeee sspace of global domain index
Domain datasets
Local domain index
u
13 = (1* yr,, w
D ((x", w1
u
10r w)
U 5
Figure 2: The schematic diagram of domain index distributions. It shows the inference of variational Gaussian- shaped distributions for the global domain index, representing domain semanties. The process involves ranking candidate distributions in the "domain-themes" space, selecting the highest probability one, and deriving the local domain index from it.
4 Bayesian Domain Adaptation with Gaussian Mixture Domain-Indexing
4.1 Overview of GMDI
We propose GMDI in order to infer more interpretable domain indices and thereby improve domain adaptation performance. Our model is constructed in three steps: First, in generate process, model global domain indices as a dynamic Gaussian mixture model, with local indices generated from global domain index. Second, in inference process, build structured variational inference to approximate the posterior of the latent variables. Finally, we train the model using an evidence lower bound with robust theoretical guarantees and an adversarial loss. Under this framework, GMDI has several significant advantages: (1) With global domain indices following dynamic mixture of Gaussian distributions adaptively determined by CRP, provides a higher level of flexibility in larger latent space. (2) The evidence lower bound of our GMDI is more stringent, leading to more interpretable and optimal domain indices. The overview of GMDI is presented Algorithm
4.2
Mixture of domain index distributions
Similar VDI, GMDI (Figure (right)) also considers the intermediate latent variable of local domain index u. The local domain index u contains instance-leved information, meaning that each data point has unique local domain index. In contrast, the global domain index 0 contains domain- level information, indicating that all data points within the same domain share the same global domain index. In VDI, with the local domain index derived from the global domain index O, the data distribution p(y. æ
E) is expressed as:
ply x E) / p(0 e)p(u 0)p(z | u)p(2 æ; u. O)p(1 2) dzdudo,
(3)
where e denotes the parameters for the prior probability distribution of the global domain index 6.
With the setting of unsupervised domain adaptation, domains are not i.i.d, existing domain shift between domains. It implies that there may substantial differences distribution between domains. This leads to problem that disparate target domains require more significant degree of adaptation. Although we compute a distribution for domain index 0 to enhance the capability of domain adaptation, it may not be effective enough to aid in adapting to diversity of different target domains. Therefore, if local domain index u are adapted from a simple Gaussian distribution of global domain index 0, it may play a small role in improving the performance of domain adaptation.
To better model global domain index and thus enhance the effectiveness of domain adaptation, we propose to maintain mixture of dynamically updated global domain index 0 distributions in the "domain-themes" space. Intuitively, similar domains have similar global domain indices, implying that the mixture of global domain index distributions is associated with cluster of similar domains. The process of adapting local domain indices from global domain indices is illustrated in Figure 2 Specifically, we consider a Gaussian Mixture Model (GMM) as the mixture of global domain index

---

1
P.
D
N
N
Figure 3: The probabilistic graphical model of VDI (left) and GMDI (right). Edge type "..." denotes the independence between global domain index 0 and data encoding
distributions. For each distinct domain, first rank the candidate global domain index distributions in "domain-themes" space and select the distribution with the highest probability. We then derive the local domain index from the global domain index O. Therefore, the global domain index is designed to be dynamical GMM.
Let denote the latent categorical variable indicating the assignment of domain to a cluster, which equivalent to selecting components mixture distribution. Based on the definition of v, we derive the updated representation of the distribution p(y, æ |E::
ply,
E) ((v)
e)p(u 0" (")p(( u)p(= w, u, Oz p())(())))))))))))))))))) z) dzduddde.
The component distribution 0" selected from the mixture distribution of 0, and afterward, the local domain index is obtained from global domain index 0" for generating domain-invariant data encoding z. Equation and Equation 4 both represent the factorization the distribution p(y. æ s). 0 in Equation is the global domain index. While GMDI models the global domain index mixture of Gaussian distributions, Ø" in Equation 4 indicates the e-th component of the mixture distribution of A with the prior p(v). Compared to the single distribution, the mixture of global domain index distributions adequately model the domain index of different domains, enhancing the effectiveness of domain adaptation in the face of varying degrees even significant domain shifts. However, a remaining challenge determining the number of components in the mixture distributions, especially when there are numerous domains, or even possibly infinite ones.
4.3 Generative process of GMDI
extreme cases, there may be infinite domains. Due to CRP's flexibility dynamically determining the number of domain indices mixture components, employ CRP determine which cluster domain belongs to (i.c., which component distribution domain index corresponds to). Specifically, define the prior for domain indices cluster CRP, where the generation of new domain indices clusters controlled by parameter a. Thus, the probability of a domain belonging to cluster is calculated by Equation 2 Since CRP is an infinite mixture model, it is able easily adapted to infinite number of domains.
Mixture of domain indices need stick-breaking representation of CRP to obtain component weights. Stick-breaking representation indicating an infinite construction, considering the Dirichlet prior with parameter each element in the probability vector [M1, Tr,... is non-negative and the sum of the elements
3R a ~ Beta(1, a:) for k=l,
(5)
mT&
BR
II (1 3i) for k=l,,.,oc.
(6)
[=1
Equation 6 is equivalent to the weights implied by CRP. Based on Equation 6 and Equation 4 the
generative process of GMDI is as follows:
Categorical (T))
k N( o
(8)
a 0 v= p(u (²=k)
(9)
u ~ p(æ u)
(10)
α. u. nd p(2 2 u. 0"),
(11)
5

---

where Na and o are mean vector and semi nomitposiivve covariance matrix of the k-th component in dynamic Gaussian mixture domain indices, respectively. Figure iillustrates the generative process of VDI with a single distribution and GMDI with mixture of distributions for domain indices. Since CRP is computationally intensive. improve computational efficiency, we consider the stick- breaking construction to transform the infinite Gaussian mixture of domain indices into finite one. It can be achieved by directly specifying an upper bound K for the number of components in Gaussian mixture of domain indices. Selecting an appropriate K allows to effectively reduce computational overhead. The finite version of the generative process GMDI is available Appendix A
Accordingly, the generative model can be factorized as follows:
p(z, u, 0, z, 3 a)) p(3 a)p(v 33)(("))0 ( 0 (")p(z u)p(= æ, u. 0 ())(pp 2). (12)
The predictor p(y z) is a categorical distribution for classification tasks and a Gaussian distribution for regression tasks.
4.4 Evidence Lower Bound
The exact posterior of all latent variables, i.e., p(u, 0, v, x)) is intractable, variational inference used to approximate the posterior. Compared to the Monte Carlo sampling, variational inference allows both uncertainty quantification and computational efficiency. We employ structured variational inference to approximate the exact posterior, factorizing the approximate posterior q(u, 0, 2, U, x):
qu, 0 2, 2, x) ((::))))))))))) ()q((;))((( w; (a)q(("") u; 1 ()))(() x; 0" bb)),
(13)
where 7, n. 3b. 2o and 2b respectively represent the parameters of the variational distributions (()., (()), quu 2). q u)) and q(z æ, u. 0").
We train GMDI by maximizing the evidence lower bound(ELBB) to obtain the optimal variational distributions which best approximate exact posterior distributions. $ection|5jeemonsttr that our proposed GMDI has a more stringent evidence lower bound. Based on generative and inference process of GMDI, caluculate the ELBO as follows:
LELBO Ey(u. "ylu.."...u::))ae. =(e:)))(::)))
[log((y) =)] + E (loggre))
[log p(u[0') KL (I
E,t,, [KL[g (; m)lppp [; +b,)) g (o m (u:
Egtr:n)otn..0 m=)) [g[[(= x. u. 0" 2b ux, () [log quu zz; vb,)]
u.
(14)
where @ and & represent the parameters of the variational distributions q(u, 0", 2/xx) and g(u, 0*1x))
respectively, and KT. [:]1-] is the Kullback-Leibler divergence.
4.5 Adversarial loss with discriminator
To ensure the independence between global domain index and data encoding x as defined, we follow VDI [4]] by training an additional discriminator D with adversarial loss. As we prove in Section 5 that the independence between global domain index @ and data encoding relies on the independence between domain identity and data encoding z, the adversarial loss is simplified to discriminate the domain identity w:
LD = Ep(w,, Bggtln (log ((w[z)]
(15)
4.6 Objective function
Combining Equation 14 and Equation [5 the final objective of GMDI is:
LGNDI = max min LELBO - À LD:
(16)
D
where À denotes the hyper-paramest that balances two terms. Since the exact posterior of all latent variables is intractable, we propose to use structured variational inference method to approximate the exact posterior. More details can be viewed in the appendix B

---

Variational distribution of 3. To derive the optimal variational distribution of B3 we only consider the terms related to 3 LELBO. then we can get the posterior ((3 Yk)) Beta( 7e,1 7s.2)
with parameters Ye.1 = 1 + 7 and 7x.2 = a + Σ 77:-
Variational distribution of Similarly, the variational posterior of be calculated Categorical distribution q(e; 7) = @ategoricalggffo (u; 17), where the pareameters can be updated as:
log 7h ((:r)) [x] ( ", (u. (0(u000(x: [a:E] ( 0, 4/ (1ppp1////u/// x)]]
(17)
where Σx= -= 7h = and (B: = II k=1 (B3 7k).
Variational distribution of f.u and With assuming that the latent parameters are sampled from Gaussian, we have the following forms:
q00 u: vb) NN oî),
(18)
quu ax; 2b, Nu oú);
(19)
(22 a:, u, 0" </2))) (N,,,
(20)
where μl and o² are mean vector and semi-positive covariance matrix of Gaussian distribution. The parameters are updated by gradient descend. Specifically, we follow VDI [41] by using Earth Mover's Distance (EMD) and Multi-Dimennsioal Scaling (MDS) infer 0" from u.
5 Theory
In this section, we provide significant theoretical guarantees for our GMDI method. First, we give the upper bound for ELBO and adversial loss respectively. Second, we prove the upper bound of the whole loss with mutual information and entropy only. Moreover, we show that the upper bound can be achieved when the conditions are satisfied. Finally, we proove the significant result that our ELBO is better than the VDI, which means that mixture of Gaussian prior can get better results. See Appendix C for detailed proof.
Lemma The ELBO of p(æ, y) is bounded by the following formula with the Mutual Information, the Entropy and the KL-divergence:
Bpt (ELEO(p((: v))] <I(;; + [x:; u, 0, 2, -)) H())) (,)) kL[g(u, 0 v. lpp),,((u, 0, v.=
(I(())
0, v, =))pp(<) 0, v, x)]]
The main difference between and Lemma I in GMDI and Lemma 4.1 in VDI 41 is the last two KT. terms and the inclusion of v.
Lemma (Information Decomposition of the Adversarial Loss 47 We can decompose the global
maximum of adversial loss as follows:
max Emp(m, a (z(z)))) [log D(w) z)] [(2; 0) + 0) H(w).
D)
The global minimum of the function is achieved if and only (z; = 0 and 1 (z; w/) = 0.
Theorem The upper bound of the funcciion can be decomposed as follows:
LGMDI & T(y: 2) + [xx; u. 0 z, v) T(z; T(2; w () (H(x) + H(y) H H())).
The main difference between Theorem T iin GMDI and Theorem 4.1 in VDI 41 is the inclusion of e.
Theorem The global optimum achieved if and only if: (1)I 0)
1(z; w
(2)T(;:2) z) and I(x; O, z, v) are maximized, (3)KL/g(u,, 0, v, z = =| u, 0, 0,u.2) [T a,,,0,,,20 2,,,2)]lp(a) O, =,,=))==0 =.
0 and
The main difference between Theorem in GMDI and Theorem 4.2 in VDI 41 is that T(x;; u, 0 v) which includes v, needs to be maximized, and the two KT. divergences should equal zero.
7

---

10
0.7
MSE of Domain Indices
0.25
MSE of Domain Indices
0.20
0.5
0.15
0.4
0.2
0.05
0.1
0.00
10
30 Domain lds
20
40
50
10
20
30
40
50
Domain ids
Figure 4: MSE of domain indices on TPT-48 dataset. Left:N (24) - S (24), ground-truth domain indices are latitude. Right: W (6) E (42), ground-truth domain indices are longitude.
Table 1: Accuracy on binary classification tasks (Circle, DG-15, and DG-60) and 4-way classification task (CompCars).
Method
Dalasel
Source-only
DANN
ADDA
CDANN
MDD
SENTRY
D2V
VDI
GMDI (Ours)
Cincle
51.4
DG-15
39.7
43.3
33.5
38.8
37.2
42.6
79.9
94.7
96.5
DG-60
55.0
66.3
60.8
65.3
54.6
51.3
82.1
95.9
99.3
CompCars
38.9
41.4
41.8
42.
Theorem Assuming the ELBO and objective VDI are LVDI-ELBO and LVDI respectively, where domain indices are sampled from a simple Gaussian prior, we can prove that our objective achieves more stringent evidence lower bound which is closer the log- oog-likelhhood and also a tighter upper bound of the objective: LVDI-ELBO I CELBO I log p(z, y) and LVDI S -GGDDI
6 Experimental Study
We verify the effectiveness of GMDI via experimental comparison and analysis. In particular, we answer three research questions: (RQ1) Can the performance of GMDI for domain adaptation outperform baselines? (RQ2) How effective is the global domain indices inferred by GMDI? (RQ3) How does the number of mixture components K affect results? Additional experimental results are available in Appendix
6.1 Experimental setup
Datasets. We compare GMDI with existing DA methods on the following datasets (see Appendix H
and Appendix [ for more details): Circle [36] used for binary classification task. DG-15 and DG- 60 [40] are synthetic datasets used for binary classification task. TPT-48 [40] dataset is real-world dataset used for regression task. W (6) + (42): Adapting models from the states in the west to the 42 states in the east. N (24) + S (24): Adapting models from the 24 states in the north to the 24 states in the south. level-/ targer domains: one hop away from the closest source domain. level-2 target domains: two hops away from the closest source domain. level-3 target domains: more than two hops away from the closest source domain. CompCars dataset is a real-world dataset for 4-way classification task.
Baselines. To evaluate our proposed GMDI, compare against eight state-of-the-ar domain adaptation methods: Domain Adversarial Neural Networks (DANN) 44, Adversarial Discriminative Domain Adaptation (ADDA) [5]. Conditional Domain Adaptation Neural Networks (CDANN) 48
Margin Disparity Discrepancy (MDD) 47). SENTRY (27]. Domain to Vector (D2V) (26]. and Variational Domain Index (VDI) 41 Additionally, we include the results for models trained and tested only on the source domain (Source-only). Note that D2V is not applicable to regression tasks, so results are not reported on the 7PT-48 dataset. Moreover, since our proposed GMDI focuses on inferring domain indices when they unavailable, whereas [36] and 40) assume domain indices
"Reproducdd result from VDI.

---

Table 2: MSE for various DA methods in both tasks W (6) - E (42) and N (24) - S (24) on TPT-48. We report the average MSE of all domains well more detailed average MSE of level-1, level-2. level-3 target domains, respectively. Note that there is only one single DA model per column. We mark the best result with bold face.
Task
Domain
Source-only
DANN
ADDA
CDANN
MDD
SENTRY
VDI
GMD)(Ours)
Average 4 level-1 I domains
1.184
1,984
5,448
6.168
5,544
2.515
2.160
1.346
W (6) - E (42)
Average of level:2 damains
3,128
5.112
7.624
7.016
7.912
5.136
3.000
2.393
Averaee level-3 domains
5.272
5,880
7.256
6,986
8.008
5,872
2.448
2.122
Average of all domzins
4.576
6.896
7.76
5.456
2.496
2.087
Average level-1 domains
1.648
5,872
1.832
2.736
3.976
1.536
1.479
(24) S (24)
Average of level-2 domains
3,128
3,296
6,888
2,856
6.144
3.760
2.584
2.119
Average of level 3 domains
9.280
6.744
7,088
7.688
10.608
3.672
5.624
3.942
Average of all 24 domains
4,560
3,840
6,528
4,040
6.216
3,816
3,160
2.493
are available, they are not applicable to our setting. Detailed explanations of these algorithms can be found in the respective references.
6.2 Results and discussion
RQ1: Performance on classification and regression tasks. (1) Cirele, DG-15 DG-15 and DG-60. The results in Table ssow that, on all three datasets, the performance of basclines other than D2V and VDI is only marginally better or worse than random guess (accuracy of 50% ). This is likely due to the complex relationships between domains within the datasets, making it diffficult to adapt to target domains. Additionally, Source-only performs poorly due to overfitting. Compared to VDI, our GMDI improves accuracy by up to 3.4%, achieving very high accuracy (over 96.5%). This improvement is attributed to proposal of modeling the global domain index as the mixture distributions (e.g.. Figure (8. (2) TPT-48. In Table we report the mean square error (MSE) of the evaluated methods on TPT-48. In both E (6) - W (42) and N (24) - S (24) regression tasks, all methods except DANN, SENTRY, and VDI performed worse than Source-only, indicating the occurrence of negative transfer. In contrast, GMDI sigmificantly reduced the MSE compared to VDI, with average MSE decreases of 16% and 21% 1, respectively. (3) CompCars. The results Table show that our method achieves the best classification accuracy. All domain adaptation methods improved to varying degrees compared to Source-only, but our method achieved the highest increase in accuracy, with an improvement of up to 5%. In Figure Ileft), the data encoding generated by VDI are clustered together, indicating a mixture of points from different class labels. In contrast, in Figure rright), the data encoding of GMDI are separated by class label, demonstrating that GMDI can better distinguish points by class label. Across all datasets, GMDI significantly outperforms baselines, with minimum accuracy of 96.5 % on synthetic datasets, while MSE is reduced by at least 16 % on TPT-48 dataset.
RQ2: Effectiveness of inferred domain indices. Note that our proposed GMDI have no access to ground-truth domain indices O during traning. To evaluate the effectiveness of GMDI in inferring domain indices, we compare the inferred domain indices with the ground-truth domain indices and calculate MSE between them. As shown in Figure 5 for nearly all 30 domains on Circle dataset, the MSE of the domain indices inferred by GMDI is significantly lower than that inferred by VDI. On TPT-48 dataset, the domain indices for E (6) - W (42) and N (24) - S (24) regression tasks correspond to longitude and latitude of 48 states. Therefore, we use longitude and latitude as the ground-truth domain indices and calculate the corresponding MSE. In Figure [ it is evident that the MSE of the domain indices inferred by GMDI is still substantially lower than that of VDI. Although the CompCare lacks ground-truth domain indices, the data encoding visualization of VDI and GMDI(Figure (7) show that data encoding generated by GMDI form more distinct clusters compared to VDI. It indirectly indicates the effectiveness of the domain indices inferred by GMDI, demonstrating the considerable impact of modeling the global domain indices as Gaussian Mixture Model. On all datasets, the domain indices inferred by GMDI outperform those by VDI, owing to the dynamic mixture of domain indices distributions.
RQ3: Number of mixture components. We utilize the stick- bieckbreaking representation of the CRP to improve computational efficiency, setting an upper bound K on the number of components in GMM. To study the impact of the number of components on domain adaptation performance, we report classification accuracy on CompCars, more complex dataset for different values of K. For other datasets, the best results are achieved with K = 2, while for CompCars, the best performance

---

50.0
0.10
47.5
MSE of Domain Indices
0.08
42.5
0.06
Accuracy
40.0
0.04
37.5
35.0
0.02
0.00
30.0
10
15
25
30
2
Domaln
Figure 5: MSE of domain indices Circle dataset.
Figure 6: Accuracy (%) CompCars with different K.
0.7
0.5
0.6
0.3
0.2
0.1
0.1
0.0
0.1
0.2
0.4
0.5
0.1
0.2
0.5
0.6
0.8
0.9
Figure 7: t-SNE visualization of data encoding on CompCars dataset. Colors indicating different d2, 4). Left: data encoding generated by VDL. Right: data encoding generated by GMDI.
obtained with K 3. The results in Figure show that accuracy the lowest when K suggesting that maintaining single domain index distribution is insufficient for diverse target domains. The choice of is related to the concentration parameter CRP and dataset; the larger the concentration parameter and the more complex the dataset, the larger the value of K should be.
7 Conclusion and Limitations
In this work, we propose GMDI, a novel Gaussian Mixture Domain- Indexinn algorithm, to address the challenge of inferring domain indices when they unavailable. Unlike existing methods that assume global domain indices are sampled from a single static Gaussian, GMDI the first one to utilize mixture of dynamic Gaussians. The number of mixture components is determined adaptively by the Chinese Restaurant Process, enhancing the flexibility and effectiveness of domain adaptation. Our theoretical analysis confirms that GMDI achieves more stringent evidence lower bound, closer to the log- bog-likelihood. Extensive experiments validate the effectiveness of GMDI in inferring domain indices and highlight its potential practical applications. Specifically, for classification tasks, GMDI outperforms all approaches, and surpasses the state-of-the-an art method, VDI, by up to 3.4%, reaching 99.3%. For regression tasks, GMDI reduces MSE by at least 16% (from 2.496 2.087) and by 21 % (from 3.160 to 2.493), achieving the lowest errors among all methods. Despite these advantages, GMDI still relies on the availability domain identities and cannot infer them as latent variables. Future work will focus developing algorithms capable of inferring domain indices together with domain identities to further enhance the robustness and applicability our approach.
Acknowledgments
This research was partly supported by the Fundamental Research Funds for the Central Universities, Sun Yat-sen University (67000-31610047).
10

---

References
[1] Shuanghao Bai, Min Zhang, Wanqi Zhou, Siteng Huang, Zhirong Luan, Donglin Wang, and Badong Chen. Prompt-based distribution alignment for unsupervised domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 38. pages 729-737, 2024.
[2] Ingwer Borg and Patrick JF Groenen. Modern multidimensional scaling: Theory and applications. Springer Science & Business Media, 2005.
[3] Li Fei-Fei and Pietro Perona. A bayesian hierarchical model for learning natural scene categories. In 2005 IEEE computer society conference computer vision and pattern recognition (CVPR'05), volume 2, pages 5311. IEEE, 2005.
[4] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In Interna- tional conference on machine learning, pages 1180-1 189, 2015.
[5] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette. Mario March, and Vietor Lempitsky. Domain-adversarial training of neural networks. Journal of machine learning research, 17(59): 1- 35,,, 2016.
[6] Saurabh Garg. Nick Erickson, James Sharpnack, Alex Smola, Sivaraman Balakrishnan, and Zachary Chase Lipton. Rlsbench: Domain adaptation under relaxed label shift. In International Conference on Machine Learning, pages 10879-1099 10928. PMLR, 2023.
[7] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
[8] Huan He, Owen Queen, Teddy Koker, Consuclo Cuevas, Theodoros Tsiligkaridis, and Marinka Zitnik. Domain adaptation for time series under feature and label shifts. In International Conference on Machine Learning, pages 12746-12774. PMLR, 2023.
[9] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[10] Hengguan Huang, Xiangming Gu, Hao Wang. Chang Xiao, Hongfu Liu, and Ye Wang. Extrapolative continuous- bayesian neural network for fast training- free test-time adaptation. Advances in Neural Information Procexsing Systems. 35:36000-3601 2022.
[11] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint 16 1.01 144, 2016.
[12] Sang- Yeong Jo and Sung Whan Yoon. Poem: polarization of embeddings for Comain-invariin represen- tations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8150 8158, 2023.
[13] Jiyong Li, Dilshod Azizov, LI Yang, and Shangsong Liang. Contrastive continual learning with importance sampling and prototype-instane instance relation distillation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 13554-133 13562, 2024.
[14] Xiaotong Li, Yongxing Dai, Yixiao Ge, Jun Liu, Ying Shan, and Lingyu Duan. Uncertainty modeling for out-of-distribution generalization. In International Conference Learning Representations, 2023.
[15] Zijian Li, Ruichu Cai, Guangyi Chen, Boyang Sun. Zhifeng Hao, and Kun Zhang. Subspace identification for multi-source domain adaptation. Advances in Neural Information Processing Systems, 36, 2024.
[16] Shangsong Liang, Emine Yilmaz, and Evangelos Kanoulas. Dynamic clustering of streaming short documents. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 995 1004, 2016.
[17] Shangsong Liang, Zhaochun Ren, Emine Yilmaz, and Evangelos Kanoulas. Collaborative user clustering for short text streams. In Proceedings of the AAAI Conference Artificial Intelligence, volume 31, 2017.
[18] Shangsong Liang, Emine Yilmaz, and Evangelos Kanoulas. Collaboratively tracking interests for user clustering in streams of short texts. IEEE Transactions on Knowledge and Data Engineering, 31:257-272, 2018.
[19] Chen-Hao Liao, Wen-Cheng Chen, Hsuan- Tung Liu, Yi-Ren Ych, Min Min-Chun Hu, and Chu-Sons Chen. Domain invariant vision transformer learning for face anti-spoofing. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pages 6098 010886607,200 2023.
11

---

[20] Wang Lu, Jindong Wang. Xinwei Sun, Yiqiang Chen, and Xing Xie. Out-of-distribution representation learning for time series classification. arXiv preprini arXiv:2209.00027, 2022.
[21] Yawei Luo, Liang Zheng, Tao Guan, Junqing Yu. and Yi Yang. Taking closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2507-2516, 2019.
[22] Toshihiko Matsuura and Tatsuya Harada. Domain generalization using a mixture of multiple latent domains. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1174-11756, 2020.
[23] A Tuan Nguyen, Toan Tran, Yarin Gal, and Atilim Gunes Baydin. Domain invariant representation learning with domain density transformations. Advances in Neural Information Processing Systems, 34:5264-5275. 2021.
[24] Le Thanh Nguyen- Meiddne, Atif Belal. Madhu Kiran, Jose Dolz, Louis-Antoine Blais-Morin, and Eric Granger. Unsupervised multi-target domain adaptation through knowledge distillation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1339-1347, 2021.
[25] Xingchao Peng. Qinxun Bai, Xide Xia. Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1406-1415, 2019.
[26] Xingchao Peng, Yichen Li, and Kate Sacnko. Domain2vec: Domain embedding for unsupervised domain adaptation. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VI 16, pages 756-774, Springer, 2020.
[27] Viraj Prabhu, Shivam Khare, Deeksha Kartik, and Judy Hoffman. Sentry: Selective entropy optimization via committee consistency for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8558-8567, 2021.
[28] Sanqing Qu, Tianpei Zou, Florian Röhrbein. Cewu Lu, Guang Chen, Dacheng Tao, and Changjun Jiang. Upeycling models under domain and category shift. In Proceedings of the IREF/CVF Conference on Computer Vision and Pattern Recognition, pages 20019-20028, 2023.
[29] Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. Learning multiple visual domains with residual adapters. Advances in neural information processing systems, 30, 2017.
[30] Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. The earth mover's distance metric for image retrieval. International journal of computer vision, 40:99-121, 2000.
[31] Lianghe Shi and Weiwei Liu. Adversarial self training improves robustness and generalization for gradual domain adaptation. Advances in Neural Information Processing Systems, 36, 2024.
[32] Kunpeng Song, Ligong Han, Bingehen Liu, Dimitris Metaxas, and Ahmed Elgammal. calllgan-fusion Diftusion guided domain adaptation of image generators. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5453 5463, 2024.
[33] Tao Sun, Cheng Lu, and Haibin Ling. Domain adaptation with adversarial training on penultimate activations. In Proceedings the AAAI Conference on Artificial Intelligence, volume 37, pages 9935-9943, 2023.
[34] Eric Tzeng, Judy Hoffmann, Trevor Darrell, and Kate Saeuko. Simultaneous deep transfer across domains and tasks. In Proceedings of the IEEE international conference on computer vision, pages 2015.
[35] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7167-7176 2017.
[36] Hao Wang, Hao Hc, and Dina Katabi. Continuously indexed domain adaptation. International Conference on Machine Learning, pages 9898-9907. PMLR, 2020.
[37] Thomas Westfechtel, Hao-Wei Ych, Dexuan Zhang, and Tatsuya Harada. Gradual source domain expansion for unsupervised domain adaptation. In Proceedings of the IÉEE/CVF Winter Conference on Applications of Computer Vision, pages 1946-1955, 2024.
[38] Yi Wu, Ziqiang Li, Chaoyue Wang, Heliang Zheng, Shanshan Zhao, Bin Li, and Dacheng Tao. Domain re- modulation for few-shot generative domain adaptation. Advances in Neural Information Processing Systems, 36, 2024.
12

---

[39] Ruicheng Xian, Honglei Zhuang, Zhen Qin, Hamed Zamani, Jing Lu, Ji Ma, Kai Hui, Han Zhao. Xuanhui Wang, and Michael Bendersky. Learning list-level -inaainnn representations for ranking. Advances in Neural Information Processing Systems, 36, 2023.
[40] Zihao Xu, Hao He, Guang- He Lee, Bemie Wang, and Hao Wang. Graph-relational domain adaptation. In International Conference on Learning Representations. 2022.
[41] Zihao Xu, Guang- Yuan Hao, Hao He, and Hao Wang. Domain-indexing variational bayes: Interpretable domain index for domain adaptation. In International Conference on Learning Representations, 2023,
[42] Jinyu Yang. Jingjing Liu, Ning Xu, and Junzhou Huang. Tvt: Transferable vision transformer for unsupervised domain adaptation. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 520-530, 2023.
[43] Linjie Yang, Ping Luo, Chen Change Loy, and Xiaoou Tang. A large-scale car dataset fine- ane-grained categorization and verification. In Proceedings of the IEEE conference on computer vision and paitern recognition, pages 3973-3981, 2015.
[44] Kaichao You, Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan, Universal domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2720 2729, 2019.
[45] Zhongqi Yue. Qianru Sun, and Hanwang Zhang, Make the in uda matter: Invariant consistency learning for unsupervised domain adaptation. Advances in Neural Information Processing Systems, 36, 2024.
[46] Qiang Zhang, Jinyuan Fang. Zaiqiao Meng, Shangsong Liang, and Emine Yilmaz. Variational continual bayesian meta-learning. Advances in Neural Information Processing Systems, 34:24556 24568, 2021.
[47] Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algorithm for domain adaptation. In International conference machine learning, pages 1404-7413 PMLR, 2019.
[48] Mingmin Zhao, Shichao Yue. Dina Katabi, Tommi S Jaakkola, and Matt T Bianchi. Learning sleep stages from radio signals: A conditional adversarial architecture. In International Conference Machine Learning, pages 4109-4109, PMLR, 2017.
[49] Jinjing Zhu, Haotian Bai, and Lin Wang. Patch-mix transformer for unsupervised domain adaptation:
A game perspective. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3561-3571, 2023.
13

---

A Finite Stick-Breaking Construction of CRP
The infinite Chinese Restaurant Process (CRP) requires substantial computational overhead. To leverage CRP with lower computational cost, we use the stick brekkbraa construction to construct it. We set an upper bound the number of Gaussian mixture components, eliminating the need for varying number of mixture components. The finite ttic-breeking construction of CRP is given as follows:
3x (1ta a)) for k=1, ..., K
k
Th = BE
Bi)
for k= K K-1,
(21)
K K-1 (1 31),
1=1
where stickbreak(O) is the prior parameters of the K-dimensional category variable v. Con- sidering the finite stick breckbb construction of CRP mentioned above, we rewrite the generative process of GMDI:
t π Categorical
0 pm (
x
u r== p(u 0 0"=k)
(22)
# ~ p(# u), σ; u 0 p(=
x. u, 0*),
where μa and of are mean vector and semi comiipositive covariance matrix of the k-th component in GMM, respectively.
B Derivation for Variational Posterior
Firstly, we factorize the generative model as follows:
p(z, u, 0, z, v. a)) p(3 a))(( (op(0"))(() 0")p(= u)p(2 æ, u, 0" (p))p))))))))))))))))))) 2) (23)
During the inference process, need infer the latent variables. Since the target posterior distributions are intractable, we employ the technique of approximate variational inference. this framework, we design variational distributions for these latent variables, aiming to approximate their true underlying posterior distributions:
quu, 0, 2, v, B x) 4B;; 7)q(v; (u; m m)q(u
w; ",)((0" u; bo)a(( x. bb))..
(24)
Thus we calculate the ELBO follows:
log a) = log
p(x, y. 0, z, G d)dzduddddd3
p (a, y, u. 0, 2,2,v. v. a) guu, 0, x, v. x)
log
q (,, 0, 2, G x>)
dzdudOdu
p(a, y. 0, g(u, 0, 0,2,
log E.
0,2,0,2,08. U,,
Ey
log
p( a)p(v B))(u 6 (())(<> u)p(= x, u. 0" (p(p 2)
(25)
Ey,
log
((B:7){(u; 7)
æ; ab (00" u; bb))((< x, u. ;b; vb.)
F (00 [log p(y
=)] + E B [log u]]
( with [log ("))
KL[g( 7) (())) - B)),)) [T |1((") [)]]
Egglulan J (0 u: (o)
-Eg o ( [LLg(
x. 2, 0" <b))pp(( x, 0"]]]
E [log(q(u x: x.,l
A LELBO
14

---

where the inequality is given by applying Jensen's 's inequality. Besides, we apply the adversarial loss ensure the independence between global domain index and data encoding. The adversarial loss with discriminator D) is designed as follows:
LD = Ept x)) (e(xler flg D(uo =]]-
(26)
Our final objective derived as:
LONDI max min LELBO X LD.
(27)
By optimizing the objective function, we can calculate all the optimal variational distributions of latent variables as follows:
Variational distribution of J.. To obtain the variational posterior of the latent variable B, We only need to consider the related terms in LCMDI- Note that the adversarial loss LD is independent of the target latent variable, with all terms pertaining encompassed within the ELBO loss, which can be formulated as follows:
((((Bxim))) = Byt (logg + logp(v))
(28)
Subsequently, by differentiating the function and setting the derivative equal zero:
F(g(Bx:7w)
eg 7k)
/
alv; (377
((3 (10gg:1)))l)((111 + log (3) -
d3 d3
@g(B)iTk) = a(:; (le logp(u (3) - log 7
(29)
where ((3;: 7w,) is the variational posterior of /3 without Ba. we derive the final variational distribu-
tion of Bg as follows:
log (B 7k) Batern E (w
Dog + log [|]]]
K-
logr(Bx) + Bg((:n)
log Bx
Up
log(1 Bi)
1=1
uk log(1 (3,)
K-1
log ((g)( + Eyto:n) lo log 3r +
Σ
Uj log(1 - Bxx) + ux log(1 -Ba)
K
1(g)((;) + mh log 3
Σ 1
n,
log(1 -xx)
(30)
Since we assume that p(3)) Beta(:; 1, a)), we have:
log ((g)( (ar 1) log(1 -3x)-
(31)
Thus we can derive the optimal variational posterior of Bg as:
log ((g) x my log + (a 1
Σ
n2) log(1 Bx),
(32)
j=k+1
which is also Beta distribution Beta(Sai 7k, 7k.2) with parameters:
K
Mb.1
1+ m 2x.2 = a +
Σ n
(33)
j=k+1
Variational distribution of v. Since v is a category variable, we assume its variational posterior to be a categorical distribution parameterized by 77 as:
q(v: m)) Categorical (v; m)
(34)
15

---

Similarly. we only consider the terms related to in LGMDI to derive the optimal varational posterior, which is calculated as:
F Fl(((; m)) = (
[K
(( "e(p:::a:))0000""00 (a(("))(( [log p [
(K 3, w; 4/ (lp()ll(aaaa))))) v.]]
[
K
(1
77 log 7 Th ( p (KLLg [KL (b))lp(("))
k=1
+ 7h 14 pos
Th Betu, (x:{) ( 0" ;o))lp((22 O"))]
(35)
By taking the derivative function of F(g 77) with respective zero:
0 F(g(u:m))
= =B loo (3]33] log 7 ( [KL lu; (p0))(((0)))
Ōnk
(36)
py Ja(0 bo [(z(0)))))
Ey () (LLa(=/a u, 0" 2b (1>(2, 0*]]],
we can finally have:
log 77 xB, [r] + Buguu ( /w: (w:00
(og
Eylullrv. [KL (0" v))])(()")]
(37)
Bylu, (x:5) [KL (1( z/l,, O, a; 2 H (lp(plu, 0,)]]
where Σx=1 7h
and (B:))) = _IK-1 (3))7
Variational distribution of 0,u and z. The distribution of 0 is assumed to include a mixture of components. The distribution of the latent variable 0 assumed be mixture of series of distributions, while the latent variables and z can be regarded following conditional Gaussian distributions. Since is highly intractable to precisely compute the posterior distributions of these latent variables, we employ variational Gaussian distributions to approximate the posterior distribution each component of 0, as well as the conditional posterior distributions for u and z:
(0 n= :e vba) -N(H. A
(38)
(ulæ: vb) -N(Hu Aa),
(39)
(< a. u, 0" +b)) = -N),.,As),
(40)
where μ and A are mean vector and semi-positive covariance matrix of Gaussian distribution. All these variational distributions be updated by maximizing LOMDI with back propagation.
C Theory Analysis
The proof process of the lemmas and theorems are partially based on VDI 41
Lemma C.1 The ELBO of p(æ, y) is bounded by the following formula with the Mutual Information, the Entropy and the KL-divergencce
( v))l <I I(xx 0, (H(x) H By(a,u,0,
0, =)),p(z//,, =)]]
KL[g(u, 0 v, =.)
The main difference between and Lemma CTT GMDI and Lemma B.I in VDI 41 the last two KL terms and the inclusion of v.
16

---

Proof. In order to give an upper bound of p(a, y), we first calculate the ELBO as follows: log p(æ, y) = log
pl pl, u, 0, v, 9, y) ded,3
p(a, z/ (e))))(((u.. 0, dzdudo" dvdB
=log|
log
0,, 3, p(a, B. (uu, 0, (,
log 1Bg
log
p(x, 0,, quu, 0, u. 2()
Egg
p(u (()z))p(z>)) u. ( 0, 0.
Eg
log
0,,00,0, B, 2 (x)
Byg [(ogp(((z)) B. Plg u. 0 B, 2)] KT.pp(u, 0, n, (, 0, 0, (,
Accordingly, have the following ELBO objective:
LELBO v)) - =)]
0, 3, 0.
0, 3,,,,(z)))
(41)
Here we aim at giving a formula only including Mutual Information, Entropy and KL-divergence bound the objective. To achieve this, we first calculate the upper bound of Ey log p(y|z), with denoting raz, U, 2) p(æ, v) E Ex(a..) (oogglggpp22
Epta (yga
(42)
1e Dogp(y|2))
(43)
Er(,. (lo.))llogp(vis
(44)
(r(,,)) [lo [y]]=)]
(45)
r(y) r(ylz)
Erri, p(y)
(r(e.))oogr(u))
(46)
[(llz) - H())
(47)
When p(y|=) r(y]=) the maximum of E. [lg is achieved and then have max E, log p(y]=)] = [[v z) - Hv))
Secondly, need to give upper bound of E, p(z// 0, B, x)] For convenient, we denote the joint distribution s(x, u, 0, B, x) = p(z)q 0, v, B, z). then we can calculate as follows: Ep(e.,) [og p( O v. 3,2]]
Eo 0,
g u 0, v. 3, ((,,))
P(p() E, Plg
p(x:)
(axl u, 0, G
E. + Ea B. log p(x ux B, 3)
pp) E.g Dlg
p(x) x
(( u. 0, B,
[x u. 0 3, 2) + ((x) - [L[g(m/ 0 v. G, 1p( 0 B, 2)]
Finally, we apply these two bounds the Eq. 41 and then can proove the Lemma C.1
Eetes)) v))l (); x) + (x: u. 0, v) - (H())+ + H(v)) E Ey(a,u.0.2.0) [L[g(< u 0, 0, =)|pp(0/ 0, z)]]
KL[g(u, 0, u. [>]]p(u, u]).
Lemma C.2 (Information Decomposition the Adversarial Loss (4T can decompose the global
maximum of adversial loss as follows:
max Be By [log D(w[=)] = I(2; 0, [(2
H(w)
The global minimum of the function is achieved if and only [ 0) = 0 and [(2, w =
Proof. With denoting t(w, xr 0 2) = p(w, x)) (z) w/wa we have: EEg [log D(w) =)] (xx,,) llg D(w [|)= Ee(w.)) [log (w]]=)]
(48)
17

---

and () [log D(w[z)) achieves the maximum when t(w, x, 0, = Dwwlz). To futher
analyze the joint distribution f(w, 0, =) := q( w ) vj [))p(w) we assume that there is function vww) mapping w to a group of domain-related weights v, then we can have:
( 0 u, w) = p(= 0, v(w), w) = p(= 0 (r)))) p(z/w)
(49)
Accordingly, we can factorize the joint distribution t(w, 8, v, =) = q( (10p:)/)(((())))))))))]m(m"))( p())p() Therefore, we can factorize I (2;; 0. U, w) into two different styles with the chain rule for mutual information:
[(z 0, v) + I (zz, w v) [z; 0 v, w) [( w) + [z 0 w/w),
(50)
where (2, 0 2/ (w) = 0 due to the chain rule. That means:
[(2:w) [(z; 0, v) + [(z, 0)
(51)
And we also have:
llg [[]][]]]]]] = (x) [log 1w1lz)
+ (w (log [w)]
(52)
(ww)
= =[(w; z) - H (w).
(53)
Thus we have:
max Ep(w,a) 2o D(w[z)] (ww; 2) H (w)) (2:0, + 1(2, (( w u H(w). (54)
D
Accordingly, min maxp (e [log D(w | = if and only if ((2; v) = [(z w 0, =
0 due the fact that I(-) 2 O.
Theorem C.1 The upper bound of the objective function can be decomposed as follows:
LGMDI S (y;; + (x:; u, 0, 2) [(z; 0) [(2; 00) - (H(x) + H(y) - H()))
The main difference between Theorem in GMDI and Theorem B.1 in VDI 41 is the inclusion of v.
Proof. To proove the theorem, we apply the Lemma and C.2 to directly get the final upper bound:
LGMDI = Ep(xs)) [ (p(x, y))l max p E (x(=)) log [(][2)]
S T(y: + (a:; u. 0 2, v) T(2; 0) [22; (H) + H(y) - H(w))
By( KL (1 u, 0, z) [p(p]//(/uu////)) 2)] KL 0 v, az) p(u, 0, 2, u)]
< [)): + [(:; u. 0, 2, v) - [(z; 0) [(2:w)0) (H + Hyy - H Hw)),
where the second equality holds when all the terms of KL-divergence are equal to zero
Theorem C.2 The global optimum is achieved if and only if: (1)T(2; 0) T(2; w 0
0, (2) x) and I [(x; u. O, 2, v) are m3ximized, O x|=> p 0, U. x) = 0 and [g u, 0,v, (,,,)][[((a)) O, v, 2) 0.
The main difference between Theorem in GMDI and Theorem B.2 in VDI is that I [x;; u. 0, v)), which includes v, needs to be maximized, and the two KL divergences should equal zero.
Proof. The theorem can be prooved by observing the conditions from Lemma C.I and Theorem
Theorem C.3 Assuming the ELBO and objective of VDI are LVDI-ELBO and LVDI respectively, Gaussian prior, we can prove that our objective
where domain indices are from achieves more stringent evidence lower bound which is closer to the log- ligeeek,, and also tighter upper bound of the objective: LVDI-ELBO I LELBO I log p(x,g) y) and LVDI S LGMDI-
from a simple
Proof. To compare our objective loss with the VDI's, we first list the ELBO loss of VDI here and provide an upper bound:
LVDI-ELBO @LloOp((;, v)) -E, log p(v =²] llo. 0 2)] - KL[p(u, 0, 2)ll 0, 0|))))
S I(y; z) + (
(H) H()))
- 0 [T =
- 2
18

---

We can observe that the most significant difference is the prior distribution, which mainly affects the term of Kllddivegeecce. To further analyze, it is obvious that when 0 gu= = = VDI is special case of our proposed method GMDI. Hence we have:
mOL (p(2 v)) max CCMDI-ELBO
(55)
Further more, we can notice that the adversarial loss independent of the prior distribution of global indices and we can have:
LVDI = max
D L ELoo- u)) - A LD
< max min (L X LD D
(56)
-GMDDI
D Visualization of Inferred Domain Indices for Circle
Figure shows the inferred domain indices for Circle dataset. GMDI's inferred indices have correlation of 0.99 with true indices, even though GMDI does not have access to true indices during training.
Normalized Estimated Domain Indices
1.0
Correlation
0.8
0.6
0.4
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
Normalized True Domain Indices
Figure 8: Inferred domain indices (reduced to 1 dimension by PCA) with true domain indices for dataset Circle. GMDI's inferred indices have correlation of 0.99 with true indices, even though GMDI does not have access to true indices during training.
E
Architecture and Implementation Details
E.1
Architecture
The latent variables are estimated by neural networks. Specifically, for the local domain index u. we employ ResNet-18 18 [9] to approximate its posterior CompCars dataset, while using multi-layer perceptrons for the other datasets. Additionally, all neural networks are implemented as multi-layer perceptrons. We use the features obtained from the ResNet-18 as inputs to our model. Furthermore, all inputs are uniformly normalized.
We implement our model based on the code of VDI[1] We appreciate the authors for making their code publicly available. We run experiments on a single machine using NVIDIA GeForce RTX 2080Ti with 1 GB memory, 56 Intel Xeon CPUs (E5-2680 v4 @ 2,40GHz). takes about 30 minutes to train GMDI with K=2 on synthetic datasets. 4 hours on TPT-48, and hours on CompCars with K=3,
19

---

E.2 Hyperparameters
We set the maximum number of mixture components K from 2.3, and the concentration parameter a to I throughout the experiments. Except for DG-15 and DG-60 datasets, which have batch size of 32, other datasets use batch size of 16. Our model is trained with 20 to 100 warmup steps, learning rates ranging from 1 x 10-5 to x 10 10-3, and À ranging from 0. to 1.
F Broader Impacts
Our model has the potential to be applied to various domain shift problems, which also implies the possibility of unintended negative consequences. However, we have not identified any specific societal harms associated with our model. If used maliciously, it could lead to negative impacts.
G Pseudo Codes
The procedure of our proposed model GMDI is summarized by the pesuedo codes in Algorithm Let yp represent the parameters of the distribution p(-) in the generative process.
Algorithm 1 Bayesian Domain Adaptation with Gaussian Mixture Domain-
Input: Dataset 'D' and 'D"', .maxmmmm number of mixture components K, concentration parameter
a, learning rate Ç.
1: Initialize parameters: 4p, 2b {2b a: 18 22) ) 77 Ak L,..., K;
2: repeat
3: Update 7x with 7 = 1 + 7 and 7k.2 = a + E ni Vk = =1, 1,...,K. K;
4: Update 17 with Equation 37
5: Sample u from Equation
6: Sample based on Equation
7: Sample according to Equation
40
8: Update &o + --+ VV CGMDI
sb &oLGGDI
9: until converge
H Dataset Summary
Table
41
summarizes the statistics for all the datasets used in our experiments.
Table 3: Summary of statistics and settings in different datasets.
Dataset
Numbers of samples
Input dim
Synthetic/Real
Task
Circle
3,000
2
Synthetic
2- 2-Way classification
DG-15
1,500
2
Synthetic
2- 2-Way classification
DG-60
6,000
2
Synthetic
2- 2-Way classitication
TPT-48
6,912
6
Real
Regression
CompCars
18,735
224 x 224
Real
4ay classification
I Dataset
L.1 Circle
Figure 9 41 visualizes the detailed information of Circle dataset. It contains 30 domains and is used for binary classification task. The data points the Circle are arranged in semicircular shape, with each domain occupying a different section of the semicircle. There is decision boundary that separates the different labels. We use the first domains as the source domains and the remaining 24 domains the target domains.
20

---

1.-
10-
60
-05-
Domain
-15
Domain
-2.
-25
-1.5
-3.0
-05
0.
0.5
3.0
1.5
Figure 9: The Circle dataset with domains. Left: Different colors indicate ground-truth domain indices. The first domains (in the green box) are source domains. Right: Ground-truth labels for Circle, with red dots and blue crosses as positive and negative data points, respectively.
1.2 DG-15 and DG-60
DG-15 and DG-60 datasets containing 15 and 60 domains, respectively. Both used for binary classification task. Adjacent domains in the datasets have similar decision boundaries. For these two datasets, select 6 connected domains the source domains, while the remaining domains serve as the target domains.
1.3
TPT-48
visualizes the detailed information of TPT-48 dataset. contains the monthly average temperatures of 48 contiguous states the United States from 2008 2019. Our regression task is to predict the average temperatures for the next 6 months using the average temperatures of the first months. We divide this task into two finerggraine regression tasks with different adaptation directions:
Figure 10 40
(6) + E (42): Adapting models from the 6 states in the west the 42 states in the east.
N (24) - S (24): Adapting models from the 24 states in the north to the 24 states in the south.
To better verify performance, the target domains in the above two regression tasks are divided into three groups based on their distance from the closest source domains. level-1 target domains: one hop away from the closest source domain. level-2 target domains: two hops away from the closest source domain. level-3 target domains: more than two hops away from the closest source domain.
is
Domatn
Lesel
Target Hematas, Targer
Targei
Domain graphs two adaptation tasks on TPT-48, with black nodes indicating source domains and white nodes indicating target domains. Left: Adaptation from the states in the cast to the 24 states in the west. Right: Adaptation from the 24 states in the north to the 24 states in the south.
Figure 10:
21

---

1.4 CompCars
CampCars dataset includes three attributes: car types, viewpoints, and years of manufacture (YOMs).
We select a subset of CompCars with 4 types (MPV, SUV, sedan and hatchback), 5 viewpoints (front(F), rear (R), side (S), front-side (FS), and rear-side (RS)), and 6 YOMs(2009, 2010, 2011, 2012, 2013, 2014) for our experiments. This subset is divided into 30 domains(5 viewpoints 6 YOMS) based on viewpoints and YOMs. The car types are used as labels for prediction. The first domain, which has the front view and YOM 2009, is treated as the source domain, while the remaining 29 domains are target domains.
J
Notation
Table 4 provides a summary of the notations used in this paper.
Table Summary of notations.
Symbol
Definition
Input data
Label of data
w
Domain identity
Global domain index
Local domain index
Data encoding Latent categorical variable
Parameter for prior probability distribution of domain index Concentration parameter of the CRP independent random variable with Beta distribution in the stick-breaking representation Probability vector in the stick-breaking
Y
Parameter of ((3)
Parameter of q(v)
Parameter of q(u
x
V
Parameter of (
u)
4b
Parameter of q(z æ. 0")
Parameter of q(u, 0" =/x)
Parameter of quu 0
Plyper-paramete that balances two terms in objective function
K
Additional Experimental Results
To evaluate the impact and computational cost of the CRP, we conduct ablation and computational cost experiments. We implement GMDI w/o CRP using Gumbel-Softmax (T]. The number of components for GMDI w/o CRP is set to the upper bound of GMDI, and the hyperparameter temperature 7 for Gumbel-Softmax ranges from 0.1 to 50 (with the best performance reported). "Total time" refers to the total training duration, which concludes when the loss converges.
Table The results the ablation and computational cost experiments on TPT-48.
Task
Method
MSE
Total time
Epochs
Time per epoch
VDI
2.496
Ih 24m 18s
400
13s
W + E
GMDI w/o CRP
2.471
2h 14m 54s
500
16s
GMDI
2.087
Ih 31m 13s
300
18s
VDI
3.160
Ih 51m 43s
500
13s
N S
GMDI w/o CRP
3,050
2h 22m 58s
500
17s
GMDI
2.493
2h 2m 35s
400
18s
22

---

Table 6: The results of the ablation and computational cost experiments on CompCars.
Method
Accuracy(%5)
Total time
Epochs
Time per epoch
VDI
42.5
3h 13m 15s
600
19s
GMDI w/o CRP
43.0
4h 3m 48s
700
21s
GMDI
44.4
4h 16m 14s
600
26s
The experimental results are shown in Table 5nd and Table We find that although the proposed GMDI has longer "Time per epoch" compared to GMDI w/o CRP, it converges faster due to the flexible number of components adaptively controlled by CRP. Therefore, the "Total time" is roughly the same GMDI w/o CRP. On the TPT-48(W~>E) dataset, due to faster convergence, the "Total time" of GMDI is less than that of GMDI w/o CRP and even comparable to VDI. In all three datasets, the performance of GMDI w/o CRP worse than that of GMDI. On the two TPT-48 datasets, compared to the baseline VDI, GMDI w/o CRP reduces MSE by 1% and 3%, whereas GMDI reduces MSE by 16% and 21 %,, surpassing GMDI w/o CRP. On the CompCars dataset, GMDI's accuracy is higher than that of GMDI w/o CRP. These results indicate that although using a fixed-component GMM is simpler, the computational costs are roughly equivalent using CRP, but the performance is inferior to CRP, demonstrating the significance of CRP in GMDI.
Additionally, compared to VDI, which models the domain index single Gaussian distribution, GMDI's computational costs are only slightly higher, yet its performance superior. For large-scale datasets with numerous domains, modeling the domain index simple single Gaussian distribution may result in poor performance due to the dataset's s complexity. The experimental results indicate that GMDI has broad applicability.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Abstract and Section 1.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

Justification: See Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibiliyy
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Codes and dataset to reproduce the experiments are included the supplemen- tal material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
25

---

Answer: [Yes]
Justification:
Data and code will be publicly available once the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://niis.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips.com wttpcccpppbccccceec guwde//ddeeeeee Cooeeeoodeesssseeeeeee http:///d///uuuiiiddoooe///ea/e/
ssinislicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Experimental training details are provided in Appendix Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report performance mean after running experiments for multiple times with different seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
26

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The amount and the type of computing resource used in our experiments are described in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
Our research is conducted with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
27

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Citations for the existing assets (code, data and baseline models) are provided in Section 6 and Appendix.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
28

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "QaWY9qK4u9", "reviewer": "Reviewer_uzxH", "summary": "The paper introduces the Gaussian Mixture Domain-Indexing (GMDI) algorithm for domain adaptation when domain indices are unavailable. Unlike traditional methods that use a simple Gaussian prior, GMDI employs a Gaussian Mixture Model adjusted by a Chinese Restaurant Process, enabling adaptive determination of mixture components.", "questions": "See weaknesses.", "limitations": "The limitations are  addressed in the Sec. conclusion.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The use of a Gaussian mixture to represent complex domain indices seems interesting.\n- The proposed method demonstrates superior performance over baseline models in experimental results.\n- The technical part (the proposed method) seems non-trivial and its complexity seems sufficient, yet it might not be necessary.", "weaknesses": "- The learning loss for the proposed model is over too complex, featuring multiple conditional Kullback-Leibler divergences, which might complicate implementation and interpretation.\n- The paper lacks clear definitions for the 'local' and 'global' domain index within the probabilistic graphical model, which could confuse readers about the model's scope and applicability.\n- While introducing the Chinese Restaurant Process (CRP) adds flexibility to the Gaussian Mixture Model, it also increases computational costs. For the problem described in Figure 1, a fixed-component GMM might have been a simpler and more effective solution, though the number of components may not be that flexible. \n- The authors should also seriously consider empirically comparing the proposed method with a fixed-component counterpart, e.g. by simply using Gumbol softmax to infer the Gaussian component.\n- The related work may not be sufficiently discussed. For example [a] discussed an end-to-end approach that learns the domain index using adversarial learning; [b] takes the domain index/identity as a latent dynamical system, coupled with adversarial learning. \n\n[a] Out-of-distribution Representation Learning for Time Series Classification\n[b] Extrapolative Continuous-time Bayesian Neural Network for Fast Training-free Test-time Adaptation\n\n----------------Minor------------------\n\n- The connection between Equation (5) and Equation (6) is not clearly explained, leaving a gap in understanding the sequential logic of the model's formulation.\n-The paper's clarity and accuracy in writing could be improved. For instance, the statement \"DP requires a predefined number of components\" is misleading, as Dirichlet Processes are inherently nonparametric and do not require a predefined number of components.\n- Several symbols used in the equations are not adequately explained (both in the major paper and appendix), making it difficult to fully grasp the proposed model and its mathematical foundations.\n- The proofs and the theoretical part seem to closely follow that of VDI, making it hard to evaluate its novelty.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed comments and constructive suggestions. The following is our responses to the questions mentioned in the comments.\n\n**1. The learning loss for the proposed model is over too complex, featuring multiple conditional Kullback-Leibler divergences, which might complicate implementation and interpretation.**\n\nThanks for pointing out this shortcoming. The learning loss for the proposed GMDI is indeed somewhat complex. GMDI follows the framework of VDI, which involves multiple latent variables and requires KL divergence constraints on these variables, making this complexity unavoidable. However, by modeling the domain index as a mixture of Gaussian distributions and using the Chinese Restaurant Process (CRP), GMDI demonstrates excellent performance on the relatively more complex *CompCars* dataset. This indicates its potential for application to even more intricate datasets, showcasing its broad applicability.\n\n**2. The paper lacks clear definitions for the 'local' and 'global' domain index within the probabilistic graphical model, which could confuse readers about the model's scope and applicability.**\n\nWe apologize for the confusion. GMDI follows the setting of VDI.\n\n(1) Local domain index $u$: It contains instance-level information, meaning each data point has a unique local domain index. Essentially, the local domain index can be viewed as an intermediate latent variable.\n\n(2) Global domain index $\\theta$: It contains domain-level information, meaning all data points within the same domain share the same global domain index. The global domain index is the true \"domain index\".\n\nOur proposed GMDI models the global domain index (which contains domain-level information) as a mixture of Gaussian distributions. This mixture provides a higher level of flexibility in a larger latent space, thereby enhancing performance. We will incorporate the above definitions into the final version.\n\n**3. While introducing the Chinese Restaurant Process (CRP) adds flexibility to the Gaussian Mixture Model, it also increases computational costs. For the problem described in Figure 1, a fixed-component GMM might have been a simpler and more effective solution, though the number of components may not be that flexible.**\n\nWe appreciate the reviewer for highlighting this issue. In the global \"Author Rebuttal\", we conduct detailed ablation and computational cost experiments comparing the proposed method with a fixed-component GMM. Please kindly refer to the global \"Author Rebuttal\".\n\n**4. The authors should also seriously consider empirically comparing the proposed method with a fixed-component counterpart, e.g. by simply using Gumbol softmax to infer the Gaussian component.**\n\nThank you for the insightful suggestion. In the global \"Author Rebuttal\", we conduct detailed ablation and computational cost experiments comparing the proposed method with a fixed-component counterpart using Gumbol softmax. Please kindly refer to the global \"Author Rebuttal\".\n\n**5. The related work may not be sufficiently discussed. For example [a] discussed an end-to-end approach that learns the domain index using adversarial learning; [b] takes the domain index/identity as a latent dynamical system, coupled with adversarial learning.**\n\nThanks for the mention of these two related works. We follow the setting of VDI, where the concepts of \"domain index\" and \"domain identity\" indeed share significant similarities, though they are somewhat different. The two mentioned papers primarily focus on \"domain identity\", and they are both highly valuable and worthy of discussion. We will discuss them in the related work section of the final version.\n\n----------Minor----------\n\n**6. The connection between Equation (5) and Equation (6) is not clearly explained, leaving a gap in understanding the sequential logic of the model's formulation. -The paper's clarity and accuracy in writing could be improved. For instance, the statement \"DP requires a predefined number of components\" is misleading, as Dirichlet Processes are inherently nonparametric and do not require a predefined number of components.**\n\nWe appreciate the reviewer for pointing out the issues.\n\n(1) Equation 5 and Equation 6 both represent the posterior factorization of data encoding $z$. $\\theta$ in Equation 5 is the global domain index. While GMDI models the global domain index $\\theta$ as a mixture of Gaussian distributions, $\\theta ^{v}$ in Equation 6 indicates the $v$-th component of the mixture distribution of $\\theta$ with the prior $p\\left ( v  \\right )$.\n\n(2) Thanks for pointing out the misleading statement. The Dirichlet Process (DP) is indeed non-parametric. Due to the difficulty of direct construction, we apply the Chinese Restaurant Process by stick-breaking with a predefined bound $K$ to implement the DP. We will correct it with the explanatory sentences above.\n\n**7. Several symbols used in the equations are not adequately explained (both in the major paper and appendix), making it difficult to fully grasp the proposed model and its mathematical foundations.**\n\nWe apologize for the missing explanations of some notations. Below is a detailed explanation of the missing notation：\n\n$\\beta$: independent random variable with a Beta distribution in the stick-breaking representation;\n\n$\\pi$: the probability vector in the stick-breaking;\n\n$\\gamma$: parameter of $q(\\beta)$;\n\n$\\eta$: parameter of $q(v)$.\n\nWe will include the explanations of the above notations in the final version.\n\n**8. The proofs and the theoretical part seem to closely follow that of VDI, making it hard to evaluate its novelty.**\n\nThank you for bringing this to our attention. Compared to VDI, the theoretical contribution of the proposed GMDI is that it proves GMDI achieves a more stringent evidence lower bound, as well as a tighter upper bound of the objective.  Theoretically, it demonstrates that GMDI can effectively infer optimal domain indices.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "daxcb8s95N", "reviewer": "Reviewer_TJok", "summary": "This paper proposes a Bayesian Domain Adaptation method with Gaussian Mixture Domain-Indexing (GMDI) to address the challenge of inferring domain indices when they are unavailable. Existing methods often assume a single Gaussian prior for domain indices, ignoring the inherent structures among domains. GMDI models domain indices as a mixture of Gaussian distributions, with the number of components dynamically determined by the Chinese Restaurant Process. This approach provides a higher level of flexibility and effectiveness in adapting to diverse target domains. Theoretical analysis demonstrates that GMDI achieves a more stringent evidence lower bound, closer to the log-likelihood. Extensive experiments on classification and regression tasks show that GMDI significantly outperforms baselines, achieving state-of-the-art performance.", "questions": "See weaknesses.", "limitations": "See weaknesses.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. GMDI is the first to model domain indices as a mixture of Gaussian distributions, allowing it to capture the inherent structures among different domains. This approach provides a more flexible and powerful way to infer domain indices.\n\n2. By using the Chinese Restaurant Process, GMDI can dynamically determine the number of mixture components, adapting to varying numbers of domains. This enhances its capability to handle complex datasets with an unknown number of domains.\n\n3. The paper provides a detailed theoretical analysis, demonstrating that GMDI achieves a more stringent evidence lower bound and a tighter upper bound of the objective function compared to existing methods. This theoretical foundation supports the effectiveness of the proposed approach.", "weaknesses": "1. GMDI relies on the availability of domain identities but cannot infer them as latent variables. This limits its applicability to scenarios where domain identities are also unknown.\n\n2. The use of the Chinese Restaurant Process and Gaussian Mixture Model can be computationally intensive, especially for large-scale datasets with numerous domains. This could hinder the scalability of GMDI.\n\n3. In the experiment, the binary classification task is not challenging, which makes the performance advantage not convincing. The multi-classification tasks are necessary.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive comments, the insightful questions, and helpful suggestions. The following are our responses to the questions mentioned in the comments.\n\n**1. GMDI relies on the availability of domain identities but cannot infer them as latent variables. This limits its applicability to scenarios where domain identities are also unknown.**\n\nThank you to the reviewer for pointing out this weakness. Our proposed GMDI focus on inferring domain indices following a mixture of Gaussian distributions, with the number of mixture components dynamically determined by a Chinese Restaurant Process (CRP). Extensive experiments on classification and regression tasks demonstrate the strong domain index modeling capability of GMDI, significantly outperforming state-of-the-art methods. However, we acknowledge that GMDI's reliance on domain identities limits its applicability in situations where domain identities are unknown. We plan to address this issue in future work.\n\n**2. The use of the Chinese Restaurant Process and Gaussian Mixture Model can be computationally intensive, especially for large-scale datasets with numerous domains. This could hinder the scalability of GMDI.**\n\nWe appreciate the reviewer for highlighting this issue. In the global \"Author Rebuttal\", we conduct detailed ablation and computational cost experiments comparing the proposed GMDI with GMDI w/o CRP and VDI.  Please kindly refer to the global \"Author Rebuttal\" for a comprehensive analysis.\n\n**3. In the experiment, the binary classification task is not challenging, which makes the performance advantage not convincing. The multi-classification tasks are necessary.**\n\nThank you for bringing this to our attention. Fllowing VDI, we have tested the performance of GMDI on the multi-classification dataset *CompCars*. The *CompCars* dataset is a 4-way classification dataset containing 18,735 samples across 30 domains. Due to the scarcity of datasets with ground-truth domain indices, we leave the extension of experiments on new datasets for future research. More details about the datasets used are available in Appendix H of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ngFazUUsRw", "reviewer": "Reviewer_72He", "summary": "The paper proposes a novel method, \"Gaussian Mixture Domain-Indexing\" (GMDI), to address domain adaptation with inaccessible domain indices. The technique improves upon prior work by modeling the domain indices prior with a Gaussian Mixture. Empirically, it has been shown that the proposed method achieves state-of-the-art performance in classification and regression tasks.", "questions": "**Typos?** Should the left side of Equation (1) be $p(z|x,\\epsilon)$ or should $\\int_{x}$ be added to the right side? A similar issue seems to exist in Equation (5) as well.\n\n**Abstract Claim** I do not understand the authors' claim in the abstract, “For classification, GMDI improves accuracy by at least 63% (from 33.5% to 96.5%).” It seems from Table 1, the result for DG-60, the authors picked the worst performance of all baselines, ADDA with an accuracy of 33.5%. It is unclear why the authors claim the improvement is at least 63%.", "limitations": "I do not see specific limitations that might lead to potential negative societal impact.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Novelty**: The paper proposes a novel technique to address the issue of domains in domain adaptation having multiple semantics. The method is a natural extension from prior work (VDI) by changing the Gaussian prior to a Gaussian Mixture.\n\n**Theoretical Support**: The paper proves the correctness of the proposed method with solid theoretical derivations.\n\n**Empirical Validation**: Extensive experiments show that GMDI significantly outperforms state-of-the-art methods in both classification and regression tasks, with substantial improvements in accuracy and mean squared error (MSE). I particularly like the illustration of the learned domain indices. Figures 4 and 5 clearly show that GMDI learns the domain indices more accurately compared to the prior state-of-the-art method, VDI. Good job!", "weaknesses": "**Clarity**: Some equations in the paper might have typos. Please see the details in the next section.\n\n**Computational Overhead**: The use of CRP and dynamic mixtures increases computational overhead, which might make the method less practical for large-scale or real-time applications without further optimization. It would be helpful if the authors could provide some comments and empirical analysis on the computational overhead of GMDI.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed and positive comments. The following is our responses to the questions mentioned in the comments.\n\n**1. Computational Overhead: The use of CRP and dynamic mixtures increases computational overhead, which might make the method less practical for large-scale or real-time applications without further optimization. It would be helpful if the authors could provide some comments and empirical analysis on the computational overhead of GMDI.**\n\nWe appreciate this useful suggestion of the reviewer. In the global \"Author Rebuttal\", we conduct detailed ablation and  computational overhead experiments comparing the proposed GMDI with GMDI w/o CRP. Please kindly refer to the global \"Author Rebuttal\".\n\n**2. Typos? Should the left side of Equation (1) be $p\\left ( z\\mid x, \\varepsilon   \\right )$ or should $\\int_{x}$ be added to the right side? A similar issue seems to exist in Equation (5) as well.**\n\nThanks for pointing out the typos. The left side of Equation (1) and Equation (5) should be corrected to $p\\left ( z\\mid x, \\varepsilon \\right )$. We will include the correction  in the final version.\n\n**3. Abstract Claim I do not understand the authors' claim in the abstract, “For classification, GMDI improves accuracy by at least 63% (from 33.5% to 96.5%).” It seems from Table 1, the result for DG-60, the authors picked the worst performance of all baselines, ADDA with an accuracy of 33.5%. It is unclear why the authors claim the improvement is at least 63%.**\n\nWe apologize for the misleading claim. We will correct it in the final manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the respected reviewers for their detailed comments and believe that all the mentioned issues can be properly addressed in the final version of our paper. The major concerns lie in the ablation and computational cost experiments. We take this opportunity to clarify these issues and present our responses accordingly. Regarding the ablation and computational cost experiments, we have followed the suggestions and provided detailed comparison results in the following Ablation Study section.

We appreciate the reviewers for highlighting the issue of computational cost associated with the Chinese Restaurant Process (CRP). The CRP is indeed computationally intensive. To improve computational efficiency, the proposed GMDI employs the stick-breaking construction of the CRP by specifying an upper bound $K$ for the number of components in the Gaussian mixture. By selecting an appropriate $K$, we are able to leverage the benefits of the CRP while reducing computational cost. More details are available in Appendix A of the paper.

To evaluate the impact and computational cost of the CRP, we conduct ablation and computational cost experiments. Following the reviewers' suggestions, we implement GMDI w/o CRP using Gumbel softmax. The number of components for GMDI w/o CRP is set to the upper bound $K$ of GMDI, and the hyperparameter temperature $\tau$ for Gumbel softmax ranges from 0.1 to 50 (with the best performance reported). "Total time" refers to the total training duration, which concludes when the loss converges. The experimental results on three datasets are shown in the following tables.

(1) *TPT-48*(W->E) dataset:

| Method       | MSE       | Total time | Epochs | Time per epoch |
| ------------ | --------- | ---------- | ------ | -------------- |
| VDI          | 2.496     | 1h 24m 18s | 400    | 13s            |
| GMDI w/o CRP | 2.471     | 2h 14m 54s | 500    | 16s            |
| GMDI         | **2.087** | 1h 31m 13s | 300    | 18s            |

(2) *TPT-48*(N->S) dataset:

| Method       | MSE       | Total time | Epochs | Time per epoch |
| ------------ | --------- | ---------- | ------ | -------------- |
| VDI          | 3.160     | 1h 51m 43s | 500    | 13s            |
| GMDI w/o CRP | 3.050     | 2h 22m 58s | 500    | 17s            |
| GMDI         | **2.493** | 2h 2m 35s  | 400    | 18s            |

(3) *CompCars* dataset:

| Method       | Accuracy(%) | Total time | Epochs | Time per epoch |
| ------------ | ----------- | ---------- | ------ | -------------- |
| VDI          | 42.5        | 3h 13m 15s | 600    | 19s            |
| GMDI w/o CRP | 43.0        | 4h 3m 48s  | 700    | 21s            |
| GMDI         | **44.4**    | 4h 16m 14s | 600    | 26s            |

We find that although the proposed GMDI has a longer "Time per epoch" compared to GMDI w/o CRP, it converges faster due to the flexible number of components adaptively controlled by CRP. Therefore, the "Total time" is roughly the same as GMDI w/o CRP. On the *TPT-48*(W->E) dataset, due to faster convergence, the "Total time" of GMDI is significantly less than that of GMDI w/o CRP and is even comparable to VDI. In all three datasets, the performance of GMDI w/o CRP is significantly worse than that of GMDI. On the two *TPT-48* datasets, compared to the baseline VDI, GMDI w/o CRP reduces MSE by only 1% and 3%, whereas GMDI reduces MSE by 16% and 21%, far surpassing GMDI w/o CRP. On the *CompCars* dataset, GMDI's accuracy significantly higher than that of GMDI w/o CRP. These results indicate that although using a fixed-component GMM is simpler, the computational costs are roughly equivalent to using CRP, but the performance is far inferior to CRP, demonstrating the significance of CRP in GMDI.

Additionally, compared to VDI, which models the domain index as a single Gaussian distribution, GMDI's computational costs are only slightly higher, yet its performance is far superior. For large-scale datasets with numerous domains, modeling the domain index as a simple single Gaussian distribution may result in poor performance due to the dataset's complexity. The experimental results indicate that GMDI has broad applicability.

Meta-review: The paper builds upon Variational Domain Indexing (VDI) from ICLR 2023 to introduce the Gaussian Mixture Domain-Indexing (GMDI) algorithm to tackle the challenge of domain adaptation when domain indices are not available. Unlike traditional approaches that assume a single Gaussian prior for domain indices, GMDI models these indices using a Gaussian Mixture Model, with the number of components dynamically determined by a Chinese Restaurant Process. This approach is intended to provide more flexibility and better adaptation to diverse domains.

### Strengths

- GMDI is innovative in its use of a Gaussian Mixture Model to capture the inherent structures among different domains, which allows for a more flexible and powerful inference of domain indices.
- The method is theoretically well-founded, with proofs that demonstrate a more stringent evidence lower bound and a tighter upper bound compared to existing methods.
- Extensive experiments show that GMDI outperforms state-of-the-art methods in both classification and regression tasks, achieving significant improvements in accuracy and mean squared error (MSE).

### Weaknesses

- The model’s learning loss is complex, involving multiple conditional Kullback-Leibler divergences, which could make implementation and interpretation challenging.
- The use of the Chinese Restaurant Process and Gaussian Mixture Model increases computational overhead, potentially limiting the scalability of the method for large-scale datasets or real-time applications.
- The paper lacks clarity in some areas, such as the definitions of 'local' and 'global' domain indices, and certain equations may contain typos, which could confuse readers.

Some concerns have been addressed by the authors during the rebuttal period. 

Near the end of the discussion, there are new concerns on the citation of VDI on the theorems, raised by the reviewers. As prompted by the reviewers, I also briefly checked the paper myself. I think these concerns are kind of valid. For example, most theorems/lemmas in the paper are from the VDI paper, and therefore should be proper attributed to VDI by citation. Another example is Line 255, where "VDI" should be "GMDI"? That said, I do agree with the majority that the paper has its technical merit. What is more important at this point is proper attribution and citation, especially on the theory part. In my opinion, properly attributing the theories will not affect the value of GMDI; on the contrary, it enhances GMDI's credibility and rigor. The authors did reply to me, promising that they will address these concerns fully in the revision. 

Below I summarize several points of revision suggestions after I went over GMDI and VDI:

+ Fix the typo in Line 255 from VDI to GMDI
+ **Lemma 1 and Lemma 4.1 in VDI:** After stating Lemma 1, **cite VDI** and **clearly state** that `“the main difference between and Lemma 1 in GMDI and Lemma 4.1 in VDI [37] is the last two KL terms and the inclusion of $v$”`. 
+ **Lemma 2 and Lemma 4.2 in VDI:** Lemma 2 is exactly the same as Lemma 4.2 in VDI. Therefore, please properly **cite VDI inside the bracket of Lemma 2’s title, i.e., `“(Information Decomposition of the Adversarial Loss **[37]**)”`.
+ **Theorem 1 and Theorem 4.1 in VDI:** After stating Theorem 1, **cite VDI** and **clearly state** that `“the main difference between VDI and GMDI is the inclusion of $v$"`.
+ **Theorem 2 and Theorem 4.2 in VDI:** After stating Theorem 2, **cite VDI** and **summarize** the main difference between VDI and GMDI. There are more differences in this theorem; I believe it would give more insight to readers on the contribution of GMDI beyond VDI if the authors can clarify them well here. 

I agree with the majority that this is an interesting paper, with solid extension to VDI, which itself is a promising new direction in domain adaptation and (hierarchical) Bayesian deep learning. 

I also tend to trust the authors’ promise on the revision and proper citation/attribution and will give a *conditional accept*. Note that 
+ I will help shepherd the revision, and make sure all points, especially those I listed above, are fully incorporated in the revision. Since this is a matter of academic rigor; otherwise this paper would have the risk of plagiarism. 
+ While I recommend acceptance for now, I will stay in touch with the PC Chairs and SAC, and reserve the right to retract the paper if the camera ready does not fully address the concerns above.
Decision: Accept (poster)