Domain-Indexing Variational Bayes: Interpretable Domain Index for Domain Adaptation
Authors: Zihao Xu
Guang-Yuan Hao
Hao He
Hao Wang
Venue: ICLR 2023 notable top 25% (2023)

Abstract: Previous studies have shown that leveraging "domain index" can significantly boost domain adaptation performance (Wang et al., 2020; Xu et al., 2022). However, such domain indices are not always available. To address this challenge, we first provide a formal definition of domain index from the probabilistic perspective, and then propose an adversarial variational Bayesian framework that infers domain indices from multi-domain data, thereby providing additional insight on domain relations and improving domain adaptation performance. Our theoretical analysis shows that our adversarial variational Bayesian framework finds the optimal domain index at equilibrium. Empirical results on both synthetic and real data verify that our model can produce interpretable domain indices which enable us to achieve superior performance compared to state-of-the-art domain adaptation methods. Code is available at https://github.com/Wang-ML-Lab/VDI.

Published as a conference paper at ICLR 2023
DOMAIN-INDEXING VARIATIONAL BAYES: INTER- PRETABLE DOMAIN INDEX FOR DOMAIN ADAPTATION
Zihao Xu'", Guang- Yuan Hao"", Hao He,, Hao Wang'
'Rttgers University. 'Hong Kong University of Science and Technology, 3 "Massachusetts Institute of Technology,
zihao. xuê rutgurg edu, guangyuanhao@uut.oo... haohe@mit. edu,, hw4 rutgers.
ABSTRACT
Previous studies have shown that leveraging domain index can significantly boost domain adaptation performance (Wang et al., 2020; Xu et al., 2022). However, such domain indices are not always available. To address this challenge. we first provide formal definition of domain index from the probabilistic perspective, and then propose an adversarial variational Bayesian framework that infers domain indices from multi-domain data, thereby providing additional insight on domain relations and improving domain adaptation performance. Our theoretical analysis shows that our adversarial variational Bayesian framework finds the optimal domain index equilibrium. Empirical results on both synthetic and real data verify that our model can produce interpretable domain indices which enable us achieve superior performance compared to -tate-of-the-art domain adaptation methods. Code is available https: //g. thub. com/ 'Wang- ML- Lab/ /VDI.
INTRODUCTION
In machine learning, it is standard to assume that training data and test data share an identical distribution. However, this assumption is often violated (Ganin & Lempitsky, 2015; Romera et al., 2019; Sun et al., 2017; Yuan et al., 2019; Ramponi & Plank, 2020) when training and test data come from different domains. Domain adaptation (DA) tries to solve such cross-domain generalization problem by producing domain-invarian features. Typically, DA methods enforce independence between data point's latent representation and its domain identity, which is a one-hot vector indicating which domain the data point comes from (Ganin et al., 2016; Tzeng et al., 2017; Zhao al., 2017; Zhang et al., 2019).
More recent studies have found that using domain index, which is a real-value scalar (or vector) to embed domain semantics, as replacement of domain identity, significantly boosted domain adaptation performance (Wang et al., 2020; Xu et al., 2022). For instance, Wang et al. (2020) adapted sleeping stage prediction models across patients with different ages, with "age" as the domain index, and achieved superior performance compared to traditional models that split patients into groups by age and used discrete group IDs as domain identities (more discussion in See. J).
Although significant progress has been made in leveraging domain indices to improve domain adaptation (Wang et al., 2020; Xu et al., 2022), a major challenge exists: domain indices are not always available. This severely limits the applicability of such indexed DA methods. Thus a natural question is motivated: Can one infer the domain index as a latent variable from data?
This prompts us to first develop expressive and formal definition of "domain index". We argue that an effective "domain index" (1) is independent of the data's encoding, (2) retains as much information on the data as possible, and (3) maximizes adaptation performance, e.g., accuracy (see Sec. 3.2 for rigorous descriptions). With this definition, we then develop an adversarial variational Bayesian deep learning model (Wang et al., 2015; Wang & Yeung, 2016; 2020) that describes intuitive conditional dependencies among the input data, labels, encodings, and the associated domain indices. Our theoretical analysis shows that maximizing our model's evidence lower bound while adversarially
These authors contributed equally this work.

---

Published as a conference paper at ICLR 2023
training an additional discriminator (Ganin et al., 2016; Wang et al., 2020) is equivalent to inferring the optimal domain indices (according to our definition) that maximize the mutual information among the input data, labels, encodings, and the associated domain indices while minimizing the mutual information between the data's encodings and the domain indices. Our contributions are as follows:
We identify the problem of inferring domain indices latent variables, provide rigorous definition of "domain index", and develop the first general method, dubbed variational domain indexing (VDI), for inferring such domain indices.
Our theoretical analysis shows that training with VDI's final objective function is equivalent to inferring the optimal domain indices according to our definition.
Experiments on both synthetic and real-world datasets show that VDI can infer non-trin non-trivial domain indices, thereby significantly improving performance over state-of-the-art DA methods.
2 RELATED WORK
Typical Domain Adaptation. There rich literature on domain adaptation (Pan & Yang, 2009;
Pan et al., 2010; Long et al., 2018; Saito et al., 2018; Sankaranarayanan et al., 2018; Peng et al., 2019; Prabhu et al., 2021; Wang et al., 2020; Xu et al., 2022). Typically they try to align source-domain and target-domain data in the latent space, with the hope that such can generalize well on unseen data. There are multiple ways to achieve such aligmment, including distribution matching (Pan et al., 2010; Tzeng al., 2014; Sun & Saenko, 2016; Peng al., 2019; Nguyen-Meidig et al., 2021), self-training (Zou et al., 2018; Kumar et al., 2020; Prabhu et al., 2021), domain-specific normalization (Maria Carlucci et al., 2017; Mancini et al., 2019; Tasar et al., 2020), and deep learning models with adversarial training (Ganin et al., 2016; Tzeng et al., 2017; Zhang al., 2019; Zhao al., 2017; Chen et al., 2019; Dai et al., 2019). Most of these methods rely on (one-hot) domain identities for feature alignment.
encodings
Domain Adaptation with Domain Identities and Domain Indices. There are also works that generate domain identities from data to improve domain adaptation. Chen & Chao (2021) generates a sequence of domain identities for intermediate domains between a source domain and a target domain, trying to facilitate better incremental domain adaptation (Bobu et al., 2018). Deecke et al. (2021) generates a set of domain identities to split dataset into different domains and perform multi-domain learning. Du et al. (2021); Lu et al. (2022) partition time series data by learning domain identities that maximize the domain-wise distribution gap. All these works focus on generating (ordinal or one-hot) domain identities. In contrast, VDI assumes such domain identities are already given and focuses on inferring (continuous) domain indices, which contain richer and more interpretable information. Note that in our setting, since domain identities are given, methods such Deecke et al. (2021) are equivalent to typical domain adaptation methods (Ganin et al., 2016; Tzeng et al., 2017; Prabhu al., 2021), which are considered as our basclines in Sec.
Recent studies have found that replacing (one-hot) domain identities with (continuous) domain indices improves adaptation performance (Wang al., 2020; Xu et al., 2022). However, none of these works provide a canonical definition of "domain index"; instead they mainly rely on intuition, e.g., using rotation angles as domain indices for RotatingMNIST (Wang al., 2020) and using graph node embeddings as domain indices for adaptation across graph-relational domains (Xu et al., 2022). More importantly, they assume that such domain indices are always available, which may not be true (Matsuura & Harada, 2020; Rebuffi et al., 2017), hence they are not applicable to our setting. Also related to our work is Peng et al. (2020a), which generates embeddings of visual domains to represent domain similarities; however, they do not formally define "domain embedding" and only work for classification tasks. In contrast, our VDI provides a rigorous and formal definition for "domain indices" and handles both classification and regression tasks.
METHOD
In this section, we formalize the definition of "domain index" and describe our VDI for inferring domain indices. We provide theoretical guarantees that VDI infers optimal domain indices in Sec. 4.
2

---

Published as a conference paper at ICLR 2023
N
N
Z
B
u
X
Dk
DK
X
u
B
Figure 1: Left: Probabilistic graphical model for VDI's generative model. We introduce new edge type, "...", to denote independence. ~ 2 enforces independence between z and Bx. i.e, p(z(3,) p(z) (see Appendix Sec. I for detailed discussion). Right: Probabilistic graphical model for the VDI's inference model.
3.1 PROBLEM SETTING AND NOTATION
consider the unsupervised domain adaptation setting with N domains in total. Each domain has domain identity & K [N] _ {1...., J.....,NJ,, N}; ls is in either the source domain identity set K. or the target domain identity set KG. Each domain & has D2 data points. Given labeled data points {(x, yr, bk,kk)) (+))==1 from source domains (A: E K.2), and m unlabeled data points {xf, k from target domains (K KT). want to (1) predict the label { for target domain data, and (2) RRB for for [ 's prior distributions. Note that each domain has only global domain index, but has multiple local domain indices, one for each data point in the domain (more details in Sec. 3.3). We denote as z E RB, the data encoding generated from encoder that takes input. We use ((::) to denote mutual information.
infer global domain indices By E BR for each domain and local domain indices each data point. a a Ta) are the
3.2 FORMAL DEFINITION OF DOMAIN INDEX
We formally define "domain index" follows (please refer to notations in Sec. 3.1 if needed):
Definition 3.1 (Domain Index). Given data and label y. domain-level variable f and data-level variable u are called global and local domain indices, respectively, if there exists a data encoding such that the following holds:
(1) Independence between B3r and z: Global domain index 3 is independent of duta encoding i.e., Blll,, equivalently I(3; z) 0. This is encourage encoding z.
data
(2) Information Preservation of x: Data encoding z. local domain index and global domain index B3 preserves as much information on x as possible, iee, maximizing T(x; u. B3 z). This is prevent B and from collapsing trivial solutions.
(3) Label Sensitivity of z: The data encoding z should contain as much information on the label y as possible to maximize prediction power, i.e., maximizing [yy; z) conditioned on I. This is to make sure the previous two constraints on B, u, and z do not harm prediction performance.
To summarize,, [3 and u are considered the global and local domain indices, respectively, if (B, u) argmax [xx; u, 3,z) T(y;z) s.t. [(3 z.
Later in Sec. 4, our theoretical analysis shows that maximizing our model's 's evidence lower bound while adversarially training an additional discriminator (Sec. 3.4) is equivalent inferring the optimal domain indices according Definition 3.1. Appendix A, we provides a rigorous discussion on the definition of "domain index".
3.3 DOMAIN-INDEXING VARIATIONAL BAYES FOR DOMAIN ADAPTATION
Generative Process and Probabilistic Graphical Model. Based on our definition, we propose our model: Variational Domain Indexing (VDI). The basic idea is to infer the domain indices as latent variables during domain adaptation. VDI is generative model assuming the following generative process (see the corresponding graphical model in Fig. ((eftt).. For each domain k,
(1) Draw global domain index Bg from the Gaussian distribution (8))ax).
(2) For each data point with domain identity k:
(a) Draw local domain index u, from the Gaussian distribution Po (u, (B.)
(b) Draw input x; from the Gaussian distribution po po(x{(u))

---

Published as a conference paper at ICLR 2023
x, u.
px||u)
p(y(2)
p( u.
Reconstruct
Reconstructu
Prediction
((uxx)
(BIu))
2.
Regularizer
Figure 2: Network structure. For clarity, we omit subscripts of qo and po as well as po(2)x, x, u, 3)'s.
input (x, u).
(c) Draw data encoding 2; from the Gaussian distribution po(zi|x,, Uk (3,).
(d) Draw label yı from the distribution pe (y//z.).
Besides typical conditional dependencies defined in the graphical model (Fig. (left)), we enforce additional independence between ( and z; such independence is represented as dashed line "-.-" in Fig. 1(left). Note that there are multiple ways to satisfy such constraints during learning, c.g., using adversarial methods (Ganin al., 2016; Tzeng al., 2017; Zhang et al., 2019; Wang et al., 2020; Xu et al., 2022) and using the concentration loss (Xiao et al., 2021).
Generative Model and Inference Model. Based on Fig. I(left), factorize the generative model px(x, u. B, (x) into five conditional distributions (omitting the subscript for clarity below):
pa u, 3, z, a/ (3 px)p (u) pp) (x))) u, (ul2),
(I)
where 0 denotes the collection of parameters for the generative model, and pe (3) (M, o is Gaussian distribution. The predictor pe (y/ (2) is categorical distribution Cat(fy 0)) for classification tasks and a Gaussian distribution M( (z; 0), oz 0)) for regression tasks; here fy (2: 0), By (z; 0), and By (z; 0) are neural networks taking z as imput. Similarly, we have
po (ul ( (3: 0) (
pe (x|u) NN 0), o2
(3)
po (z x, u, (3) = (N u. (x))
We use an inference model 9o (u, B, z(x) to approximate the posterior distributions of the latent variables, i.e., (u B, z(x). As shown in Fig. 1(right), we factorize qo B, z/x)
go 1, 3, zzxxx (u)x))) qo ( (B) 3)
(5)
where & denotes the collection of parameters for the inference model. Specifically, have
go (u x) (x, o) (x: Φ)),
(6)
go (3 u) (us((u Φ) 03 x)),
(7)
qo (z/x, u, B) (N= u, 3: () 0f u. 3.):))).
(8)
Note that (=--) and (::-)) denote neural networks; 0, Φ are neural network parameters. Here (B(B))) u) requires special treatment and will be discussed in Eqn. (14-17) below.
3.4 OBJECTIVE FUNCTION
Evidence Lower Bound. We use an evidence lower bound (ELBO) as an objective to learn the generative and inference models. Maximizing the ELBO learns the optimal variational distribution (u, 3, z z/x) that best approximates the posterior distribution of the latent variables (including the domain indices) (u, B, x). Specifically, we have the ELBO as:
LELBO (x, y) uo (.x/x))))))))))) [po (x, u, B, 2, y/ E
3o(u,B.21x (u, 3, x|x)]-
(9)
With the factorization in Eqn. and Eqn. 5, we decompose the ELBO as (omitting a avoid clutter):
CELBO (x. v) = E (1o (x(u))
(10)
Ba [oyll)())))
(11)
E Eaglnir) (mm)) (3 log
(12)
B. ( K (x), (x9)] u1, feg (13)
4

---

Published as a conference paper at ICLR 2023
12
(c)
-20
(d)
Figure 3: (a) The Circle dataset (Wang et al., 2020) with 30 domains, with different colors indicating ground -ruuh domain indices. The first domains (in the green box) are source domains. (b) Ground- truth labels for Circle, with red dots and blue crosses as positive and negative data points, respectively. (c) Ground-truth domain graph for DG-15. We use 'red' and 'blue' to roughly indicate positive and negative data points in domain. (d) VDI's inferred domain graph for DG-15, with an AUC of 0.83.
Each term above is computable with our neural network parameterization (see the network structure in Fig. 2); for target domains Eqn. 11 is excluded. Below, we describe each term's intuition.
Reconstruct Data x from (Eqn. 10). go (u(xx) and pe (x u) aim to reconstruct data x using the inferred u, encouraging u to preserve as much information on x as possible.
(2) Predict Label y from Latent = (Eqn. 11). This term samples u, f3, and from go (u|x), go (3)u) and 4s (z|x, u. 3), respectively, and then uses z to predict y in pa (vl z), encouraging z to contain as much information on y as possible to maximize prediction performance.
(3) Reconstruct Local Domain Index u from B (Eqn. 12). Eqn. 12 samples and , from (u(x) and go (3( ()) respectively, and then uses the inferred /3 to reconstruct local domain index u in pe (3. encouraging 33 to preserve much information on and possible. (4) Regularize All Latent Variables u, B, z (Eqn. 13). Equ. 13 includes two KL divergence terms between the inference model (-) and the generative model po(-) as well as an entropy term for (o(u) x); they all serve as regularizers to prevent overfitting. For example, the first regularization term implies that ((3 u) should be close to the prior distribution p(3).
Global Domain Index .9 and Local Domain Index u. VDI uses a bi-level structure for domain indices: local domain index u E R and global domain index 3 E Rd,, Both u and , are low-dimensional compared to x E Bz i... By < Bg and B3 < Bz. The local domain index is low-dimensional vector (e.g.. : Ba = 4) containing domain information for high-dimensional data x (e-g.. an image with B2 = 256 x 256). The global domain index B an aggregation of all local domain indices u for data from the same domain. Note that different data points x; and x; in the same domain (k; = k2) have different local domain indices, i.e., u, +f u,, but share the same global domain index, i.e., Brg = 3ky VDI's final goal is to infer the optimal global domain indices { given only the data (x, Vi) and domain identities k; thereby providing better interpretability and domain adaptation performance. See Sec. K for more diseussion on B3 and u.
Difference between Domain Identities and Global Domain Indices G. Note that domain identities k are discrete values and therefore cannot describe rich relations (e.g., similarity and distance) among domains. In contrast, global domain indices 33 are continuous vectors and therefore contain much richer information that describes relations (e.g., similarity and distance) among domains (see Sec. 5 for empirical results). Our VDI assumes l is available and tries to infer (3.
Inferring Global Domain Indices qs (B)u) in Egn. 5. For each domain k, global domain index 33 should aggregate domain information of all data in this domain. We therefore propose to leverage local domain indices of all domain k's data points. U = [u,]a== E Rr, B., to infer the global domain index B,. Specifically, our process consists of four steps: (1) Grouping u, in Domain k. Group all local domain indices from the same domain into one local index matrix (set), i.e., Uz k E D (2) Pairwise Domain Distance. Calculate the Earth Mover's distance (EMD) (Rubner et al., 2000) between each pair of local index matrices (scts) Skj = fEMD( Uk. U,) E NxN, where Sk., is the EMD between domain À and j. (3) Raw Global Domain Indices. According to the pairwise domain distance matrix S, use multi-dimensional scaling (MDS) (Borg& & Groenen, 2005) to map each domain into Bg-dimensional space and obtain the
raw global domain index 3 E R RB, RBo,i i.e., [ (S) | (S)) k> (4) Final Global Domain Indices. Feed the raw index 3 into the inference neural network to obtain the variational
distribution N (n+ Φ), (3) x)) for the final global domain index B, E RBa, where is the
5

---

Published as a conference paper at ICLR 2023
inference network parameters. We summarize these four steps below:
Grouping u, in Domain
Uk [u.]k==k E DexB.
(14)
Pairwise Domain Distance:
= FEMD(UR U,lk= E RNNN
(15)
Raw Global Domain Indices:
8 (MDD((S) E R
(16)
Final Global Domain Indices:
32 (N, o (P)): E R
(17)
Discriminator with an Adversarial Loss. To enforce independence between 3 and z, i.e., Part (1) of Definition 3.1, we train an additional discriminator D with an adversarial loss while maximizing
the ELBO in Eqn. 9. The discriminator is neural network (-)) that takes z as imput and predicts the global domain index and domain identity k. Essentially, DC.) plays minimax game with the encoder inference network (o(z)x, u, 3): D-)) tries to reconstruct the global domain index B and domain identity k while the encoder qo (z|x, xx u, (3) tries prevent D(-) from doing so by generating domain- invariint encoding z. Denoting as RD the reconstruction loss, the discriminator loss Lo can be written as:
(z(z)xx
CD = (3,
(18)
In Sec. 4, we will prove that 3 is guaranteed to be independent of z if l: is independent of z. We therefore simplify Eqn. 18 into only classifying the domain identity and use the log-likelihood as LD.o:
LD.. = Ep(xx) Eug (x/x)) (x (log D(k
(19)
Final Objective Function. Putting Eqn. and Eqn. together, we have our final objective function:
max min LVDI max min Le.s AaCD.
0,0 D
0,6
max min (FXX.))CELBBX,,. v)] Bo (x)x) [og D(k[z)),
(20)
0.6
where Ad is a hyper-
whpeeh-prrmann parameter balancing two terms.
4 THEORY
Below we provide theoretical guarantees for VDI's objective function (Eqn. 20). We analyze the first term in Lemma 4.1 and the second term in Lemma 4.2, show that Eqn. 20 lower-hounds combination of mutual information terms (including [y;; 2). T(x; u, B, 2). and 7(z; ()) plus some constants in Theorem 4. and then show that one can learn domain indices 3 that satisfies Definition 3.1 when Eqn. 20's global optimum is achieved (Theorem 4.2). All proofs are in Appendix B.
We start by analyzing VDI's ELBO term (EIBBO y) in Eqn. 20 and proving that it is upper bounded by I(y: z) (x; u, 3, z) plus some constants in Lemma 4.1 below.
Lemma 4.1 (Upper Bound the ELBO of px(x. y)). The ELBO of po (x(x, y) is upper bounded by the mutual information among observable variables x. y and latent variables u, B, below:
Ep(x.r) (ELBOO(P (x, v))] < (1y; [xx; u, B,z) [M (x)).
(21)
Since the entropy terms (y) and (x) in Eqn. 21 are both constant, maximizing the ELBO term LELBO (x, y) Eqn. 20 is equivalent to maximizing T(x; u, 3,2 and T(y; 2), corresponding to Parts (2) and (3) of Definition 3.1, respectively.
Next we analyze VDI's adversarial term LD. of Eqn. 20 in Lemma 4.2 below.
Lemma 4.2 (Information Decomposition the Adversarial Loss). The global maximum of adversarial loss w.r.t discriminator D is decomposed as below:
max Ep(k,,x) (xxx) D(klz)] = 1(z;3) + [(z;k|B) (K),,
(22)
and the global minimum of maxp Ep(k.x) Be (x) log (kk|z)) is achieved if and only if [(z (3)
T(x; kil3) 0.

---

Published as a conference paper at ICLR 2023
Domuins
Leve-3 Torget Domuins
Lave-1 Taraet Domsins
Exset-2
Level3
Target Ilumaies
Tirget Domaiios
Figure 4: Domain graphs for two adaptation tasks on TPT-48; black nodes indicate source domains, and white nodes indicate target domains. Left: Adaptation from the 6 states in the west to the 42 states in the east. Right: Adaptation from the 24 states in the north to the 24 states in the south.
Lemma 4.2 above shows that one can decompose maxp xp [log (A into several information theoretic terms, including [(z; (3), which is related to Part (1) of Definition 3.1.
With Lemma 4.1 and I Lemma 4.2, we then show that VDI's objective function in Eqn. 20 lower-bounds a combination of mutual information terms plus some constant entropy terms in Theorem 4.1 below.
Theorem 4.1 (Objective Function as a Lower Bound). The objective function involves both the ELBO of po (x. y) and adversarial loss Expkk.) (x)x) [log D (1:(z)),, and it is the lower bound for combination mutual information and entropy terms:
Fx (EELBOX v)) max pp E. (2 Dog D [z)
(23)
S I(y: + [(x; 1, 3, [(z 3) [(z: k| (H + H()) Hk)]..
(24)
With Theorem 4.1, we Theorem 4.2 (Global Optimum of VDI). In VDI, when the global optimum (Eqn. 20) is achieved, is guaranteed that (1) [(2; 3) = 0, (2) [(x: u. B, z) is maximized, and (3) [[y; 2) is maximized.
ready analyze the global optimum of the minimax game in Eqn. 20.
As Theorem 4.2 states, the global optimum of Eqn. 20 guaranteed to satisfy all three conditions in Definition 3.1; therefore training VDI using the minimax game objective Eqn. 20 is equivalent to inferring the optimal domain indices.
5 EXPERIMENTS
In this section, we compare VDI with existing DA methods on both synthetic and real-world datasets.
5.1 DATASETS
Circle (Wang et al., 2020) is a synthetic dataset with 30 domains for binary classification. Fig. 3(a) shows 30 domains of Circle in different colors. Fig. 3(b) shows positive (red) and negative (blue) data points, The first domains are source domains, and the remaining 24 domains are target domains.
DG-15 and DG-16 (Xu et al., 2022), DG-15 (Fig. 3()) and DG-60 (Fig. 9()) are synthetic datasets with 15 and 60 domains for binary classification, respectively. In both datasets, we use connected domains the source domains and use others as target domains (see Table in Sec. F for details).
TPT-48 (Xu et al., 2022) is a real-world regression dataset that contains monthly average temperature for the 48 contiguous states in the US from 2008 to 2019. We use the first 6 months' temperature as model input to predict the next months' temperature. We formulate two DA tasks (Fig. 4):
W (6) + E (42): Adapting models from the 6 states in the west to the 42 states in the east.
N (24) + S (24): Adapting models from the 24 states in the north to the 24 states in the south.
We treat target domains hop away from the closest source domain as Level-1 Target Domains, those two hops away as Level-2 Target Domains, and those more than two hops away as Level-3 Target Domains (see Fig. 4 for an illustration).
CompCars (Yang et al., 2015) is a car image dataset with attributes including car types, viewpoints, and years of manufacture (YOMs). The task is to recognize the type given image. In CompCars, data with each view point and each YOM is treated as a single domain. We choose a subset of CompCars with 4 types (MPV, SUV, sedan and hatchback), 5 viewpoints (front (F), rear
7

---

Published as a conference paper at ICLR 2023
Table 1: Accuracy (%) on Circle, DG-15 and DG-60.
Method
Source-Only
DANN
ADDA
CDANN
MDD
SENTRY
D2V
VDI (Ours)
Circle
55.5
53.4
56.2
54.9
53.4
59.5
60.1
94.3
DG-15
39.7
43.4
33.5
38.8
37.2
42,6
79.9
94.7
DG-60
55.0
66.3
60.8
65.3
54.6
51.3
82.1
95.9
Table 2: MSE for various DA methods for both tasks W (6) (42) and N (24) - S (24) on TPT-48. We report the average MSE of all domains as well as more detailed average MSE of Level-1, Level-2, Level-3 target domains, respectively (Fig. 4). D2V cannot perform regression and thus has no results.
Note that there is only one single DA model per column. We mark the best result with bold face.
Task
Domain
Source-Only
DANN
ADDA
CDANN
MDD
SENTRY
D2V VDI (Ours)
Average of Lovel-1 Domains
1.844
1,984
5.448
6.168
5,544
2.512
2.160
Average of 6 Level-2 Domains
3.128
5.112
7.624
7.016
7.912
5.136
3,000
W (6)-EE (42)
Average of Level-3 Domains
5.272
5,880
7.256
6.968
8.008
5.872
2.448
Average of All 42 Domains
4.576
5,400
7,136
6.896
7.76
5,456
2.4%6
Average of 10 Level-1 Domains
1.688
1.832
5,872
1.832
2.736
3.976
1.536
N (24)-5 (24) Average of 6 Level-2 Domains
3.128
3,296
6.888
2,856
6.144
3,760
2.584
Average of 8 Level-3 Domains
9.280
6.744
7,088
7.688
10,608
3.672
5.624
Average of All 24 Domains
4,560
3,840
6.528
4.040
6,216
3,816
3.160
(R), side (S), front-side (FS), and rear-side (RS)), ranging from 2009 to 2014. contains 30 domains (5 viewpoints x 6 YOMs) with 18735 images in total. We choose the domain with front view and YOM 2009 as the source domain, and all the others as target domains.
5.2 BASELINES
We compared our proposed VDI with state-of-the-art DA methods, including Domain Adversarial Neural Networks (DANN) (Ganin et al., 2016), Adversarial Discriminative Domain Adaptation (ADDA) (Tzeng et al., 2017), Conditional Domain Adaptation Neural Networks (CDANN) (Zhao et al., 2017), Margin Disparity Discrepancy (MDD) (Zhang et al., 2019), SENTRY (Prabhu et al., 2021) and Domain to Vector (D2V) (Peng et al., 2020b). We also report results when the model is only trained in the source domains without adapting to the target domains (Source-Only). Different from VDI that works for both classification and regression tasks, MDD, SENTRY and D2V only work for classification tasks. We managed to adapt MDD and SENTRY for the regression tasks on TPT-48 (see App. H for details); D2V cannot be adapted and has no results for 7PT-48. Both Wang et al. (2020) and Xu et al. (2022) assume domain indices are available; therefore they are not applicable to our settings where the goal is to infer domain indices (which are unavailable from data).
5.3 RESULTS
Circle, DG-15 and DG-60. Table 1 shows the accuracy of evaluated methods on Circle, DG-15, and DG-60; all datasets have complex domain relations, making it challenging to perform domain adaptation without knowing ground-truth domain indices. Indeed, we observe that on Circle, all baselines only perform marginally better than random guess (50% accuracy). Moreover, on DG-15 most of baselines perform even worse than a random guess, possibly due to overfitting the source !omains!. In contrast, our VDI achieves very high accuracy (over 94%) on all three datasets, significantly outperforming all baselines, thanks to the inferred indices (e.g., Fig. 3(d) and Fig. 7).
To verify that VDI infers non-trivial domain indices B, we connect the domain pairs within distance threshold (I|,,- - 3, < e) to reconstruct the domain graph on DG-15 and DG-60. Compared with the ground-truth domain graphs, VDI achieves area under the ROC curve (AUC) of 0.83 for DG-15 and 0.91 for DG-60. Fig. 3(d) shows an example inferred domain graph for DG-15.
TPT-48. Table shows the mean square error (MSE) for all the methods on TPT-48. In terms of average MSE across all domains, we observe that most methods suffer from negative transfer on both tasks, with only DANN and SENTRY marginally improving upon Source-Only. In contrast, our VDI can further improve the performance and achieve the lowest average MSE on both tasks.
Note that different from (Wang et al., 2020; Xu et al., 2022), all evaluated methods only have access to domain identities k but ground-truth domain indices B, since the goal is to infer & in our setting.
8

---

Published as a conference paper at ICLR 2023
B (Colors Indicate Latitude)
(Colors Indicate Longitude)
4745'
2.0
5146"W
2.0
wi
Wi
1.5-
BID
1.5-
d40K
gx
Mss NVA
ORCO
GCT
MM
1.0
gx
JL
peço
GNT
MN
AR
MA
0.5
dbz
AR
MMA
MM
0.0
MS
DE
MD
MD
-05
JN
VA
PAAL
JN
00.5
FL
JN
PPAA
dil NVT
IN
-1.5-
-1.0-
GGA
MM
KY
OR
ME
NH
MO
SC
KY
ON
ME
dIH
-2.0
N)
NY
-2.-
JNY
2760N
12744'W
Figure 5: Inferred domain indices for 48 domains in TPT-48. We color inferred domain indices according to ground-truth indices, latitude (left) and longitude (right). VDI's inferred indices are correlated with true indices, even though VDI does not have access to true indices during training.
B (Colors Indicate Viewpoints)
B (Colors Indicate YOMs)
1.5
27
27
2009
1.0
37
1.0
2011
0.5
28
28
J1
2012
0.0
0.5
#9
24
20
33
19
24
20
33
2014
Front-side (FS)
20
24
29
25
Side (5)
J4
25
8
-15-
(RS)
8
Rear (8)
0.5 0.0
1.
1.0
1. 155
Figure 6: Inferred domain indices for 30 domains in CompCars. We color inferred domain indices according ground-truth indices, viewpoints (left) and YOMs (right). Numbers the circles are domain identities (discrete values). VDI's inferred indices are correlated with true indices, even though VDI does not have access true indices during training. See larger figures in App. L.
Table 3: Accuracy (%) on CompCars (4- (4-Way Classification).
Method
Source-Only
DANN ADDA
CDANN
MDD
SENTRY
D2V
VDI (Ours)
CompCars
39.1
38.9
42.8
41.8
41.4
41.8
40.7
43.9
Fig. plots the inferred domain indices B 2²2 for all 48 domains. For reference, we color the inferred domain indices according to ground-truth latitude (Fig. 5(cfft) and longitude (Fig. 5(right)); note that VDI does not have access to latitude and longitude during training. The plots show that VDI's inferred domain indices highly correlated with each domain's latitude and longitude. For example, Florida (FL) has the lowest latitude among all 48 states and is hence the left-most circle in Fig. 5(eftt. We also observe that states with similar latitude or longitude do have similar domain indices B. These results demonstrate that VDI can infer reasonable domain indices.
CompCars. Table shows the classification accuracy on all DA methods. Results show that most of the methods outperform Source-Oniz with our VDI achieving the most significant improvement. Fig. plots the inferred domain indices B E 2R22 for all 30 domains. For reference, we also color the plotted circles according to YOMs (Fig. 6(left)) and viewpoints (Fig. 6(right)); note that VDI does not have access YOMs and viewpoints during training. Interestingly, we have the following observations that are consistent with intuition: (1) domains with the same viewpoint or YOM have similar domain indices; (2) domains with "front-side" and "rear-side" viewpoints have similar domain indices; (3) domains with "front" and "rear" viewpoints have similar domain indices.
CONCLUSION
We identify the problem inferring domain indices as latent variables, provide a rigorous definition of "domain index", develop the first general method for addressing it, and provide detailed theoretical analysis as well as empirical results. We demonstrate the effectiveness of our proposed VDI for inferring domain indices and show its potential for significant practical applications. As a limitation, our method still assumes the availability of domain identities to identify different domains. Therefore it would be interesting future work to explore jointly inferring domain indices and domain identities.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENT
The authors thank the reviewers/AC for the constructive comments to improve the paper. ZX and HW partially supported by NSF Grant IIS-2127918. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed implied, of the sponsors.
REFERENCES
Shai Ben-David John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, (911-))11511155 2010.
Andreea Bobu, Eric Tzeng, Judy Hoffman, and Trevor Darrell. Adapting to continuously shifting domains. 2018.
Ingwer Borg and Patrick JF Groenen. Modern multiddimensional scaling: Theory and applications. Springer Science & Business Media, 2005.
Hong-You You Chen and Wei-Lun Chao. Gradual domain adaptation without indexed intermediate
domains. Advances in Neural Information Processing Systems, 34, 2021.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33:22243 22255, 2020.
Ziliang Chen, Jingyu Zhuang, Xiaodan Liang, and Liang Lin. Blending-targed domain adaptation by adversarial meta-adaptattin networks. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 2248-22222222 2019.
Shuyang Dai, Kihyuk Sohn, Yi-Hsuan Tsai, Lawrence Carin, and Manmohan Chandraker. Adaptation across extreme variations using unlabeled domain bridges. arXiv preprint arXiv: (rXiv:/906.02238, 2019.
Lucas Deecke, Timothy Hospedales, and Hakan Bilen. Visual representation learning over latent domains. In International Conference on Learning Representations, 2021.
Yuntao Du, Jindong Wang, Wenjie Feng. Sinno Pan, Tao Qin, Renjun Xu, and Chongjun Wang. Adarnn: Adaptive learning and forecasting of time series. In Proceedings of the 30th ACM international conference information & knowledge management, pp. 402 1022-11,200
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pp. 1800-1889. PMLR, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Franccois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. JMLR, 17(1)200962203), 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. corr abs/1512.03385 (2015), 2015.
Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding self-training for gradual domain adaptation. In International Conference on Machine Learning, pp. 5468-5479. PMLR, 2020.
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. Conditional adversarial domain adaptation. In NIPS, pp. 1647-1657, 2018.
Wang Lu, Jindong Wang, Xinwei Sun, Yiqiang Chen, and Xing Xie. Generalized representations learning for time series classification. arXiv preprint arXiv:2209.07027, 2022.
Massimiliano Mancini, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci. Adagraph: Unifying predictive and continuous domain adaptation through graphs. In CVPR, pp. 6568-6577, 6577,
Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, and Samuel Rota Bulo. Autodial: Automatic domain alignment layers. In Proceedings of the IEEE international conference on computer vision, pp. 5067-5075, 2017.
10

---

Published as a conference paper at ICLR 2023
Toshihiko Matsuura and Tatsuya Harada. Domain generalization using mixture of multiple latent domains. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp- 11749-11756, 2020.
Le Thanh Nguyen-Meidine, Atif Belal, Madhu Kiran, Jose Dolz, Louis-Antoine Blais-Morin, and Eric Granger. Unsupervised multi-target domain adaptation through knowledge distillation. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 1339-
1347, 2021.
Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. TKDE, 22(10):1345-135) 2009.
Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. Domain adaptation via transfer component analysis. TNN, 2292):199-210, 2010.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1406-1415, 2019.
Xingchao Peng, Yichen Li, and Kate Saenko. Domain2vee: Domain embedding for unsupervised domain adaptation. arXiv preprint arXiv:2007.0923. 099577 2020a.
Xingchao Peng, Yichen Li, and Kate Saenko. Domain2vec: Domain embedding for unsupervised domain adaptation. In Computer Vision-ECCC 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VI 16, pp. 756-774. Springer, 2020b.
Viraj Prabhu. Shivam Khare, Deeksha Kartik, and Judy Hoffman. Sentry: Selective entropy opti- mization via committee consistency for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3558-8567, 2021.
Alan Ramponi and Barbara Plank. Neural unsupervised domain adaptation in nlp a survey, arXiv preprint arXiv:2006. 20.02000000002,, 2020.
Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. Learning multiple visual domains with residual adapters. Advances in neural information processing systems, 30, 2017.
Eduardo Romera, Luis M Bergasa, Kailun Yang, Jose M Alvarez, and Rafael Barea. Bridging the day and night domain gap for semantic segmentation. In 2019 IEEE Intelligent Vehicles Symposium (IV), pp. 1312-1318. IEEE, 2019.
Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. The earth mover's distance as a metric for image retrieval. International journal of computer vision, 40(2):99-121 2000.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In CVPR, pp. 3723-3732, 2018.
Swami Sankaranarayanan, Yogesh Balaji, Carlos D. Castillo, and Rama Chellappa. Generate to adapt: Aligning domains using generative adversarial networks. In CVPR, pp. 8503-8512, 2018.
Baochen Sun and Kate Saenko. Deep CORAL: correlation alignment for deep domain adaptation. In
ICCV workshop on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV), pp. 443-450, 2016.
Sining Sun, Binbin Zhang, Lei Xie, and Yanning Zhang. An unsupervised deep domain adaptation approach for robust speech recognition. Neurocomputing, 257:79-81 87, 2017."
Onur Tasar, Yuliya Tarabalka, Alain Giros, Pierre Alliez, and Sébastien Clere. Standardgan: Multi source domain adaptation for semantic segmentation of very high resolution satellite images by data standardization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 192-193, 2020.
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv: 1412.3474 2014.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In CVPR, pp. 7167-7176, 2017.
11

---

Published as a conference paper at ICLR 2023
Hao Wang and Dit- Yan Yeung. Towards bayesian deep learning: A framework and some existing methods. IEEE Transactions on Knowledge and Data Engineering, 28(12):3395 3408, 2016.
Hao Wang and Dit-Yat Yeung. A survey bayesian deep learning. CSUR, 53(5):1-37, 2020.
Hao Wang, Naiyan Wang, and Dit- Yan Yeung. Collaborative deep learning for recommender systems. In KDD, pp. 1235-1244, 2015.
Hao Wang, Hao He, and Dina Katabi. Continuously indexed domain adaptation. ICML,, 2020.
Zehao Xiao, Jiayi Shen, Xiantong Zhen, Ling Shao, and Cees GM Snoek. A bit more bayesian: Domain-invariant learning with uncertainty. arXiv preprint arXiv:2105.0400 2021.
Zihao Xu, Guang- He Lee, Yuyang Wang, Hao Wang, et al. Graph-relational domain adaptation.
arXiv preprint wxxv:2202.006688 2022.
Linjie Yang. Ping Luo, Chen Change Loy. and Xiaoou Tang. A large-scale car dataset for fine-graine categorization and verification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3973-3981, 2015.
Feng Yuan, Lina Yao, and Boualem Benatallah. Darec: Deep domain adaptation for cross-domain recommendation via transferring rating patterns. arXiv preprint 1r0iv:1905. 2019.
Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael I Jordan. Bridging theory and algorithm for domain adaptation. arXiv preprint 19X4v:/904.058/0) 2019.
Mingmin Zhao, Shichao Yue, Dina Katabi, Tommi S. Jaakkola, and Matt Bianchi. Learning sleep stages from radio signals: A conditional adversarial architecture. In ICML, pp. 4100-4109, 2017.
Yang Zou, Zhiding Yu, B. V.K. Vijaya Kumar, and Jinsong Wang. Unsupervised domain adaptation for semantic segmentation via balanced self- traintraa In Proceedings of the European Conference on Computer Vision (ECCV), September 2018
A FORMAL DEFINITION OF DOMAIN INDEX
Given predefined variables of data x, label and domain, let ,3 be domain associated variable, i.e., in each domain it has the same value, and let be data associated variable, i.e., for each data point it can have a different value.
Definition A.1 (Orthogonal Information). For any domain associated variable B3 and data associ- ated variable u, we define their orthogonal information as the following:
2:2 B T(x: u. 2). (2) Label orthogonal information (B): [3(3 = max 22. 2.2 T(y; 2).
(1) Data orthogonal information (B, u): Ix(B,u) u) == max
Definition A.2 (Domain Index). We call (3, u) domain index if they maximize the sum of data and label orthogonal information. i.e., B, B,u+ argmax .. Ix(B', uz) + (B())..
Lemma A.1. By Definition A.2, Domain Index maximizes both data orthogonal information and label orthogonal information, i.e., if (B, u) is domain index, then [x u) = H(x) and I,( = I[y: x).
Proof. We prove the lemma above by first constructing a trivial domain index. Let B un be variables that are independent of x, i.e., B, xxx and un IIx; let Zg = x and hence [(x; zo) = (x). We have
[x un) 2 [(x; up, By. Zn)
(x) and (B,) 2 I(y: zo) I(y: x). For any domain index
(B, by Definition A.2, we have H (x) + I(y: < Ix Un) + I, < 12 u) (3) <
H (x) + I(y:x); therefore Ix u) + (3( = H(x) + x(y:)). Since ( u) S H(x) and
Ty (3) T(y; x), we then have (3, Hxx) and (Ø)) = x(y; concluding the proof.
12

---

Published as a conference paper at ICLR 2023
B THEORETICAL ANALYSIS
Lemma B.1 (Upper Bound of the ELBO of (x y)). The ELBO of po(x, y) upper bounded by the mutual information among observable variables and latent variables u, B, as below:
Ertx:a][CrL.mooooos y)) - [(: z) + [xx u, 3,z) [y) Hx)]
(25)
Optimizing the ELBO of (x, y) is equivalent to enhancing mutual information between label y and classification latent variable z and between data variable x and all latent variables u, B, z.
Proof. Firstly, we provide the ELBO of log px y) as below:
log Po (x y) = log /
(x, u, B, y3d
(x, u, 2. vo B, z|x)
log
dzdBdu
2o G. 3,,11,,2111 2/x)
Po u, 1,,3,,80 v.
log F.
qo (u, B, x|x)
px(x, u, B3, z, y)
2 Egg log
(u, |3 x
Po u, B, z) (x|u, 2)po 3.2) Lo u1, B, z/x)
=E,, flog
E, (z)lz) E, xx u, B, z)] 3.
Then we have:
CELBO(PO(X v)) Eg log (vl()) + Pog po 3.2) (L 3, 322 (26)
To help our analysis, we introduce a helper joint distribution of variables which is rxx, = p(x, y) This distribution has the following properties,
(1) r r(x, v) p(xx v). r(x) px)), r(u) p(y).
(2) r(zly, x) = -o (z) = r(z)x).
(3) x)) -((/x) r(y/xx)
(4) 4 z/x) = O. i.. y under distribution r. (implied by 2, 3).
The First Term of the ELBO. Its upper bound is provided as follows:
E Ep (z/x)) [log po [y]z))
= (x(x))) [og (oggo())))
Er(v.)) log po po lz)]
s Er(y.)) [log (y/ 2)]
r(y(z)
Er((..) p(v) p(y)]
r(ylz)
Er(y.e)(oog p(v)
+ E (lgp(v))
[,(1; (1
when loglo(ulz) = r(ulz), we have Ep(x,)) Bng
(x (po(
I,(iz)) + H(v). So.
maxp0 (ys(ex) log (z)lz = ((v:z) (y))
Then have:
E (eo())) (y)lz( & (v(1)))
H(y)
(27)
13

---

Published as a conference paper at ICLR 2023
Furthermore, we prove another upper bound of the first term:
Ep(x.y) (x1x)) [log Pe (u(1z)
- Eptx,e) Le (v/z)]
- Eptx,e) [log p(y)x)]
= -Epix,) [log P(u x
p(u)]
p(u)
= -v: - (v)
The first inequality takes the equal sign when (a(y)z) is constant w.r.t given æ, i.e, po (y)l,,x) Pe (y)()) The second equality takes the equal sign, when (a and pe (e(lz) as 9 (z/x) and (5 1)) when the optimality of 4s (z|x) and ps (ulz) is achieved, i.c.
p(y[x). We set gol (z(2)))
log (xs( = log p())x) = (x)x) log Po (B/22) Then we have:
Ep(x,y) log (x)( (1 [v]22) = Ep(x,v) Bog (alx)) [log PO (u(z)]
Exp(o) Rog (2 (x) Bpg R (et (elx)) [lg pā (vl2)]
-Ep(a) Bog (x) (2/ R pol Pog
z); H(y)
= =p(1xx
H (y)
Thus, we prove the relationship among the two forms of upper bound for the first term and the mutual information L. (y; w.r.t. the optimal (z(z(x) and pe z).
max E Eptx. R Rott po (y 2)] = (,( H(y)
max (xxx.) By(z(x) po po (v)z)) (u; H(y) In(us H1))
9.Po
Moreover, we prove the upper bound for I. (v; and (p(1; x):
(n x) = E E, (p(y))))))))))))))))))()))))l[ H (1) 50+
Finally, we have Ep(x,)) Egg (x)x)) po 2)] - I, (y: S (y; I, x) S H()).
The Second Term of the ELBO. We introduce another helper joint distribution of variables x, u, G which is s(x, u, B, z) = p(x) (u, 3. z/x). Its upper bound is proved as follows:
Bx(x,w) E, [og po X B3 2) Epcx) E.g [oloooo(((x (x B,))
Ep(x) Ey Po u, =
go (x Iu, B, p(x)po (x), B,
B. (lgg
p(x)
9 ( u, B,
-Ep()) llg 2o (x B, 2)
E Ex(x B.g [og)(x)) + Exp( E. log (x 3,
p(x)
(x(
(x u, 3,
-, (x) + E
blog
40 (x,
go (x u, B3
Dlo po(x|u1 u1 3, no (x 3, (, (1
1xx,11,,,2) (x)) + E
R 4: (u,3. (.,B.z)
90 (x
-1xx; u, 3,2) H (xx E (n,B.2)
K KL
< U, (,z) H(x) -0 where [x(x; u, 3, z) is w.r.. s(x, u, B. 2). i.e.,
Then have:
(x 3,2)] < U, H(x)
(28)
Applying Eqn. 27 and Eqn. 28 Equ. 26, we have:
CeL v))
= Ep log [2]2]] + (1(1))) + Ixxx B.z) [ [(x))
- E (.(,.B.2) [LLgg((( 2,2)pl B, 2)]
14

---

Published as a conference paper at ICLR 2023
For clarity, use (1; z) and I(x; u. 3. in place of (v; and [x; u, rrppcctivlly, later in the Appendix and the main paper.
Lemma B.2 (Information Decomposition of the Adversarial Loss). The global maximum of adversarial loss w.E.t discriminator D is decomposed as below:
max Bp( (m)x) [log (kk|z)] = [(z + [(z, - H(k)
(29)
and the global minimum of maxp Bp(k.x) (x) log (kkl))] achieved if and only if (z; (3
[zz, 13 0.
The optimization of adversarial loss is to reduce the mutual information between data encoding and domain identity while the mutual information is reduced between data encoding z and domain identity B.
Proof. Define
skk,x, = (1) s(k, B.
s(k 2) - (0((22)k))(kk) = (z)33 k) (31k)p(()) (H122))0(z) - p(k ( Bg (2/x)
Then we have:
BB (elx) [log D[k[2)] EE [log D(k/2)) S -Ea Dlo skk|z)
When the equality holds, the discriminator becomes optimal, i.e., D(k|z) = s(k|z)) Due to p(3)k) is function fkk mapping from & to B. Therefore, have:
p k) p(z),((),k)) p(z)k)
Thus, skk, B, z) ((1)))))) The three random variables meet the Markov chain 3 <- > z. By chain rule for mutual information, we have:
1(z: 1 (z, (3) (2, [zz;k( (,(2B11k)
where (2, 0. So.
T(2; k) = T(z;; 33 + k33)
We also have:
(kk22) 2)
Es(z, [log skk[z)] = (2) (log
9s (k) 4st (A:)]
s(k) z))
= E
+ (,(,, log (k))
9:.
=I(z; k)) - H (kc)
max Bp( (x/x) (log D(k|z)) = 1(2; k) (k)) = ((; ((2, H(:)
Therefore, min, maxp Epck, B [log D(klz)) [(z: (3) [zz, k((3) O.
H(k) while 1(z; k) and thus
Theorem B.1 (Objective Function Lower Bound). The objective function involves both the
ELBO of (x y) and adversarial loss E (mix))) [log D(klz)),, and it is the lower bound for
combination mutual information and entropy terms:
Ep(x.)) (Cr.s))CEL.BO y)] max Ey Dlo D z)]
(30)
< (() + (x: u, 3, z) (2z (z, ( (H (x)) [1)]..
(31)
Proof. Theorem B.I 3.1. is proved by applying Eqn. 25 mine Eqn. 29.
15

---

Published as a conference paper at ICLR 2023
Theorem B.2 (Global Optimum of VDI). In VDI, the global optimum (Eqn. 20) is achieved, if and only (1) [2: 3) kz = 0, (2) (x; u, 3,z) is maximized, and (3) I[y; z) is maximized.
Proof. The condition (1) gotten from Lemma B.2. The last two conditions (2) and (3) are gotten from Lemma B.I.
As Theorem B.2 states, the global optimum of Eqn. 20 guaranteed to satisfy all three conditions in Definition 3.1; therefore training VDI using the minimax game objective Eqn. 20 is equivalent to inferring the optimal domain indices.
C VISUALIZATION OF INFERRED DOMAIN INDICES FOR CIRCLE
Fig. 7 shows the inferred domain indices for Circle.
Normalized Estimated Domain Indices
1.0
0.8
0.6
0.4
0.2
0.0
0.0
0.2
0.4
0.6
0.8
1.0
Normalized True Domain Indices
Figure 7: Inferred domain indices (reduced dimension by PCA) with true domain indices for dataset Circle. VDI's inferred indices have a correlation of 0.97 with true indices, even though VD/ does not have access true indices during training.
D VISUALIZATION OF LOCAL DOMAIN INDICES FOR DG-15
Fig. 8 shows the local domain indices for DG-15. This inspired us to derive global domain index by Earth Moving distance.
16

---

Published as a conference paper at ICLR 2023
Local Indices for all domains in DG-15
Local Indices for selected domains ns DG-15
2-
12
11
-1.
-10
Figure 8: Local Domain Indices for DG-15. Left: Local domain indices for every domain. Right: Local domain indices for selected domain. We select each domain by their location in domain graph (see Fig. 9())
E
ARCHITECTURE AND IMPLEMENTATION DETAILS
E ARCHITECTURE
As is shown in Fig. 2, we use neural networks to estimate the density function of each distribution. For Circle, D5-15, DG-60 and TPT-48, we use multi-layer perceptrons for estimating q(u x), while for CompCars, we use ResNet-18 (He et al., 2015) for g(u(x). All the other neural networks are multi-layer perceptrons. To ensure robustness training, we fix the variance of some distributions and only use a neural network to estimate the mean of these distributions. All the input data are normalized with their mean and variance.
E.2 HYPERPARAMETERS
For experiments all 4 datasets, set the dimension of global domain indices to 2. For Circle, DG-15, DG-60, the dimension of local domain indices is 4, while for TPT-48 and CompCars, the dimension of local domain indices is 8. Our model is trained with 20 to 70 warmup steps, learning rates ranging from x 10-5 to x 10-1, and xa ranging from 0.1 to 1.
E.3 ADDITIONAL LOSS ON LOCAL DOMAIN INDICES
During the inference of local domain indices u. we incorporate modified contrastive loss from Chen et al. (2020) to improve the coherence of u. The loss aims to maximize the agreement between local domain indices of data points the same domain. Specifically, we sampled a minibatch of size from each of the N domains, resulting in a large batch of size bNN For each X4,i, the i-th sample in the minibatch of domain kt we pair with (x.(i++1) b. the neighbouring data point within the same domain. We denote corresponding un,. ü from 4s (u]x) (Eqn. 6). With multi-layer perceptron, we map Ug,i Uki to ha bags which are subsequently used calculate the contrastive loss as follows:
(i + 1) mod b, and refer the sampled pair as (Xk,s, Xk.). We then sample
exp(sim(hj hx.))/T)
Con
log
Σ Eb= 1xym exp(sim(he.s hm,))/7)
where sim(-..) :) is the similarity function between vectors, and 7 is the temperature. In practise, we use the cosine similarity and set 7
F DATASET SUMMARY
Table 4 summarizes the statistics for all the datasets used in experiments.
17

---

Published as a conference paper at ICLR 2023
Table Summary of statistics and settings different datasets.
Dataset
Number of Samples
Input Dim
Toy/Real
Adaptation Task
Circle
3,000
2
Toy
2- 2-Way Classification
DG-15
1,500
2
Toy
2- 2-Way Classification
DG-60
6,000
2
Toy
2- 2ay Classification
TPT-48
6,912
6
Real
Regression
CompCars
18,735
224 x 224
Real
4- 4-Way Classification
(a) DG-15
(b) DG-60
Figure 9: Visualization of the DG-15 (a) and DG-60 (b) datasets. We use 'red' 'blue' to indicate positive and negative data points inside a domain. The boundaries between 'red' half circles and 'blue' half circles show the direction of ground-trute decision boundaries in the datasets.
N
N
Z
a
β k
u
X
Dk
Dk
X
u
B
Figure 10: (Copied Fig. here for easier reference.) Left: Probabilistic graphical model for VDI's generative model. We introduce a new edge type, "---", to denote independence. B enforces independence between and 3x. i.e, p(z (x) p(z); note that this does not contradict B% + in our case. Right: Probabilistic graphical model for the VDI's inference model.
G DG-15 AND DG-60
For completeness, we show DG-15 and DG-60 in Fig. 9. We use 'red' and 'blue' to indicate positive and negative data points inside domain. The boundaries between 'red' half circles and "blue" half circles show the direction of ground-truth decision boundaries in the datasets.
DG-15 is synthetic dataset with 15 domains for binary classification. As shown in Fig. 3(c), these domains form domain graph (DG) of 15 nodes, with adjacent domains having similar decision boundaries. Each domain contains 100 data points. We use connected domains as the source domains and use others as target domains. Similarly, DG-60 is another synthetic dataset with 60 domains, each of which contains 100 data points. We use 6 connected domains as source domains and the remaining 54 domains target domains.
H ADAPTING MDD AND SENTRY FOR REGRESSION TASKS
For MDD, we simply replace cross-entropy loss with an Ly loss. For SENTRY, since it requires confidence scores during training, include another classification network that predicts whether the average temperature of next months will go or down compared to the previous months, thereby providing confidence scores.
18

---

Published as a conference paper at ICLR 2023
1 NEW EDGE TYPE FOR PROBABILISTIC GRAPHICAL MODELS
In Fig. 10, we introduce a new edge type, "...", to denote independence. Gr z enforces indepen- dence between and Bg. i.e, px) p(z), note that this does nor contradict Bx - 2, long as there are multiple paths between 3g and z. In this case, we have two paths between 3r and z, 3x z and Bx
As a simplified example, consider only a sub-graph in Fig. 10 with only three nodes Gx. u, and Essentially our algorithm tries learn two conditional dependencies p(u)33) and p(z) u). Adding dashed edge B - ensures that p(z(3x) = p(z), which is equivalent to p(z|3,) = p(z)))) v/ Sk.
which is equivalent to
f p(u133)p(2133. u)du = p(ul31)p(z1( u)du, vB.
(32)
is easy to see that Eqn. 32 only introduces an additional constraint, but does not contradict the learning of conditional dependencies p(ulBx) and p(z|3xx
Combining the ELBBO below
LELBO (x. y) E
(u, (21) (Po 2, [ -E (,, (go(u, 3, x/x)],
(33)
with the additional constraint in Eqn. 32, we have the following objective function
max0 Ep(x,) [e [po (u, x, z, y, Blax]] 3. (u, 996 B, 2 z|xi]
(34)
s.l. / po (2) u)du -aaao==ooouullllaaa po(u((8))po(2133 uddu, vB Bz.
(35)
which is equivalent to Eqn. 20.
DOMAIN IDENTITIES VERSUS DOMAIN INDICES
Compared domain identities, domain indices better capture the similarity and complex relations among different domains, thereby better guiding the adaptation process. This is also empirically verified in previous works such Wang et al. (2020); Xu et al. (2022). As a simple example, note that domain identities are one-hot vectors, the distance between any two domain identities are identical; in contrast, domain indices are real-value vectors and therefore contain richer information.
More specifically, since the encoder takes domain indices as additional input, domain indices tend to encourage data of similar domains to go through similar transformations to produce the encoding consequently, the predicted decision boundaries of similar domains will also be similar.
Theoretically, the target domain's 's error is upper-bounded by three terms, i.e., source error, domain gap. and optimal joint error of the source and target domains (Ben-David et al., 2010). Encouraging similar transformations for similar domains can reduce the joint error term of the upper bound, thereby achieving better performance.
K GLOBAL VERSUS LOCAL DOMAIN INDICES
Global versus Local Domain Indices. Local domain indices u contain instance-level information, i.e., each data point has a different local domain index u; in contrast, global domain indices contain domain-level information, i.e., all data points of the same domain share the same global domain index G.
Necessity of Local Domain Indices. Essentially, local domain indices u serve extremely low- dimensional summaries of representations z; for example, in CompCars experiments the numbers of dimensions for and 4 and 4096, respectively. With such cow-dimensional global domain indices can be efficiently inferred from u in the 4-dimensional space using Eq. (8-11). 1). In contrast, if one only uses global indices without u, one would need to compute the Earth Mover's distance
19

---

Published as a conference paper at ICLR 2023
(EMD) and multi diteesioann scaling (MDS) in Eq. (8-11) in the 4096-dimensional space; this is much more computationally expensive and dramatically slows down model training. Therefore local domain indices necessary in VDI.
Necessity of EMD and MDS. It is also worth noting that EMD and MDS are also necessary when inferring global domain indices (3 from local domain indices The main reason is that u's distribution tends to be multi-modal. As an example, we plot the local indices from 3 domains in DG-15 in Fig. 8 of Appendix D. We can see that the u's in Domain 11 contain clusters (i.e. two modes), with one cluster corresponding to positive data and the other corresponding to negative data. This also the case for both Domain and 0, but the distance between the two clusters gets smaller and smaller. Therefore, directly using the mean of u ''s as the global domain index / does not work because all three domains will have similar means, and consequently similar f's. Using EMD and MDS fixes this issue since EMD can naturally compute the distance between two multi-modal distributions. Our preliminary experiments also confirm their necessity; removing EMD and MDS will significantly bring down VDI's performance.
Independence between and the Global Index B. In typical domain adaptation methods, an encoder first takes x as input to produce the representation 2, and predictor then takes input to predict the label. To improve z's generalization across different domains, it is common practice (Ganin al., 2016; Zhao et al., 2017; Zhang et al., 2019; Tzeng et al., 2017; Wang al., 2020; Xu et al., 2022) to enforce independence between the domain index ,3 and representation z such that doeain- information is removed from Therefore the assumption/conssrrint of independence between 3 and z is natural.
Why Independence between and the Local Index is Not Required. As mentioned above, we need the local indices u to capture the multi-cluster structure in the data, with each cluster corresponding to local indices for data with the same label y. Since contains label-specific information. if we enforce independence between z and u, such label-specific information will be removed from u, making it impossible for capture the label-specific multi-cluster structure in the data.
How the Global Index /3 Indicates Different Domains from the Definition. One key property that makes sure the global index B can distinguish different domains is the second point of Definition i.e, information preservation of x. Specifically, maximizing mutual information [(x; u, 3, z) ensures that 3 contains as much information independent. Since we assume covariant shift exists (i.c., different domains have different distributions x), this property helps B to distinguish (indicate) different domains. This is also empirically verified by results in Fig. 3(d), Fig. 5, and Fig. 6, where VDI successfully inferred meaningful domain indices from different datasets.
possible under the constraint that , and is
L LARGER FIGURES
In this section, provide larger versions of figures for domain index visualization in the main paper.
20

---

Published as a conference paper at ICLR 2023
B (Colors Indicate Latitude)
47'5'N
2.0
NE WI
1.5
SD
ND
TX
JL
IA
NM
JD
WY MT
MN
1.0
LAOK
WA
NV
CA
ORCO
AR
0.5
UT
MA
AZ
0.0
MS
DE
MD
WV
-0.5
FL
PAAL
IN
TN
VA
RI NCCTVT
-1.0
SC
OH
MO
KY
ME
GA
-1.5
MI
NH
-2.
NJ
NY
27.60'N
Figure 11: Inferred domain indices for 48 domains in TPT-48. We color inferred domain indices according ground-truth latitude. VDI's inferred indices are correlated with indices, even though VDI does not have access to true indices during training.
β (Colors Indicate Longitude)
81"46'W
2.0
NE WI
1.5
SD
ND
TX
IL
IA
KS CA NV
NM
JD
WY MT
MN
1.0
LAOK
WE
ORCO
AR
0.5
UT
MA
AZ
0.0
MS
DE
MD
WV
-0.5
FL
PAAL
IN
TN
VA
RI NOSTUT
OH
-1.
SC
MO
KY
ME
GA
-15
MI
NH
-2.0
NJ
NY
120'44'W
-2
-1
2
Figure 12: Inferred domain indices for 48 domains in TPT-48. We color inferred domain indices according to ground-truth longitude. VDI's inferred indices are correlated with true indices, even though VDI does not have access true indices during training.
21

---

Published as a conference paper at ICLR 2023
B (Colors Indicate Viewpoints)
1.5
27
12
22
1.0
.77
21
26
?
16
8
0.5
23
28
L1
6
d
Front (F)
15
Front-side (FS)
Side (S)
-0.5
19
24
20
13
Rear-side (RS)
10
Rear (R)
1..0
14
29
25
-1.
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
Figure 13: Inferred domain indices for 30 domains in CompCars. We color inferred domain indices according to ground-trut truth viewpoints. VDI's inferred indices are correlated with true indices, even though VDI does not have access Irue indices during training.
B (Colors Indicate YOMs)
1.5
27
12
22
1.0
17
21
26
2
18
16
2009
0.5
28
6
23
11
2010
15
0.0
2011
-0.5
19
24
20
13
2012
2013
10
-1.0
2014
25
14
29
-1.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
Figure 14: Inferred domain indices for 30 domains in CompCars. We color inferred domain indices according to ground-truth years of manufacture (YOMs). VDI's inferred indices are correlated with true indices, even though VDI does not have access to true indices during training.
22


Reviews:
[{"review_id": "MD0q-L0H27Z", "reviewer": "Reviewer_YbH3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a domain adaptation (DA) method that, as an additional outcome, learns continuous domain indexes directly from multi-domain data and annotated domain identities. Experiments show that the learned indexes improve the predictive performance of the model in DA. The method relies on a graphical model to describe the joint distribution of inputs $x$, targets $y$, latent domain indexes $\\beta$ and $u$, and domain-independent latent codes $z$. Learning is performed by the maximization of the usual ELBO for the likelihood $p(x, y)$. In addition, domain-independent latent codes $z$ are enforced by an adversarial domain discriminator.", "summary_of_the_review": "This well-written paper presents solid contributions to the challenging problem of unsupervised multi-source, multi-target DA. Weaknesses 1. and 3. can be solved easily. Weakness 2. might be harder to solve, but I think that, even if Lemma A.1 cannot be proved and has to be removed, this result is not crucial for the paper. Thus, I recommend acceptance on the condition that the authors will address these concerns appropriately in their rebuttal and change the manuscript accordingly.", "strength_and_weaknesses": "Strengths:\n- The paper is well-motivated and its contributions are presented clearly.\n- The proposed method is technically sound and elegant, although some relevant details seem to be missing (see 'Weaknesses' below).\n- The methodology is well grounded by the presented theoretical results.\n- The experiments confirm the effectiveness of the method. In particular, the visualizations of the learned global domain indexes in Figures 5 and 6 are quite interesting.\n\nWeaknesses:\n1. As is apparent from section 3.1, the paper considers the unsupervised multi-source, multi-target DA problem. However, it is not entirely clear which terms in the loss (eq. (14)) are active for unlabeled target examples. I suppose it should be all the terms of the ELBO except for eq. (5), plus the domain discriminator loss, but this should be clarified. From a theoretical perspective, I think this would correspond to the maximization of the ELBO for $p(x)$ plus domain discriminator loss.\n2. In the proof of Lemma A.1, the authors claim that \"since $z_1 \\perp \\beta$ and $z_2 \\perp \\beta$, then $(z_1, z_2) \\perp \\beta$\", but this is not true in general. E.g. suppose $z_1$ and $z_2$ are Bernoulli r.v.'s and $\\beta = z_1  \\mathrm{xor}  z_2$. The authors should explain why the implication holds in their case or otherwise review the proof and reformulate the remainder of the theoretical analysis in case the Lemma cannot be proved.\n3. Architecture and implementation details are missing from the experimental section. This should be provided as supplementary material. Releasing the source code would also be beneficial for reproducibility.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "KDlLcfcRRaJ", "reviewer": "Reviewer_uSSJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper addresses the domain adaptation problem in VAE.\nThe authors proposed a method to explicitly estimate the domain index (latent variable) while the domain identity (ID) is known.\nThe domain index is modeled to be divided into global and local ones and estimated hierarchically. It is proved that the proposed method can optimally obtain them under certain assumptions with the optimization using the proposed loss. The proposed learning procedure imposes that the representation z is learned to be domain invariant.\nExperimental results on both synthetic and real-world datasets demonstrate that the proposed method performs much better than existing methods.", "summary_of_the_review": "The theoretical analysis is solid, but the assumptions behind that are not well justified, such as the independence of beta from representation z and the necessity of beta and u (global and local domain indexes). Clarity is low in general.\n\n================ Update: After the revision in the discussion phase, the assumptions are well justified. Clarity is improved. Thus, I upgraded my score to 6. In the final version, I hope that the authors include a brief description of the efficiency of introducing \\u into the main text, which is described in the paragraph \"Necessity of Local Domain Indices.\"", "strength_and_weaknesses": "*Strength\n- The authors provide a formal definition of domain index from the probabilistic perspective.\n- Their theoretical analysis justifies the proposed method.\n\n*Weaknesses\n- The assumption of the independence of beta from representation z is not justified well.\n- The introduction of the hierarchical modeling of global and local domains is not well supported and motivated. Is there any problem when we only use the global one?\n- The paragraph \"Inferring Global Domain Indices\" in Section 3.4 is hard to follow, and it is a different formulation from Eq.2, which is confusing. Is it impossible to use ELBO simply?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HE186QPNfMI", "reviewer": "Reviewer_eAJo", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Domain-Indexing Variational Bayes for Domain Adaptation\n\nThe manuscript proposes a domain adaption which can infer domain indices (continuous values encoding domain semantics) under the assumptions that domain identities are known but domain indices are not available. The authors first define (local and global) domain indices in the probabilistic viewpoint and then propose an adversarial variational Bayesian framework (the evidence lower bound to learn the approximate posterior distribution for the latent variables and the discrimination term to enforce independence between the global domain indices and the data embeddings) to infer the domain indices. They also provide a theorical analysis which shows that the optimal solution of the proposed objective function is guaranteed to satisfy the three conditions of the domain index they propose to define in the manuscript.  \n", "summary_of_the_review": "I think that the manuscript has overall high quality. ", "strength_and_weaknesses": "Strengths.\n1.\tI think that the problem the authors try to solve is a new (domain adaptation learning with domain indices) and interesting (inference of unknown domain indices) problem.\n2.\tI think that the manuscript is well written and organized. Although I did not check all the detailed derivations, the theoretical analysis well supports the proposed method.   \n\nWeaknesses\n1.\tThe manuscript does not include the analysis of the computational complexities of the proposed method.\n2.\tRegarding the experiments. I think it would be better to include a table summarizing the data sets, e.g., the input dimension, the number of samples. I also think that it would be better if the previous work, Wang et al. (2020) Xu et al. (2022), had been included in the comparison as all the datasets used in the experiments already include the domain indices. With the same reason, I think that the experiments did not really explain well about what advantages of using the proposed method in practice would be. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "0W9hbx4qp-A", "reviewer": "Reviewer_SAsR", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "- To alleviate the reliance on the domain index for domain adaptation, this paper proposes an adversarial variational Bayesian framework that infers domain indices from multi-domain data and provides insights on domain relations to improve domain adaptation performance.\n- The main contributions include\n    1. analyze and define what is an effective domain index\n    2. given the domain identities, infer the domain index as a latent variable from data by developing an adversarial variational Bayesian model.\n    3. provide the corresponding theoretical analysis.", "summary_of_the_review": "In general, the paper is well written and the idea is novel. However, there are still some issues that are required to be addressed as shown in weakness.", "strength_and_weaknesses": "- Strength\n    1. The paper is well-written and clearly presented.\n    2. The proposed method is well-motivated and is theoretically guaranteed.\n    3. The idea of automatically inferring the domain index is interesting.\n- Weakness\n    1. Some of the statements should be further justified. For example, why is leveraging the domain index better than domain identity? Though previous work can somewhat support the statement, it would be better to briefly but explicitly justify it again here.\n    2. The domain index is formally defined in section 3.2. However, the definitions and differences between the local index and the global index are not quite clear.  For example, the global index \\beta is required to be independent of z, but this is not required for the local index, and why? In addition, the definition does not show the global index is responsible for indicating different domains. Moreover, why we explicitly need both the global index and local index is also not well justified. \n    3. The global domain index is inferred based on the local index. Is it necessary to introduce the additional local index? Is it possible to infer the global domain index directly? More analysis and experiments are encouraged for supporting this.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%