Cooperative Distribution Alignment via JSD Upper Bound
Authors: Wonwoong Cho
Ziyu Gong
David I. Inouye
Venue: NeurIPS 2022 Accept (2022)
Keywords: Unsupervised dataset alignment
Invertible flows

Abstract: Unsupervised distribution alignment estimates a transformation that maps two or more source distributions to a shared aligned distribution given only samples from each distribution. This task has many applications including generative modeling, unsupervised domain adaptation, and socially aware learning. Most prior works use adversarial learning (i.e., min-max optimization), which can be challenging to optimize and evaluate. A few recent works explore non-adversarial flow-based (i.e., invertible) approaches, but they lack a unified perspective and are limited in efficiently aligning multiple distributions. Therefore, we propose to unify and generalize previous flow-based approaches under a single non-adversarial framework, which we prove is equivalent to minimizing an upper bound on the Jensen-Shannon Divergence (JSD). Importantly, our problem reduces to a min-min, i.e., cooperative, problem and can provide a natural evaluation metric for unsupervised distribution alignment. We show empirical results on both simulated and real-world datasets to demonstrate the benefits of our approach. Code is available at https://github.com/inouye-lab/alignment-upper-bound.

Cooperative Distribution Alignment via JSD Upper Bound
Wonwoong Cho" Purdue University cho436@purdue. edu
Ziyu Gong" Purdue University gong 1 23@purdue edu
David I. Inouye Purdue University?
dinouye@purdue. edu
Abstract
Unsupervised distribution alignment estimates a transformation that maps two more source distributions shared aligned distribution given only samples from each distribution. This task has many applications including generative modeling, unsupervised domain adaptation, and socially aware learning. Most prior works use adversarial learning (i.e., min-max optimization), which can be challenging to optimize and evaluate. A few recent works explore non-adversarial flow-base (i.c.. invertible) approaches, but they lack unified perspective and are limited in efficiently aligning multiple distributions. Therefore, we propose unify and generalize previous flow-based approaches under single non-adversarial framework, which we prove is equivalent to minimizing an upper bound on the Jensen-Shannoe Divergence (JSD). Importantly, our problem reduces to min- min, i.e., cooperative, problem and can provide a natural evaluation metric for unsupervised distribution alignment. We show empirical results on both simulated and real-world datasets to demonstrate the benefits of our approach. Code available at https:::///// // //ggitu.bb..... com/ homp/inooye--eee lab/al aaabbalignnn wwlggnennttupee upper bound.
Introduction
In many cases, a practitioner has access multiple related but distinct distributions such as agricultural measurements from two farms, experimental data collected in different months, or sales data before and after a major event. Unsupervised distribution alignment (UDA) the ML task aimed at aligning these related but distinct distributions in shared space, without any pairing information between the samples from these distrbibutions (i.e., unsupervised). This task many applications such as generative modeling (e.g., [I), unsupervised domain adaptation (e.g., 2 3), batch effect mitigation biology (e.g., g.g. and fairness-aware learning (e.g., (5))
The most common approach for obtaining such alignment transformations stems from Generative Adversarial Networks (GAN) 66, which be viewed minimizing a lower bound on the Jensen- Shannon Divergence (ISD) between real and generated distributions. The lower bound is tight if and only if the inner maximization is solved perfectly. CycleGAN [ maps between nwo datasets via two GAN objectives between the domains and cycle consistency loss, which encourages approximate invertibility of the transformations.
However, adversarial learning can be challenging to optimize practice (see e.g. because of the competitive nature of the min-max optimization problem. Perhaps more importantly, the research community has resorted to surrogate evaluation metrics for GAN because likelihood computation is intractable. Specifically, the commonly accepted Frechet Inception Distance (FID) 12 is only applicable to image or auditory data for which there exist publicly available classifiers
in part
"Equal contributions
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

trained on large-scale data. Moreover, the concrete implementation of FID can have issues due to seemingly trivial changes in image resizing algorithms [3]
Recently, flow-based methods with a tractable likelihood have been proposed for the UDA task 2 I4 [6]. Specifically, iterative flow methods [14 15] proposed alternative approaches to distribution alignment via iteratively building up deep model via simpler maps. AlignFlow [2] leverages invertible models make the model cycle-consistent (i.e., invertible) by construction and introduces exact log-likelihood loss terms derived from standard flow-based generative models that complement the adversarial loss terms. On the other hand, log- likelihood ratio minimizing flows (LRMF) [[6] use
invertible flow models and density estimation for distribution alignment without adversarial learning and define new metric based on the log- likelihodoooo ratio.
However, iterative flow models 14 15 do not explicitly reduce a global divergence measure and actually solve non-standard adversarial problems via alternating optimization (see [3]). AlignFlow [2 assumes that the shared density model for all distributions is a fixed Gaussian and lacks an explicit alignment metrie. LRMF [16) may only partially align distributions if the target distribution is not in the model class. Also, the LRMF metric is limited because it is only defined for two distributions and depends on the shared density model class.
To address these issues, we unify existing flow-based methods (both AlignFlow and LRMF) under common cooperative (i.e., a framework by proving that minimization over shared density model is a variational upper bound of the JSD. The unifying theory also suggests a natural domain-agnostic metric for UDA that can be applied any domain including tabular data (where FID is inapplicable). This metric is analogous to the Evidence Lower Bound (ELBO), i.e., it is variational bound that is useful for both training models and comparing models via held- out test evaluation. Furthermore, this unification enables straightforward and parameter-efficient multi-distribution alignment because the distributions share latent space density model.
We summarize our contributions as follows:
We prove that minimization over shared variational density model is variational upper bound on generalized version of JSD that allows for more than two distributions. Importantly, we theoretically quantify the bound gap and show that it can be made tight if the density model class is flexible enough.
Based on this JSD upper bound, we derive novel unified framework for cooperative (i.e., AUB metric and
aon-adversarial) flow-based UDA that includes novel explain its relationship to prior flow-based alignment methods.
Throughout experiments, we demonstrate that our framework consistently shows superior performance in both our proposed and existing measures on simulated and real-works datasets. We also empirically show that our model is more parametcr-efficient than the baseline models.
Notation We will denote distributions as (x) where X is the corresponding random variable. Invertible functions will be denoted by T(-). We will use X, Px, to denote the observed random variable from the j-th distribution. We will use Z, A (X,)) ~ Pz, = Pr,(X,) )) to denote the latent random variable of the j-th distribution after applying T, to X, (and note that X, T, (22)) )) We will denote the mixtures of these observed or latent distributions as Px... A ,, Px, and Pz.. E,u,P2, where is a probability vector. We denote KL divergence, entropy, and cross entropy as (-,-,,) H(-), and He(-.-), -), respectively, where KL(P, Q) H.(P, Q) H(P)
2 Alignment Upper Bound Loss
In this section, we will present our main theoretical result by proving an upper bound on the generalized JSD divergence, deriving our loss function based on this upper bound, and then showing that minimizing this upper bound results in aligned distributions assuming the model components have a large enough capacity.
Background: Normalizing Flows and Invertible Models. Normalizing flows are generative models that have tractable distributions where exact likelihood evaluation and exact sampling are

---

Pz
T
T2
P
Px
(r,(
(Ta(X))
Px2
(a)
(b)
Figure I: In our AUB framework, domain specific transformation functions (T) (-))
j=1 and a density
model Q are cooperatively trained to make the transformed representations be indistinguishable in the shared latent space, i.e., aligned. (a) 1-D example. By minimizing our proposed AUB loss, the transformation functions 7i and T2 are trained to map the corresponding distributions Px, and Px2 to latent distributions (x,( and PI,(X2) that have higher likelihood with respect to a base distribution Q. The density model Q. on the other hand, is trained to fit the mixture of the latent distributions (x)) and (Tz(=))- (b) Intuitively, the optimization process our method can be viewed as making the Q distribution tight around the mixture of latent distributions to increase the likelihood (i.e., MLE) while the transformation functions Ti and T2 are encouraged to expand to fill the latent space defined by Q. Eventually, the latent distributions and converge the same distribution, which means that they are aligned.
Flow models leverage the change of variables formula to create an invertible mapping such that (xx = (2( Jr(x) where Pz is known latent distribution and [x) is the absolute value of determinant of the Jacobian of the invertible map T. To sample from Px. one needs to first sample from the latent distribution Pz and then apply the inverse transform T T-1 Therefore, the key challenge in designing invertible models is to have computationally efficient inverse evaluation for sampling and Jacobian determinant calculation for training. Many approaches have been proposed by parameterizing mapping function T deep neural networks including autoregressive structures [8 19. coupling layers (20 21], ordinary differential squations(22) and invertible residual networks 23 24). Flow models can be then learned efficiently by maximizing the
possible 177.
likelihood of the given data.
Background: Generalized JSD. We remind the reader of the generalized Jensen- -hannonn divergence for more than two distributions, where the standard JSD is recovered if wi W2 0.5. Definition 2.1 (Generalized Sassen-Shanoon Divergence (GJSD) (25). Given k distributions {Pz, 1/ and a corresponding probability weight vector w, the generalized divergence is defined as (proof of equivalence in ??.):
GJSD Pz.) & E, (L E, = H (E,w,P2) H(P2,
2.1 GJSD Variational Upper Bound
The goal of distribution alignment to find a set of transformations (T) (which will be invertible in our case) such that the latent distributions align, i.e., Pr,(x,) (X,)) or equivalently Pz, Pz,, for all + j'. Given the properties divergences, this alignment will happen if and .P..) Thus, ideally, we would minimize GJSD directly, i.e.
only if GJJDD(P,,
min GJSD(Pr. ... .P.)).)) min H (E,w, E, H(Pr, TEET
T. FTe
where T a class of invertible functions. However, we cannot evaluate the entropy terms in Eqn. because do not know the density of Px,; only have samples from Px,- Therefore, will upper bound the first entropy term in Eqn. (H(),, Pr, using variational density model and decompose the other entropy terms via the change of variables formula for invertible functions.
Theorem 2.2 (GJSD Variational Upper Bound). Given an variational density model class Q, we form a GJSD variational upper bound:
(P, ... Pz.) S min Ho(P () -,,w, H(Pz,),
QEQ
where Pz A E,wpPa and the bound gap is exactly minqeo KL Q).
3

---

Proof of Thhoremm 2.2 GJSD (Pzz,". Pz,) = Ho Ho Q) Ho(( Q) + H(Pzaa)
For any Q Q we have the following upper bound:
- -E, w H(Pz,)
HP (m 1)-9QKKK9 KK- 7ane 1) E, H(Pz,)
- (Pm.: (pmm::91 Q) -,)- H(P2,),
where the first equals is merely inflating by Ho (F Q), the inequality is by the fact that KL divergence is non-negative, and the bound gap is equal to KL Q). The Q that achieves the minimum in the upper bound is equivalent to the Q that minimizes the bound gap, i.c., Q) Σ, , H(Pz,) = arg min Qeo (4'zai Q) H(Pz..)
Q' arg min QE (P arg min qee (I,(P Q), where the second equality because the entropy terms are constant with respect to Q and the last is by the definition of KI. divergence.
The tightness of the bound depends on how well the class of density models Q (c.g., mixture models, normalizing flows, or autoregressive densities) can approximate Pz... notably, the bound can be made tight if Pz... E Q. Also, one key feature of this upper bound is that the cross entropy term can be evaluated using only samples from Px, and the transformations Tj. i.e., He Q) = E, [- log (Tx) However, we still cannot evaluate the other entropy terms H H((z) since we do not know the density functions of P2, (or Px,). Thus, we leverage the fact that the T} functions are invertible to define an entropy change of variables.
Lemma 2.3 (Entropy Change of Variables). Let X ~ Px and Z A T(X) ~ Pz, where T is an invertible transformation. The entropy of Z can be decomposed as follows:
H(P2) = H(Px) + Exx (log (r(())],,
(2)
where Jr(()) is the absolute value of the determinant of the Jacobian of
The key insight from this lemma is that H(Px) is a constant with respect to T and can thus be ignored when optimizing T, while Ep [log (x)]] can be approximated using only samples from Px (formal proof in ??).
2.2 Alignment Upper Bound (AUB)
Combining Theorem 2.2) and Lemma 2.3, we arrive our final objective function which is equivalent to minimizing the variational upper bound on the GJSD:
,.,, ,P,)) < min Ho Q) - W, H(Pz,) QEQ
(3)
Σ, Wr,, log Q(T, (x)] - H(Px)), QEQ
min
(4)
where the cross entropy term is replaced by its definition provided above, and the last term -E, wy H(Px,) is constant with respect to Tý functions so they can be ignored during optimization. We formally define this loss function as follows.
Definition 2.4 (Alignment Upper Bound Loss). Given & continuous distributions {Pxx, 1 class of continuous distributions Q, and a probability weight vector w, the alignment upper bound loss is defined as follows:
LAUB (Ti, ... Th: (Px, 11 Q, w)
min E, WyyBErx [- log JJr, (2:) (=),
(5)
QEE
where T) are invertible and JJr, (z)] is the absolute value of the Jacobian determinant.
Notice that this alignment loss can be seen as learning the best base distribution given fixed flow models Tj. We now consider the theoretical optimum if we optimize over all invertible functions. Theorem 2.5 (Alignment at Global Minimum of LAUB). If LAUB minimized over the class of all invertible functions, global minimum of LAUB implies that the latent distributions are aligned, i.e. x() for all j + j'. Notably, this result holds regardless of Q.
Pr,(x,) = (x,
Informally, this can be proved by showing that the problem decouples into separate normalizing flow losses where Q is the base distribution and the optimum is achieved only if Prę(,,) = Q for all T₃ (formal proof in ??). This alignment of the latent distributions also implies the translation between any of the observed component distributions. The proof follows directly from Theorem 2.5 and the change of variables formula.

---

Corollary 2.6 (Translation at Global Minimum of LAUB). Similar to Theorem 2.5 global minimum of LAUB implies translation between any component distributions using the inverses of J i.e. (T,,)) = y for all j..
As seen in Alg. use simple alternating optimization scheme for training our translation models and variational distribution with cooperative (i.e., min-min) AUB objective, i.e., we aim to optimize:
min min Σ, "I Brx |- log Jry (z) Q(T)(x)))
(6)
Ti JRET QEQ
We emphasize that our framework allows any invertible function for T3 (c.g.. coupling-based flows 20. neural ODE flows (22), residual flows (3 and any density model class for Q: (e.g.. kernel densities in low dimensions, mixture models, autoregressive densities 26), or normalizing flows (2). Even VAEs (77 could be used where the log likelihood term is upper bounded by the negative
ELBO, which will ensure the objective still an upper bound of GISD.
Algorithm 1 Training algorithm for AUB
Input: Datasets {x,) ز for domains; n as the batch size; xy as a minibatch for the j-th domain;
normalizing flow models (zy; 0,) density model Q(2; Ø); learning rate ny maximum epoch Emax Output: {0)} 1=1 for epoch 1, Emaz do for each batch (r, } do
① + + 1) V o 2j=1 Σ log (1T
end for for each batch (-, ) do
Vj, 02 02 + V Σ log JJT (I
end for end for
AUB for UDA is like ELBO for density estimation. Although AUB and ELBO are for fundamentally different tasks, we would like to point out the similarities between AUB and ELBO. First, both are variational bounds of the quantity of interest where the tightness of the bounds depend on the optimization of the variational distributions. Second, both can be made tight if the class of variational distributions powerful enough. Third, both can be used to train model by minimizing the objective on training data. Fourth, while neither can be used as an absolute performance metric, they can both be used evaluate the relative performance of models on held-out test data. Thus, AUB can be used for UDA as BLBO has been used for density estimation with the same strengths and weaknesses such as being a relative metric and requiring an auxiliary model for evaluation.
3 Relationship to Prior Works
AlignFlow without adversarial terms is special case. As illustrated in Fig. 2 AlignFlow 2 without adversarial loss terms is a special case of our method for two distributions where the density model class Q only contains the standard normal distribution (i.e., singleton class). Thus, AlignFlow can be viewed as initially optimizing a poor upper bound on JSD; however, the JSD bound becomes tighter as training progresses because the latent distributions independently move towards the same normal distribution. By using the same architecture of T and Q, AlignFlow can be also viewed as sharing the last few layers of the T's; however, we note that our approach allows for Q that are not flows, e.g., autoregressive densities or mixture models as in our toy experiments that even have alternative non-SGD learning algorithms.
LRMF special case with only one transformation. As illustrated in Fig. fog-likelihood ratio minimizing flows (LRMF) [66] is also special case of our method for only two distributions, where one transformation is fixed at the identity (i.e., T2 Id). While the final LRMF objective a special case of ours, the theory is developed from a different but complementary perspective. The LRMF metric depends on the shared density model class, which enables zero point (or absolute value) of the metric to be estimated but requires fitting extra density models. Usman et al. [16] do not uncover
5

---

learnable
fixed Q
learnalee
T2
T2
X
Z
Z
X2
X1
X
X1
Z
X2
Getting
deesed
Getting "llsed
Getting
AlignFlow (MLE only)
LRMF
Ours
Invertible function
Fixed dist.
Changed dist.
Q dist.
Figure 2: These illustrations of flow-based alignment methods AlignFlow, LRME, and AUB demonstrate the differences between each setup. Transformation functions in AlignFlow are independently trained to map the distributions trained to directly map the source distribution Px, onto the target distribution Px,, i.e., T2 is the identity. The density model Q LRMF is not fixed and is trained to fit the mixture J P In our AUB setup, Ti and T22 are trained to map the source distributions Px, and Px, onto the shared Q distribution, while Q is trained to fit the mixture wy P2, + wzl'za. In every setup, the latent distributions Z1 and Zz move closer to the target distribution as training progresses. Details are provided section
fixed standard normal distribution. Ti in LRMF is
(a) Original
(b) LRMF
(c) AUB(ours)
(d) Original
(e) AlignFlow
(f) AUB(ours)
Figure 3: (a-c) LRMF, which only has one transformation T may not be able align the datasets if density model class not expressive enough (in this case Gaussian distributions), while using two transformations as in our framework can align them. (d-f) AlignFlow (without adversarial terms) may not align because Q is fixed to a standard normal, while our approach with learnable mixture of Gaussians for Q is able to learn alignment (both the same T3 models). Top row is latent space and bottom is the data translated into the other space.
connection of objective as an upper bound on JSD regardless of the density model class Additionally, to ensure alignment, LRMF requires that the density model class includes the true target distribution because only one invertible transform used, while our approach can theoretically align even if the shared density model class is weak. In other words, bound holds regardless of the model class Q, whereas the LRMF-JSD discussion further assumes that the target distribution needs to be in the Q model class (see Theorem 2.5 and our simulated experiments).
4 Experiment
We analyze the performance of our proposed framework comparing the relevant flow-hased baseline models. In jsubsection 4.1, we experiment on toy dataset to clearly show the benefit our method over the bascline flow-based models controlled environment. In Insuubsettion 4.2 aand/subsection 4.3 we demonstrate AUB's superiority over baseline flow-based models on real-world datasets including tabular data and high-dimensional MNIST data. In subsection 4.4, additionally conduct Domain Adaptation 2 28 experiments for validating AUB alignment in the context of downstream task. Implementation details are provided in ??.
4.1 Toy dataset comparison with related works
Single T Double T (LRMF vs. Ours). We first compare method with LRMF [66 method. The task is to translate between the two half-circle distributions (i.e., "moons"). We compared
'LRMF did discuss a connection with JSD but only as "biased estimates of JSD", rather than theoretic upper bound of ISD.

---

our AUB setup with two maps T1 and T- to the LRMF setup with only Ti where Q is the set of independent Gaussian distributions. As illustrated Fig. 3 the LRMF method fails transform between X1 and X2. Even though Q can model the transtormed data Ti (X1), a Gaussian- cannot fully model the half-cirele distribution of X2. Therefore, LRMF fails to transform between two distributions. However, in the AUB setup, both latent distributions of Ti (X,) and (x(x)) can be modeled by the same Gaussian distribution Q because of the flexibility in both transformations, which leads to better translation results in this illustrative example. In conclusion, the performance of LRMF is limited by the choice of density models Q; Q fails to model the target distribution, distribution alignment may not be achieved.
Q
Simple Fixed Q vs. Learnable Q (AlignFlow vs. Ours). Next we compare our AUB setup with the AlignFlow 2 setup. The task to translate between two random Gaussian mixture datasets (i.e., "blobs"). We compared our AUB setup where Q is mixture of Gaussians with the AlignFlow setup where Q a fixed standard normal distribution (the models for Ti and 72 are the same). As illustrated in Fig. 3 the AlignFlow method fails to transform between X1 and X2, because the transformed dataset Ti (X)) and (X2) failed to reach the fixed standard normal distribution Q. However, in the AUB setup, the shared density model Q adapted to the distributions of Ti (X1) and T2(X2) to enable a tighter alignment bound and thus the translation results are better. In conclusion, the performance of the AlignFlow model is limited by the power of the invertible functions.
4.2 Unsupervised Distribution Alignment on Tabular Datasets
To showcase the application-agnostee AUB metric and the parameter efficiency of our AUB framework, we conduct two experiments real-world tabular datasets. In both experiments, we used four UCI tabular datasets [29] (MINIBOONE, GAS, HEPMASS, and POWER), following the same preprocessing as the MAF paper [30. Train, validation, and test sets 80%, 10%, and 10% of the data respectively. Also, the experiments are measured by test AUB defined in Definition 2.4] where lower AUB score indicates better performance (see end secciin 2.2 for discussion on using test AUB for evaluation). We emphasize that there is no natural metric for evaluating GAN-based alignment methods on tabular datasets. Thus, these experiments demonstrate one of the key benefits of our proposed framework over a GAN-based approach.
Our first experiment designed to compare alignment performance between our proposed method and the baseline methods. To separate each dataset into two distributions, we choose the last input feature from each dataset and discretize it based whether it is higher or lower than the median value, which ensures the datasets are of equal size. Given the divided dataset, two transformation functions (T) are trained to align the distributions. For baselines, we use AlignFlow MLE, Adv. only, and hybrid versions and LRMF on top of the original implementation. Because AlignFlow Adv. and hybrid setups optimize over mixed objective of AUB (special case) and adversarial losses, to be fair, we additionally fit an identity-initialize Q models wherever possible across all methods (e.g., the same Q is used for LRMF and AUB but AlignFlow has a fixed Q)..
flow model Q to the final T's. We use the same T and
As shown in Table our method shows better performance compared to other methods across all datasets. In particular, because ours and LRMF (trained with learnable Q) outperform AlignFlow MLE (trained with fixed Q). can see that learnable plays an important role in distribution alignment. We also observe that ours shows better performance than LRMF, where the gap may come from aligning in the shared latent space (ours) rather than aligning in the original data space (LRMF). AlignFlow Ady. only and hybrid versions show worse performance than ours with a large gap, which implies our proposed cooperative training is competitive to adversarial methods in aligning distributions of tabular data.
The second experiment demonstrates alignment between 8 distributions and is designed to show that our proposed method can be more efficient in terms of model parameter compared to the baseline methods. For this multi-distribution experiment, we separate each dataset into 8 domains by choosing the last three features from a given dataset and dividing the dataset by the three medians, e.g., (+++ + +) + -), (- -). The results shown in Table 2 where RealNVP (5) and RealNVP (10) indicates the number of coupling layers used in the RealNVP architecture respectively. LRMF is excluded because LRMF is only designed for aligning two distributions.
Comparing AlignFlow RealNVP (5) and RealNVP (10), we can see that AlignFlow can achieve better alignment with more than double amount of parameters. However, as can be observed in
7

---

Table 1: Our method outperforms baselines in terms of AUB score (in nats, lower is better) for the domain alignment task on four tabular datasets. Numbers below each dataset indicate the number of features for the dataset.
Table 2:
Our AUB method may be more parameter efficient than AlignFlow especially for the multi-distribution setting (8 distributions in this case) where the number of parameters (in millions) and AUB score are show below.
MINIB
GAS
HEPM
POW
#
#Q
Total
AUB
(42)
(7)
(20)
(5)
AF (MLF)
1.46
0
1.46
10.16
LRMF
12.79
-6.17
18.49
-0.93
RealNVV (5)
AF (MLE)
14.08
-6.52
19.37
-0.77
AF (MLE)
4.54
0
4.54
19.85
AF (Adv. only)
18.18
-3.15
21.70
-0.39
T: RealNVV (10)
AF (hybrid)
19.49
-3.76
21.42
-0.43
Ours
Ours
12.11
-7.09
18.26
-1.19
T: R.R.( Q: R.(10)
1.46
0.57
2.03
18.82
comparison between AlignFlow with RealNVP (10) and ours, our proposed method can achieve similar performance with the baseline model but with less than half the parameters. These two results suggest that our model can be more parameter efficient than AlignFlow for multi-distribution alignment. We hypothesize that our approach may scale better with respect to the number of distributions because our Q shares parameters across the distributions and can capture the similarities between distributions.
4.3 Unsupervised Distribution Alignment on MNIST Dataset
perform an image translation task on MNIST datase to demonstrate that our distribution alignment method can be applied to high-dimensional datasets and to validate our AUB metric in case where FID is also available. We train ours and baseline models with the digit images of O, 1, and 2, and compare the translated results both quantitatively and qualitatively. Specifically, RealNVP invertible models for all translation maps Tj. as well as the density model Q. Note that all methods are flow-based models and thus images translated back to the original domain are exactly the same, which implies exact cycle consistency.
use
As represented in Table 3, both of our approaches outperform the baseline models in terms of FID and AUB. The lower AUB score indicates our method seems to align the distributions better than baselines We believe this result comes from our model setup, i.e., learnable shared density model and transformation distribution as their Q obtains worse AUB because the Q is not powerful enough to model the complex shared space trained from the real world dataset. On the other hand, LRMF shows the lack of stability when trained with the relatively simple models that RealNVP T and RealNVP Q. We expect this caused by the restrictions of only using one for translation without shared latent space and the fact that the Q distribution must be able to model the target distribution to ensure alignment.
5
shared latent space. Specifically, AlignFlow with a fixed standard normal
using across all methods, i.c.
Table This table FID (top) and AUB (bottom) scores for the three pairwise MNIST translation tasks show that our method has overall better performance than baselines across both metrics. FID score for each translation task is calculated by averaging scores from each direction and AUB score is shown nats. AF short for AlignFlow.
0
0 0
0
2
2
2
0
0
1
0 2
2
/ 2
0
O
/
2
0
2
2
0 2
Figure 4: Qualitative translation results among MNIST digits 0-2 show that our method has better translation results than baselines where in each block the first column is the original digit, the second is the latent image, and the last three translated results. Each row from the top indicates LRMF, AlignFlow, and ours respectively.
06+11
0x+2
1e+2
Avg.
AF (MLE)
38.90
71.17
61.33
57.13
LRMF
224.02
141.70
182.31
182.68
Ours
31.26
43.29
41.55
38.70
AF (MLE)
-4797
-4504
-4834
-4711
LRMF
-713
-592
-1323
-876
Ours
-4824
-4555
-4662
-4747
'under the terms the Creative Commons Attribution Share Alike 3.0 license 'While lower AUB scores may only mean tighter upper bound, as with ELBO (see section 2.2. upper bound generally means a better model.
lower

---

The better quantitative performance of our methods can be corroborated by the qualitative results as seen in Fig. 4 AlignFlow shows less stable translation results than our method especially for translating to digit '2' from digits '0" and 'I'. The second column of LRMF is set to be black because does not have latent representation, and LRMF fails to translate this situation, which is why the translated results are nearly the same. We believe this phenomenon comes from the lack of expressivity of their Q model. On the other hand, our model shows stable results across all translation cases, which are also quantitatively verified via the lower FID score. Note that our method has shared space, so different transformation functions for each domain are trained together. This shared space may provide advantage in terms of sample complexity and computational complexity compared to multiple independent flows as in AlignFlow. Similarly, we conducted experiments on a ten-domain translation in ?? to illustrate that our method with shared space can be easily scaled to more domain distributions with less number of parameters than the baseline model. Additionally, we visualize an interpolation over the shared space from the ten-domain experiment. Detailed descriptions and the results are provided in ??
4.4 Domain Adaptation on USPS-MNIST dataset
We additionally conducted a domain adaptation (DA) experiment between MNIST to USPS for exterally validating our alignment performance compared to baseline methods. We first reduce the dimensionality of both datasets to 32 using a pretrained variational autoencoder (VAE), which is trained jointly on both MNIST and USPS images without any label information (i.e., an unsupervised VAE). We then learn a translation maps in this latent space via our method and baselines. Finally, following the typical DA evaluation protocol, we train the classifier with source domain data and evaluate the performance by applying the classifier the target domain data translated to the source domain. Image translation results can be obtained by forwarding the latent translated results to the pretrained decoder.
As seen in Table 4| our method (right column) performs better than baseline methods. This result is further empirical evidence that our novel cooperative training can be comparable to adversarial training in certain cases. As shown in Fig. 5) adversarial loss shows mode collapse, e.g., AF AAdd. only), AF(1e-2). without careful hyperparameter tuning. The other thing to note is AF (MLE) works for domain alignment itself (i.e., the translated result is in MNIST domain), but it does not maintain the class information (i.e., the digit changes from USPS to MNIST). We conjecture that this issue stems from AlignFlow's simple and fixed Q distribution where arbitrary rotations of the latent space have equivalent likelihoods throughout the whole training process and thus class information is lost while transforming to and from such Q distribution. On the other hand, our learnable Q guides the alignment training process such that class information is partially preserved during transformation. LRMF failed to align since the Q distribution is not complex enough to model the density of the target distribution. This is the consistent result with toy dataset experiment (Fig. 3 (a)-(c)). LRMF does not work well with the simple Q while ours can align two different distributions with a simple Q because of the shared space. Please note that all models in the table approximately the same number of model parameters. Further comparisons are provided in ??.
Table 4: Test classification accuracies for domain adaptation from USPS MNIST. The number associated with AlignFlow (AF) is the coefficient of MLE term in its hybrid objective. Adversarial methods (i.e, AlignFlow Hybrid/Alignnlow Adv. only) are set to stop at 200 epochs.
LRMF AF(Adv.nll)
AF(le-2)
AF(Ie-I) AF(ee)
AF(el))
AF(MLE)
AUB(ours)
Accuracy 12.7%
11.1%
16.6%
63.8%
68.6%
35.7%
22%
77.5%
5
Discussion
Pros and cons of our method compared to adversarial method. Flow-based methods have different benefits and limitations compared to adversarial methods for distribution alignment. As one clear difference, our framework provides an rnplication-annostt yet theoretically grounded evaluation metric for comparing alignment methods. Additionally, our min-min problem is fundamentally different than a min-max problem and avoids issues unique to min-max problems (as seen in Fig. 6). However, a key limitation of our approach compared to GANs that is restricted to invertible models. Also, our method requires training a density model, whereas GANs train discriminator. Overall, we suggest our alignment approach a feasible and fundamentally different alternative to adversarial, which is currently the only dominant approach. With our foundation, future work could

---

00000600
0000
0600
/
2
0. 3.
f
2
2
5 02555.0 5.55
7 4
5
6
5
6
335 6
55
6 6 6 6
4
6
6
9
4 +
2772
2 8 2 88
9
9 9
7 7999
4 9
7 9
9
9 9
USPS
LRMF (limited density distribution)
AF (Adv. only) (mode collapse)
AF (MLE) (failed to maintain
AF (1e0)
AUB (ours)
[after carefully tuned)
class label)
Figure 5: Samples of translated images from USPS dataset.
The leftmost image shows the original
digits from USPS: all remaining images are the translated digits MNIST domain.
focus on the performance aspects just as adversarial learning has been improving over the past seven years but is still an active area of research.
(a) Discriminator loss for (b) Validation metric for AlignFlow
(c) Training loss for AUB(ours)
(d) Validation loss for AUB(ours)
AlignFlow
Figure 6: The losses for AlignFlow with adversarial learning oscillate and are unstable during training, while our AUB losses show show smooth convergence because of cooperative (i.e., min-min) training.
Alternating minimization against vanishing gradient problem. As revealed in previous works [16 32), the vanishing gradient problem can occur while minimizing a JSD approximation. On the same line, AUB can also suffer from vanishing gradient if the distributions are disjoint, and Q is fit very well. However, any Q that does not match Pzmiz provides an upper bound that we conjecture smoother than the true JSD. Thus, in experiments, we have found that alternating minimization does not suffer from vanishing gradients because Q is not fully fitted at each step but provides smooth upper bound. Yet, deeper theoretic and empirical analysis needed in future work fully understand this case.
Extensibility. As mentioned in section and sscciin 2. our proposed idea is general framework that can be used with any invertible flow models (e.g., Residual Flows (23), Flow++
and any density model (e.g., PixelCNN++ [26]. FFJORD (22)). Hence, we argue our proposed idea is not limited specific flow-based model or density model. We expect our framework could show better components.
performance in distribution alignment if it is combined with the aforementioned state-of tatte-of-ther the-arr
Conclusion
In this paper, we propose novel variational upper bound on the generalized JSD that leads to theoretically grounded alignment loss. We then show that this framework unifies previous flow-based distribution alignment approaches and demonstrate the benefits of our approach compared to these prior flow-based methods. particular, our framework allows a straightforward extension to multi- distribution aligmment that could be more parameter efficient than naïvely extending prior approaches. More broadly, we suggest that our AUB metric can be useful as an peplication-agnonn metric for comparing distribution alignment methods (analogously to how ELBO is used to evaluate density estimation methods). An alignment metric that is not tied to particular pretrained model (as for FID) or a particular data type will be critical for systematic progress in unsupervised distribution alignment. We hope this paper provides one step in that direction.
The authors thank the anonymous reviewers for their insightful comments and helpful discussion that improved the final paper. The authors acknowledge support from ARL IN9 and NSF (2212097).
Acknowledgement.
10

---

References
[1] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223-2232, 2017.
[2] Aditya Grover. Christopher Chute, Rui Shu, Zhangjie Cao, and Stefano Ermon, Alignflow: Cycle consistent learning from multiple domains via normalizing flows. In AAAI, 2020.
[3] Lanqing Hu, Mcina Kan, Shiguang Shan, and Xilin Chen. Duplex generative adversarial network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
[4] Laleh Haghverdi. Aaron TL Lun, Michael Morgan, and John Marioni. Batch effects in single-cell ma-sequencing data are corrected by matching mutual nearest neighbors. Nature biotechnology. 36(5): 421-427, 2018.
[5] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In Sanjoy Dasgupta and David McAllester, editors, International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 325-333, Atlanta, Georgia, USA, 17-19
Jun 2013.
[6] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, 2014.
[7]
Mario Lucic, Karol Kurach, Marcin Michalski, Sylvain Gelly, and Olivier Bousquet. Are gans created equal? a large-scale study. In Advances neural information processing systems, pages 700-709, 2018.
[8] Karol Kurach, Mario Lucic, Xiaohua Zhai, Marcin Michalski, and Sylvain Gelly, The GAN landscape: Losses, architectures, regularization, and normalization, 2019. URLLastpp://oppnzeeiiee http://wwwweeeeoorrreeeeee Id=rkGG6s0qKQI
[9] Farzan Farnia and Asuman Ozdaglar. Do GANs always have Nash equilibria? In Hal Daumé III and Aarti Singh, editors. Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages
PMLR, 13-18 Jul 2020, URL
https://proceedding. mlr,
html
[10] Weili Nie and Ankit B Patel. Towards a better understanding and regularization of gan training dynamics. In Uncertainty in Artificial Intelligence, pages 281-291. PMLR, 2020.
[11] Yue Wu, Pan Zhou, Andrew Wilson, Eric Xing. and Zhiting Hu. Improving gan training with probability ratio clipping and sample reweighting. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors. Advances in Neural Information Processing Systems, volume 33, pages 5729 5740. Curran Associates, Inc., 2020. | 0ale:33666a55d997755566666
https: httpss////cccoooon...e.e hwtp://gg...iiir.ccc..cccco cctp:/ppp.ccppppe///0000.. 11d999addffaa a88 Paper. pdf
[12] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a time-scale update rule converge to local nash equilibrium. In Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL htttps://proceeeee cc/ cc/paper/ 2ttps//0111ffffl 8a 06947777777677 ebOf wwweeefeeeeeee808 16911399267772228 Paperr. .dd
neurips
[13] Gaurav Parmar, Richard Zhang, and Jun-Yan Zhu, On buggy resizing libraries and surprising subtleties in fid calculation. arXiv preprint arXiv:2 11222, 2021.
[14] Biwei Dai and Uros Seljak. Sliced iterative normalizing flows. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 2352-2364. PMLR, 18-24 Jul 2021. URL. https: p/tpe///i/ooeecossoo/o
mlr. wwwpppeeaaaaae.p v139/ daiila.lla..... .htll
[15] Zeyu Zhou, Ziyu Gong, Pradeep Ravikumar, and David Inouye. Iterative alignment flows. In International Conference on Artificial Intelligence and Statistics, pages 6409-6444 6444. PMLR, 2022.
[16] Ben Usman, Avncesh Sud, Nick Dufour, and Kate Saenko, Log- likeliikld ratio minimizing flows: Towards robust and quantifiable neural distribution alignment. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balean, and H. Lin, editors, Advances Neural Information Processing Systems, volume 33, pages 21 118- 21129. Curran Associates, Inc., 2020. URL Paper pdf
hatppeer/222//11e1 file/
[17] Ivan Kobyzev, Simon J.D. Prince, and Mareus A. Brubaker. Normalizing flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11); 3964-3979, Nov 2021. ISSN 1939-3539, doi: 10. 1091ppam..20002 2992944.. URL http://p://dt doi org/ 10. 1w..9/99TAAII. 2020.02222222 2992934

---

[18] Durk P Kingma, Tim Salimans, Rafal Jozefowicz. Xi Chen, Ilya Sutskever, and Max Welling, Improved variational inference with inverse autoregressive flow. In Advances in neural information processing systems, pages 4743-4751, 2016.
[19] George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanaa. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1 64, 2021. URL http:://// http:://jmmlr.rrppppppppeeee wttggpaaper//222115 1028. 1028.html
[20] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. In International Conference on Learning Representations, 2017.
[21] Durk P Kingma and Prafulla Dhariwal. Glow; Generative flow with invertible Ixl convolutions. In Advances in neural information processing systems, pages 10215-10224, 2018.
[22] Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, and David Duvenaud. Scalable reversible generative models with free-form continuous dynamics, In International Conference on Learning Representations 2019. URL htt..ttttp::/oppereeiiie
fortforumm Swwrummidddr-xxxnCeeee
[23] Ricky T. Q. Chen, Jens Behrmann, David K Duvenaud, and Joern-Henrik Jacobsen. Residual flows for invertible generative modeling. H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-uuc. E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https::/ http:s//ppoceedings.. peroceedings.neuur.p... www...pp..c/pppprrre pttpssscpppp/p////////aaa /ttp:///1///f//// 03110 Paper.
[24] Jens Behrmann, Will Grathwohl, Ricky TQ Chen, David Duvenaud, and Jörn- Jörn-Henrik Jacobsen. Invertible residual networks. In International Conference on Machine Learning, pages 573-582, PMLR, 2019.
[25] Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Transactions on Information theory, 37(1):145-155 151, 1991.
[26] Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P. Kingma. PixeICNN++: Improving the PixclCNN with discretized logistic mixture likelihood and other modifications. In International Conference on Learning Representations, 2017. URL URL htt.:httt////oett hattpe://peereeiieet http://ew....eeeooordeddddeeeeee
[27] Diederik P. Kingma and Max Welling. An introduction to variational autoeneoders. Foundations and Trends@ in Machine Learning, 12(4):307-392, 2019. doi: 10.1561/2200000056 URL lttpss//:////o
org/10.1 15111222222000000556
[28] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain adversaral training of neural networks. The journal of machine learning research, 17(1):2096 2030, 2016.
[29]
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL htthht:///rrchve..rt icc. uci edu/ml
[30] George Papamakarios, Theo Pavlakou, and lain Murray, Masked autoregressive flow for density estimation. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL htttp::/pproceeeiiiei:e wwwpdiiinnnnnuuua..coo wwwpa/ppacccpppe////////.. Caca8866622267822211101111133113
[31] Li Deng. The mnist datahase of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 141-:144 2012.
[32] Martin Arjovsky and Leon Bottou. Towards principled methods for training generative adversarial networks. In International Conference on Learning Representations, 2017. URL httpss://openreviet net/forum? LiddHk44quus00
[33] Jonathan Ho, Xi Chen, Aravind Srinivas, Yan Duan, and Pieter Abbeel. Flow++:
Improving flow-based generative models with variational dequantization and architecture design. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors. Proceedings of the 36th International Conference on Machine Learning. volume 97 of Proceedings of Machine Learning Research, pages 2722-2730. PMLR, 09-15 Jun 2019. URL htRP/attpp://proceedigg mlr. wwa...pressvv/// wttp//9///hhh./ html
12


Reviews:
[{"review_id": "tbq14XPRI1q", "reviewer": "Reviewer_G1Px", "summary": "This paper addresses the problem of alignment of multiple distributions. The authors propose a new method for distribution alignment based on invertible flow models and a new training objective which gives an upper bound on the generalized Jensen-Shannon divergence (GJSD) between the distributions. The method learns invertible transformations (one per each distribution) from the data space to a shared latent space. The invertible mappings are optimized so that the induced latent space distributions are as close as possible to a variational distribution Q which is chosen from an appropriate family of distributions. The parameters of the variational distribution Q are optimized (via maximum likelihood) so that Q approaches the mixture of the latent space distributions. An important feature of the approach is that the invertible transformations and the variational distribution are trained cooperatively meaning that the training is cast as  a min-min game in contrast to min-max games arising in adversarial alignment approaches.\n\nThe authors derive a variational upper bound on the GJSD and convert this upper bound to a loss function which can be computed (and optimized) using only samples from the original distributions. The authors prove that global minima of the objective correspond to invertible mappings that align the distributions. The authors also explain that previous flow based methods (AlignFlow with MLE objective only and LRMF) can be seen as special cases of the proposed approach. Finally, the authors suggest that the proposed training objective can be used as a relative metric for evaluation of alignment methods (similar to how ELBO is used as a metric for density estimation).\n\nThe authors conduct experiments on toy data, UCI tabular datasets, and MNIST image dataset. The proposed approach is shown to outperform the AlignFlow and LRMF in terms of alignment, parameter efficiency, translation quality, and training stability.\n", "questions": "Q1: I believe that Theorem 2.5 can be formulated as “global minima of L_AUB is attained if and only if the distributions are aligned”. So, the statement of the theorem can be stronger (implication both ways rather than only “global minima implies alignment”). The current proof already confirms the implication in the opposite direction.\n\nQ2: (line 125) “|J_T(x)| is the determinant of the Jacobian of T” -> “|J_T(x)| is the absolute value of the determinant of the Jacobian of T”\n\nQ3: legends in Figure 3 are barely readable. What do the green ellipses represent?\n\nQ4: (line 227) what does “identity-initialized” Q mean?\n\nQ5: The role of the variational distribution Q is still not completely clear to me after reading the paper. In principle the alignment of P_1, …, P_k can be achieved by optimizing the AUB loss with any fixed Q (as AUB is equivalent to the sum of KL(P_j, Q)). I understand that when Q minimizes the AUB loss (cooperatively with P_1, … P_k) it is supposed to help optimization over P_1, …, P_k. The optimal Q fits the mixture of P_1, … P_k exactly, and when Q reaches that optimum, the AUB becomes exactly equal to the GJSD between the distributions. Given this interaction between P_1, …, P_k and Q, I have two questions:\n\nQ5.1: Is it really important that AUB loss gives an upper bound on GJSD? It seems that this fact does not affect the ability of the method based on AUB minimization to align the distributions. I understand that having a limited family of variational distributions Q might be problematic (as the results of AlignFlow on synthetic data suggest). However, it is still not clear what is the best choice of Q (since any Q should work in principle).\n\nQ5.2: It seems that optimization of L_AUB w.r.t. Q could lead to a problematic optimization landscape for P_1, … P_k. Assuming that Q reaches its goal and fits the mixture of P_1, …, P_k, such Q most likely will be multimodal (especially if distributions P_i have disjoint supports at the initialization) with each mode corresponding to one of P_i. Then, the optimization objective w.r.t. P_i is equivalent to reverse-KL (KL(P_i, Q) and we are optimizing over P_i). The reverse-KL optimization is known to have a bias towards locally fitting modes of the target distribution, so P_i being a mode of the mixture might be stuck in a local optima when Q fits the mixture. \n", "limitations": "I do not foresee any direct negative societal impact of this work.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "S: strength W: weakness\n\nS1: The paper contributes to the line of work on unsupervised distribution alignment via flow models and proposes a novel (to the best of my knowledge) approach which generalizes some of the existing techniques.\n\nS2: The paper is technically sound. The theoretical results and the proofs are correct. The chosen experimental setups and metrics are adequate. \n\nS3: The paper is clearly-written and well-organized. The presentation of the theoretical motivation behind the method is precise. The experiments are described in detail and the results are reported clearly. \n\nW1: Paper is missing references to a related line of work (references [1] and [2] below) on non-adversarial alignment methods via iterative flow models based on sliced optimal transport.\n\n[1] Dai, Biwei, and Uros Seljak. \"Sliced Iterative Normalizing Flows.\" International Conference on Machine Learning, 2021\n\n[2] Zhou, Zeyu, et al. \"Iterative Alignment Flows.\" International Conference on Artificial Intelligence and Statistics, 2022.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "qRouVqQNc6j", "reviewer": "Reviewer_fkCG", "summary": "This manuscript proposed a framework for flow-based unsupervised dataset/distribution alignment, where a variational objective was derived and is shown to upper bound the JSD. Based on the analysis, the authors derive results linking deep techniques for UDA and motivated their approach. Unlike the adversarial training-based UDA methods, this work adopted the flows-based methods to minimize the JSD upper bound and has been shown tighter than other methods. The paper is generally well written and easy to follow the main idea.", "questions": "1. The proposed methodology has cooperative training (min-min optimization). Will this optimization be easy to solve, considering that the method requires the transformations should be invertible?\n\n2. Since this work claims to opt for the cooperative objective rather than adversarial training ones, the benefits of this choice should be well-explained.\n\n3. I would expect to see more comparisons with recent strong baselines in DA (especially those with adversarial training objectives) with more challenging datasets (e.g. Office-31, Office-Home or DomainNet etc.)\n\nMinor questions:\nWhat is the exact meaning of $Z_{mix}$ in Theorem 2.2?", "limitations": "The current evaluations are only conducted for some simple datasets, making it hard to check the empirical effectiveness.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. This paper investigates an interesting flow-based distribution alignment method. It provides good insights into the flow-based approaches, which is a promising research direction.\n\n2. The proposed work is well-motivated and the analysis seems to be technically sound.\n\nWeakness:\n1. The cooperative training, which looks like a min-min optimization, seems to lead the Q model to be difficult to train since the proposed methods require the transformations must be invertible. In this respect, I think adversarial training might be better. I would expect the authors to provide further explanations/ discussions for this point.  \n\n2. The empirical evaluations are not sufficient, I would expect to see more comparisons with recent strong baselines in DA (especially those with adversarial training objectives) with more challenging datasets (e.g. Office-31, Office-Home or DomainNet etc.)\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "qRAJEXw0V0t", "reviewer": "Reviewer_tc4i", "summary": "Authors propose a novel non-adversarial Alignment Upper Bound (AUB) objective that uses normalizing flows to learn invertible mappings from respective domains to a shared latent space modeled explicitly by another parametric density model Q. The objective, up to a constant, equals the weighted log-likelihood of multiple flow densities each fitted to samples from the respective domain using a shared prior Q that is also optimized. Authors show that the proposed objective upper bounds the JSD and leads to a minimization alignment problem. Authors show that the objective is a generalization of LRMF and AlignFlow, and illustrate specific toy configurations in which equivalent LRMF and AlignFlow models fail. Authors show that their method achieves better alignment on tabular data, and achieves better image realism when transforming across MNIST digit classes compared to both LRMF and AlignFlow.", "questions": "See concerns above.", "limitations": "Authors discuss that the performance of their approach is limited by the quality of the underlying density model.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. The main idea is well-motivated and is presented very well. \n2. The relation to prior work is established.\n3. The derivations are sound.\n4. Toy experiments highlight specific cases in which prior work fails.\n\nNevertheless, I still have some concerns:\n\n1. L168 “AlignFlow without adversarial loss terms is a special case of our method for two distributions where the density model class Q only contains the standard normal distribution (i.e., a singleton class).”, In the “Maximum Likelihood Estimation” section of the AlignFlow paper, Grover et al. discuss that parameter sharing across early layers might be helpful (\"Instead, we can share parameters between the two mappings\"). In my understanding, an MLE-based AlignFlow with shared layers would be equivalent to an AUB with flow-based Q - is that correct?\n\n2. L178 “[LRMF does] not uncover the connection of the objective as an upper bound on JSD”. Looking at the LRMF paper, the sub-section “Relation to Jensen-Shannon divergence and GANs” seems to establish some relationship between the introduced objective the JSD?\n\n3. Both tabular and digit experiments tell very little about the semantic quality of the alignment. The MNIST digit class transformation task is somewhat contrived. Is there a chance you could include at least an experiment with USPS-to-MNIST adaptation with post-adaptation classification scores?\n\nSome minor comments:\n\nThe equation introduced in L119-120 is critical for going from (3) to (4), but can be easily missed. I would have appreciated it if the eq in L119-120 had its own number and was referenced around L130.\n\nThere seems to be a pair of parentheses missing in eq (4-5) - the log() should be applied to the product of J and Q, but log Q |J| reads as if log Q is weighed by |J|.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "UWIEWq5UJbI", "reviewer": "Reviewer_M6As", "summary": "This paper proposes a new objective function to calculate the Generalized Jensen-Shannon Divergence, which can be used to the field of the unsupervised distribution alignment. The experimental results on different tasks show the effectiveness of the proposed objective.", "questions": "1. The impact of different values of $w$ on the final performance of the model is not clear.\n2. The main part of the article does not refer to Figure 2.\n3. As for the objective 6 and Algorithm 1, we can see that the update of $Q$ and $T$ is in a step by step manner, why not update the $Q$ and $T$ in and end-to-end manner, \n4. How to guarantee ${P_{{Z_{mix}}}} \\in Q$ ？\n5. From Figure 5, we can see that the training process of the proposed method in this paper is obviously more stable than that of AlignFlow, I hope the author can give a theoretical analysis.\n6. I hope the author can provide the results when using this objective function in the domain adaptation and image generation.", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The theoretical analysis provided is interesting and make sense.\n2. This paper is easy to read.\n3. The experimental results are good and adequate.\n\nWeaknesses:\n1. The motivation of this paper is vague and unconvincing.\n2. The novel of this paper is limited.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept