Learning representations that are closed-form Monge mapping optimal with application to domain adaptation
Authors: Oliver Struckmeier
Ievgen Redko
Anton Mallasto
Karol Arndt
Markus Heinonen
Ville Kyrki
Venue: Accepted by TMLR ()

Abstract: Optimal transport (OT) is a powerful geometric tool used to compare and align probability measures following the least effort principle. Despite its widespread use in machine learning (ML), OT problem still bears its computational burden, while at the same time suffering from the curse of dimensionality for measures supported on general high-dimensional spaces. 
In this paper, we propose to tackle these challenges using representation learning. In particular, we seek to learn an embedding space such that the samples of the two input measures become alignable in it with a simple affine mapping that can be calculated efficiently in closed-form. We then show that such approach leads to results that are comparable to solving the original OT problem when applied to the transfer learning task on which many OT baselines where previously evaluated in both homogeneous and heterogeneous DA settings.

Published in Transactions on Machine Learning Research (08/2023)
Learning representations that are closed-form Monge map- ping optimal with application to domain adaptation
Oliver Struckmeier
oliver Nibenn.ttuuckmmieeBBoattot
Aolto University, Finland Intelligent Robotics Group
levgen Redko
Noah's s Ark Lab, Huewei Technologies Aolto University, Finland
Anton Mallasto
onton. aatoo.mallasto@oaltt.
Aalto University, Finland
Department of Computer Science
Karol Arndt
karol.orudt@ualt.ff
Aolto University, Finland Intelligent Robotics Group
Markus Heinonen
morkus. o. heinonen@oalto.ff
Aollo Universily, Pinland
Department of Compuler Science
Ville Kyrki
ville.ygriiaaalto
Aolto University, Finland Intelligent Robotics Group
Reviewed OpenReview: https: openreveee... net/ forum? id= :d=n0Iff0F22m
Abstract
Optimal transport (OT) is powerful geometric tool used to compare and align probability measures following the least effort principle. Despite its widespread use machine learning (ML),, OT problem still bears its computational burden, while at the same time suffering from the curse of dimensionality for measures supported on general high-dimensional spaces. In this paper, we propose to lackle these challenges using representation learning. In particular, seek to learn embedding space such that the samples of the two input measures become alignable in it with simple affine mapping that can be calculated efficiently in closed-fore form. We then show that such approach leads to results that are comparable to solving the original OT problem when applied to the transfer learning task on which many OT baselines where previously evaluated in both homogeneous and heterogeneous DA settings. The code for our contribution is available https: h tritthu..cc com/Oleffa/LaOT

---

Published in Transactions on Machine Learning Research (08/2023)
"To design is to devise courses of action aimed at changing ezisting situations into preferred ones." " Herbert Simon, Nobel Prize winner, 1969.
1 Introduction
Optimal Transportation (OT) theory provides researchers with a large variety of tools to compare and align probability measures that are omnipresent in today's Machine Learning (ML) tasks. When the goal is to find a mapping for two continuous probability measures, one usually seeks to solve the original Monge OT formulation (Monge] 1781), while when one looks for cof-ccorrespondenees between the points in the supports of two empirical measures, the Kantorovich formulation (Kantorovich 1942) of the OT problem is usually considered. Due to its versatility, OT has recently become popular with its applications, spanning such diverse tasks and areas unsupervised learning (Laclau el al.
2016) natural language processing 2017 Rolet et al.
AAlarrz-Melle & Jaakkola
2018
Kusner et al. 2015 Singh et al. 2020 generative modelling (Arjovsky
ct al. 2017 Bunne et al. 2019).
computer vision
Kolkin et al. 2019
Mrouch 2020 and computational
biology (Demetci et al.
2020
Limitations In practice, finding an optimal map or consistently estimating OT costs on real-world high- dimensional and large-scale data is hard, due to the curse of dimensionality of OT on the one hand (Fournier &
Guillin 2013 Weed & Bach 2017), and its high computational complexity on the other (Peyré & Cuturi 2019).
One popular approach to mitigate the curse of dimennsionllity is to consider adversarial lower-dimensionall
projections of the input measures (Paty & Cuturi 2019 Dhouib et al. 2020 Alaya et al. 2022) and solve OT on the projected measures. Another example is given by the famous sliced Wasserstein distances [Bonneel et al. 2015 Deshpande et al. 2018), which leverage the closed-form solution of the OT problem in 1-dimensional space to calculate the OT cost through averaging over several such projections. These approaches, however, do not allow obtaining the mapping between the distributions, but only the OT cost. Another case of interest is the OT problem between Gaussian probability measures (Dowson & Landau 1982). and random variables linked through an affine transformation (Flamary el al., 2019; Mallasto et al. 2021), for which OT can be calculated in closod-form. However, as real-world data rarely corresponds such lavourable scenarios, this closed-form solution was only used scarcely in practice (Pitié & Kokaram 2007
Mroueh 2020).
Our contributions In this paper, we motivate our main proposal by the following question:
Can representation learning help to find an embedding space where the Monge mapping con be calculated explicitly for two discrete measures?
We answer this question positively and validate through an application to the DA problem leading to the following contributions:
1. We present new framework of learning linearly alignable representations that can be used to learn an embedding space in which the supports of two input probability mcasures become linked through an affine transformation.
2. We show that in such space closed-form linear Monge mapping can be used to align them with a very appcaling computational complexity. This is contrary to previous works on OT that either use neural networks lo approximate the Monge map between high-dimensional input distributions (Seguy et al. 2018 Kirchmeyer et al. 2022) or use digh-dimensional optimal couplings that do not scale
with the increasing sample size (Courty et al. 2014 2017a Redko et al.] 2019b Yan et al. 2018 Redko al. 2020).
3. Our computational approach to OT is then evaluated in transfer learning setting: for the the case when the two domains' input spaces are the same (homogeneous DA) or different (heterogeneous DA). This is contrary to previous works on OT in DA that need to consider OT formulations on incomparable spaces to handle the heterogeneous DA setting.
2

---

Published in Transactions on Machine Learning Research (08/2023)
doc.
z As* +
ge
doc
Figure 1: Illustration of the proposed approach for two datasets in R3. In the original 3D space, the projection obtained via linear Monge mapping (yellow points) between the two 3D datasets fails align the datasets as the data in the original space neither follows & Gaussian distribution, nor it linked through an affine transformation. Our approach learns an embedding space where the linear Monge map becomes optimal, while ensuring that the embeddings are discriminative for downstream tasks.
The rest of the paper is organized as follows. After the necessary preliminary knowledge on OT and its use in DA, we outline our main contributions and provide a theoretical analysis for DA with linearly alignable representations. Then, we evaluate our proposal on tasks for homogeneous unsupervised and heterogeneous semi-supervised DA where OT methods have previously shown to be efficient. Lastly we show that our approach of solving the OT problem in lower dimensional space reduces computational complexity. We end this paper with conclusions.
2 Preliminary knowledge
Notations In what follows, we will use the following notations. We denote spaces and sets by black-board upper-case letters (e.g. X, Y)),, probability measures are denoted by calligraphie upper-case letters (e.g. S,T), bold upper-case and lower-case Greek letters denote matrices (e.g. X, and bold lower-case letters denote vectors (c.g. x, b). We denote the marginal distribution of S with respect to X by Sx and denote by P(X) the space of probability measures supported on X with finite second moments.
Below, we present some background knowledge used in the following sections of this paper.
Optimal transport Given two metric spaces Xs. Xr, and cost function c: Xs x Kr - R, the Monge problem in OT is defined as follows:
g arg min (cx'' Sx=
(1)
Here I# Sz denotes the push-forward measure, which is equivalent to the law of g(x"), for x² ~ Sx- Unfortunately, solving equation [ is very hard in practice as its constraints are non-convex and the solutions for it may not exist in discrete case when Sy and Tx are empirical measures.
A more widely adapted approach is consider instcad the Mongo--antooovic problem (Kantorovich, 1942) and the Wasserstein distance associated to it. The latter is defined a value at the solution of the former follows:
W. WS, Tx)
min E,c(x",x) SE Tx)
(2)
where I1x5,, Tx) is the space of probability distributions over Xs x Xy with marginals Sx and Tx. When the squared Fuclidean cost. function e(;;)) . [| is used, write simply W3. Once y is obtained,
3

---

Published in Transactions on Machine Learning Research (08/2023)
uses the barycentric mapping Ferradans et al. 2013 to define an approximation to the Monge mapping g follows:
g x + arg min E (xx))
(3)
x
Domain adaptation Let Xs, Kr be two subsets of Rd and Y be discrete set of outpuls. Given two datasets
S [x), S(Xs x Y)
T = (x), (xf, ~ T(xr x Y) (x Tx,
the goal of domain adaptation (DA) (Pan & Yang 2010 Weiss et al. 2016) is to learn a hypothesis function
h : Xy + from some hypothesis class H. using the data from S and such that the true target risk (T) == TE (((((x))) is as small as possible for some loss function & Y + I. In what follows, we distinguish between unsupervised DA, ie, n4 = and, semi-supervised DA, ie, 0 < m < nt. We also deploy the term heterogeneous when considering a setup where Xs + XT.
The vast majority of algorithms solving DA follow the theoretical foundation laid out in the seminal works on DA theory (Ben-David et al. the following learning bound 4h E E
2010) (surveyed in Redko et al. (019c)). This latter can be summarized by
Ry (h) - Rs(h) + d(Sx, Tx) min ( (h) + Rs(h)),
(4)
heHt
where .l.,)) is some divergence or distance on the space probability measures. Eq. equation suggests the idea of learning an invariant feature trunsformation (Zhao et al. 2019) function g Xx U Xx - Z such that d(Sg, Tx for the distributions S%, T induced by g while ensuring that Rs o g) as small as possible. One should note that, in general, can also be applied to one of the domains only such that d(Sg Tx) 0.
This approach is often referred to asymmetric feature transformation.
As finding a way to minimize Rs(h o g) presents & common well-studied supervised learning problem, the main challenge of solving DA was thus to find a meaningful measure of divergence d(.;.) and a learning strategy to find g minimizing it. OT theory has become a popular choice to find g in order to solve both
homogencous (Courty et al. 2014 Shhnn al., 2018 Courty et al., [2017a) Redko al. 2019b Damodaran
et al. 2018 Xu el. al.] 2020 Rakotomamoniy al., 2021 Kirchtneerr el al. 2022 and heterogeneous DA
Yan et al. 2018 Redko et al. 2020
3 Proposed contributions
Motivation Previous OT approaches aiming at obtaining a mapping g aligning two arbitrary probability distributions have several important drawbacks. On one hand, the methods using the barycentric mapping derived from the sigh-dimensional optimal coupling, such as (Courty et al. 2014 2017a Redko et al., 2019b] Yan et al. 2018), are unsuitable for large-scale applications as shown in (Seguy et al. 2018). On the other hand, Monge mapping estimation methods (Seguy et al., 2018 Kirchmeyer et al. 2022)
often parametrize the
Monge mapping with neural networks that may fail to converge to the true solution (Korotin al. 2021).
In this section, we present method that relies on a closed-form solution of the Monge problem in the particular case of random variables linked through an affine transformation. As for real-world data, the relationship bet the random variables following source and target distributions is unlikely to be linear. We first present our framework of learning linearly alignable representations. In practice, we propose to achieve this by embedding the data into a space where the affine transformation between the source and target samples becomes nearly optimal. We now proceed by defining this idea more formally.
3.1 Linearly alignable representations
We propose to use generative modeling find a new data representation for which source and target distributions are linearly alignable. Of these, the latter can be formally defined based as follows.
4

---

Published in Transactions on Machine Learning Research (08/2023)
Dist. (A=0.01): 3e-2
Dist. (A=0.1): le-2
Dist. (1=0.5): 2e-3
Dist. (A=1): 1e-3
Dist. (A=5); 2e-4
/8/8/
78/
Source samples
Target samples
Transp. samples
Figure 2: Embeddings (light blue and orange points) and linear Monge mapping projection (blue points) obtained by our approach for different values of X for 9a.9t _200 + R2, We can see that the linear Monge mapping becomes more optimal in the embedding space as confirmed by smaller Wasserstein distance values for increasing values of A.
Definition 3.1. Given two distributions Sx E P(Xs) and Tx E P(Xr), the feature transformation functions 9. : :x Es, 9t XT Zi are called linearly alignable (LA) for Sg and Tx if 37 Az + with an invertible matrix A and translation vector b such that S T
Given Definition 3.1 learning LA representations thus boils down identifying two major ingredients: 1) the alignability criterion forcing (g.. 9t) to provide LA representations for samples drawn from two distributions; 2) the data fidelity term forcing (gs, 9t) to truthfully reflect the statistical distribution of the input samples in the embedding space. We discuss our choices for both these ingredients below.
Lincar Monge mapping When Sx and Tx are linked through an affine transformation T with positive definite matrix A, the OT problem admits simple solution that can be caleulated based on the Gaussian approximations N (ms, Es) and [(mr, Er) of Sx and Tx (Flamary al. 2019 Mallasto et al., 2021 p. In particular, have that for two such distributions, the Wasserstein distance between Sx and Tx admits closed-form expression for the quadratic cost Wasserstein distance:
W Tx) I|mms mr tr(Es) tr(Er) 2tr 21r(E
and the optimal transport map Tai of the corresponding Monge problem is given by:
TiS
(x) = Ax + b, b m- Ams-
aft
A = イ
(5)
When dealing with empirical measures Sx and Tx, Es, Σr, ms and my are replaced with their empirical (biased) counterparts defined from available finite samples from the supports of the two distributions. In the sequel, we denote those with a hat as well, ie, Å is defined terms of the covariance matrices Σs, Er and means my, my. We note as well that for small sample sizes (as in stochastic optimization), one can use shrinking (Ledoit & Wolf 2004) to obtain better estimate of the covariance matrix.
Based on this, we propose to define the alignability for two distributions Sx and Tx as the Wassorstein distance between the push-forward of Sx with ((x, and Tx, ie,
L1x Tx) = (T. Txl Sx, Tx),
where Tar is defined in equation 5 The intuition behind this is that when this distance is close to 0, the lincar Monge mapping Tax becomes optimal for the two distributions implying that they become lincarly alignable with Taf.
Data fidelity To preserve the information contained in the samples drawn from Sx and Tx when making them linearly alignable, we propose to model gs Xg Zgg and gr Ee - Zr encoders of two different.
5

---

Published in Transactions on Machine Learning Research (08/2023)
auto-encoders with the same dimensionality of the embedding space k, i.., Zr S RR. More formally, we have the following reconstruction term:
Rec (Sx; Txx Exx=~xx1x* (g. dec,) dec,,)x*|8 )x1 Exi I|x (gr dec,)x"
where the decoders dec,: 2s + Xs, dec; : ZT + XT seek to reconstruct the learned embeddings by mapping them back into the original space. Using two separate auto-encoders allows us further deal with the cross-domain OT setting by employing auto-encoders with different input dimensionality. This will become very useful in heterogencous DA setting considered in the evaluation part of our work.
Optimization problem Putting it all together, we propose to optimize the following objective function:
min ge.gr,decg deer
Le... (Sx, Tx) + (S
where X is a ppper-paraeeter controlling the degree to which the linear alignability is promoted illustrated in Figure 2 In nutshell, equation 0 sceks to embed the data from two distributions supported on potentially different metric spaces into two representation spaces for which there exists affine map given by the linear Monge map- that aligns them. This idea is illustrated in Figure !
Complexity analysis Flamary et al. 2019) noted that the sample complexity of linear Monge mapping estimation is dimension-free free and addresses the curse of dimensionality of solving the original OT problem. Given two samples of size n from Id, the latter is known to have a sample complexity of O(n (()),), while the former is Onn b) (Theorem 1, (Flamary et al. 2019)). Similarly, the computational complexity of calculating the linear Monge map O(nd² + d3) which particularly attractive for large-scale applications due its linearity in n. The dependence on dimensionality is alleviated by the fact that we estimate it in the embedding space of dimensionality l < d.
Lifting to the input space Minimizing equation G allows to obtain new low-dimensional embeddings of the input measures for which the linear Monge mapping is optimal. One may wonder, however, whether it is possible to lift the obtained mapping back to the original space. This question was studied in Muuzellec] & Cuturi 2019) where the authors showed how a Monge mapping that is optimal on a subspace can be used to define an optimal mapping, or a coupling, in the original space as well. In the particular case of our work that uses closed-forms Monge mapping, Muzellec & Cuturi 2019) show that it can be used to define an optimal coupling in a closod-form based on the subspace optimal solution. Unfortunately, 9s and ge are not subspace projectors in our case, meaning that identifying whet her the linear Monge mapping is optimal on the input measures is much harder. We leave this idea for future investigation.
3.2 Theoretical guarantees for domain adaptation
As explained in Section 2 OT maps can be used in DA to align the data drawn from two probability distributions order to transfer a classifier across them. Following the simplicity of our computational approach OT and the closed-form expression of the Monge mapping in the embedding space, we derive theoretical guarantees for the performance of classifier transferred from S to T via Tatt S T Before introducing them, we recall the definition of the Lipschitz function used in the statements.
Definition 3.2. A function h X + Y is called M-Lipschitz if |[h(x) h(x')ll S M| xixxxxx x'Il for all x, X.
We now present our main theoretical results for the DA task and postpone all the proofs of this paper to Scction 6.1 in Appendix.
Thcorom 3.3. (Best-case bound) Let E H be My, Lipscchtt and the loss function & be Me- -ippshhtt in its SS .T.2)
second argument. Then, if there exists a mapping m such that my = T ='"'mm("" - alf and linearly alignable feature transformation, functions gs and gt for Sx and Tx, we have that
(2"),"))
RTu (no ( off
)-t) < RSv. (h) + M (max((s, (e)
(7)
6

---

Published in Transactions on Machine Learning Research (08/2023)
As mentioned in Section 2 previous works on DA theory introduced the learning bounds on the target error following the general shape of equation 4 For instance, in Redko et al. (2017) [Shen et al. (2018) the obtained bounds corresponded exactly to equation 4 with d(Sx, Tx) = WiH- (Sx, Tx) while in Courty et al. (2017a) a similar bound was obtained with WIH (5,7) where T was defined with pseudo-labels. In the case of linear Monge mapping, however, the learning bound on the target error becomes much simpler and does not involve any additional terms under the introduced assumptions. Furthermore, it can be improved using Flamary et al. (2019) where under some additional assumptions, one can show that the true target error of the hypothesis calculated from the available source data, ie, h* E arg m Rg.. (h) converges to the optimal target classifier hn arg m R. (h), even despite the absence of labelled data in the target domain. This remarkable result thus motivales our framework of learning linearly alignable representations as it provably transposes the problem of DA to much more favourable setting.
To complete this section, we also present a more general lcarning bound close in spirit to that given in equation | For this result, we do not assume the existence of a mapping m that allows to remove the ideal joint error term ming (h) + Rsa. (h)), and do not assume that our feature transformation functions are linearly alignable. We only assume that the linear Monge mapping is used to align the two distributions in the embedding space.
Theorem 3.4. (Worst case bound) Let h E H be Mip-iiishhit. Denote by ] == -T
((TT .T2" #* and let Is 2s + Y and JT : Zr +Y be the true labelling function associated to T(S and T respeclively. Then, for two arbitrary feature transformation functions Is and gt, we have that
Ryg. (h, ft) SRTis (h. fo) + 22 tr( TT)) )+ + min Rigg (h) + RTIST (h, fa).
(8)
NEW
This result is the worst-case scenario for our proposed framework as it bounds the Wasserstein distance between T[ and T by its largest possible value given by tr(")ge)) As in practice our learning algorithm solves a non-convex optimization problem and can, in principle, converge to approxinately linearly alignable feature transformations 9, and 9t, this result suggests controlling the variance of the target embedded features to avoid having target latent space Zr that is too spread along all k directions.
3.3 Related works
Our work is situated at the cross-roads of computational OT and transfer learning. Below, we review related approaches and point out their differences with respect to our work.
Monge mapping estimation Estimating the OT map from finite samples drawn from two probability distributions is a very active rescarch topic nowadays. The vast majority of such methods (sce Table in Korotin el al.] (2021) and references therein) parametrize the Monge mapping, or the potential that defines it following Brenier theorem (Brenier 1991), using either a traditional or an input convex neural network (Amos) et al. 2017 Our work is principally different from this line of research in two main aspects. First, these contributions use the high expressive power of NNs and ICNNs to solve the hard problem of finding mapping between two continuous high-dimennional measures. Our work instead uses the power of representation learning to find a new space where the problem of mapping two distributions becomes easy, As such, neural OT mothods and our proposal solve different problems and cannot be used interchangeably. Finally, Perrot et al. (2016) approximate the barycentric mapping from equation 3 using linear or kernel regression. Contrary to it, we use closod-form expression of the true Monge mapping that is optimal in the embedding space and scales better as it never explicitly calculates the high-dimensional coupling.
Our approach is related to OT methods that use projection of the data to low-dinensional subspace (Bonneel et al. 2015 Courty et al.] 2018 Muzellee & Cuturi 2019 Bonet let al. 2021) to accelerate OT computation. In Bonneel et al. (2015) (and follow-up works (Kolouri et al.
Subspace learning for OT
2019 Deshpande et al. 2019 ) the authors propose sliced Wasserstein distance computed as an average of the Wasscrstein distances over one-dimensional projections of the high-dimensional distributions where the Wasserstein distance can be calculated in foreed-fomm. Sliced Wasserstein distances are commonly used as way to compute the approximate OT cost faster, for instance in generative modelling (Deshpande et al.
7

---

Published in Transactions on Machine Learning Research (08/2023)
2018), yet they do not provide a mapping between the considered distributions. In Courty et al. (2018). the authors embed the data into new space where the Euclidean distance between the embedded samples corresponds to the Wasserstein distance between the input empirical measure. The purpose of their method is thus different as it aims to accelcrate the OT computation. Muzellee & Cuturi (2019) (and follow-up work Bonnet et al., 2021 ) is much closer in spirit. to what we propose: their idea is to extend the Monge map that is optimal on the low-dimensional subspace to be optimal on the full space. Our approach learns a new representation, rather than finding subspace of the original space, for which the optimal Monge map is easy to compute and does not seek to lift it to the input space.
OT in DA We now briefty discuss other OT-basod DA works here. Courty ct al. (2014) is the seminal work that proposed to use OT in DA. The authors solve equation 2 with entropic and plass-based inguaaiiaations and then use equation 3 to project source data to the target domain. This method was further extended to the alignment of joint probability distributions in Courty ct al.] (2017a) and its doep version Damodaran et al. (2018). Another line of work on OT in DA is concerned with larget shift. (Redko et al. 2019b] and generalized target shift. (Rakotomamoniy et al. 2021
Kirchmeyer et. al., 2022) where 8 T due to Sy * Tr for target shift and S(xly) + (xxly) in addition to it for generalized target shift. Several methods also follow the invariant feature transformation framework such as Shen et al. (2018); Xu et al. (2020). Finally, Yan et al. (2018): Redko et al. (2020) tackle the heterogeneous DA setup using Gromov- Wasserstein (Memoli] 2011) and Co-Optimal transport problems in (Redko et al. 2020). Our work is different from all these methods as it relies on closed-fone Monge mapping and allows to unify both heterogeneous and homogeneous DA setups in one approach. Additionally, its simplicity also allows us to benefit from stronger theoretical guarantees in the embedding space that are unavailable for other existing methods. For a general survey on DA, we refer to
Weiss et al. 2016
Wilson & Cook 2019
4
Experimental evaluations
In this section, we evaluate our method, termed LaOT (Lincarly Alignable Optimal Transport) against other OT-based methods for commonly considered unsupervised homogeneous (UDA) and semi-supervised heterogeneous DA (HDA) tasks. Given a pair "Source + Target', for both settings the final goal is to learn a classifier using only the available labelled data in the source domain to further evaluate it in the target Office/Caltech10
domain. For both evaluations we use Office/ Caltecch0 10 dataset (Saenko al.) 2010) as well as the Visual Domain Adaptation dataset (visda) (Peng et al., 2017) for the UDA setting. The Office/Calted dataset consists of 4 different domains, namely: Amazon (A) (958 images), Caltech (C) (1123 images), Webcam (W) (295 images) and DSLR (D) (157 images) from 10 overlapping classes. The reason to choose this particular dataset is two-fold: first, it was used to evaluate all other b'--basdd baselines in DA thus allowing for fair comparison with them; second, it still represents a benchmark with enough room for improvement. The visda dataset was chosen to investigate LaOT in a large-scale OT setting with a large dataset of high-dimensional image data and 12 Classes. We now present in more detail the evaluation setup for the UDA and HDA settings considered below.
Implementation details We use fully connected NNs with 1 hidden layer for Is, St. dec,, dec, with ReLU activation function. In all experiments, the size of the hidden layer is fixed to half of the size of the input layer. The classifier used for UDA is a fully connected NN with softmax function applicd to the output. For HDA, none of the considered baselines learns a classifier simultaneously to solving the OT problem so that in this case we minimize equation 0 without any additional terms. The optimization is carried out using Adam optimizer (Kingma & Ba 2014] in PyTorch (Paszke et al. 2011)) with gradient normalization and default initialization of the weights. We also use POT library (Flamary et al. 2021) to minimize W3. The code, well as the visualizations of the learned embeddings and several ablation studies are provided as part of the Appendix.
Model selection As suggested in Redko et al. 2019b). we use reverse validation [hong et al., 2010) with 3NN classifier for our method in order to choose the best hyperparameters that include the size of the embedding space [64,, 128, 256). regularization strength A E [0.1, 0.05, 0.01], batch size E [32, 64, 128] and
learning rate [5e 5, le 4, 5e 4). We perform 10 runs of 10 epochs for each set of hyperparameters and
8

---

Published in Transactions on Machine Learning Research (08/2023)
Tasks
Base
OT-IT
OT-MM
JDOT
LaOT
A-C
84.77
85.93
87.36
85.22
86.02
(84.93+0.77)
A+D
86.62
77.71
79.62
87.90
92.36 (88.85:1..55)
A-W
79.32
74.24
85.08
84.75
96.95
(92.3312.83)
C-A
92.07
89.98
92.59
91.54
92.59 (90.7311.01)
C-D
84.08
78.34
76.43
89.81
93.63 (89.8711.55)
C-W
76.27
80.34
78.98
88.81
93.90 (88.07#2.27)
D-A
83.19
90.50
90.50
88.10
89.87 (86.96+0.ç6)
D-C
77.03
85.57
83.35
84.33
79.52 (76.5+0.87)
D
96.27
96.61
96.61
96.61
95.93 (94.07=1.07)
W-A
79.44
89.56
90.50
90.71
93.42
(90.16:0.74))
W+C
71.77
84.06
82.99
82.64
83.26
(75.57+1.52)
W-D
96.18
99.36
99.36
98.09
97.45
(95.9212.24)
p-value
<0.05
0.2
0.33
0.62
Table
Classification results for UDA task. Bold and underlined scores present the best and the second best. results.
Baseline results reported from Courty et al. (2017a).
Accurecy of lineat Classilie
MC aihhel pmnteettoon
Accuroy
40
0.01
0.05
0.1
0.5
3.0
Epochs
Source points
Target points
Figure 3: (left) Ilustrationn of the trade-off between data fidelity and alignability terms; (middle) Learning dynamics showing the evolution of the Wasserstein distance between the learned embeddings and the transfer accuracy; (right) Learned embedding obtained using tSNE fvan der Maaten & Hinton 2008h.
pick the model having the lowest variance of the reverse validation score. We also report the best model chosen by reverse validation, i.c. without using target labels unavailable during learning, over the runs. This latter metric common for deep DA methods (Shen et al., 2018) as the considered datasets are rather small and may lead to model converging to bad local minima.
4.1 Homogeneous unsupervised DA
Setup For this evaluation, we constitute 12 pairs of adaptation tasks for the 4 domains and use the weights of the 6th layer the DECAF convolutional neural network (Donahue et al.) 2014 pre-trained on ImageNet as their features. This leads to an adaptation problem between sparse 4096 dimensional vectors. Following Courty et al. (2017a), we use cross-validated SVC classifier with linear kernel (Pedregosa al., proposal against famous OT-based approaches used in DA, namely:
2011 for all methods. We compare ratzopy-regularrized (OT-IT) and class-wing regularized OT (OT-MM) (both from Courty et al. (2017b)) that adds group-lasso penalty on the coupling matrix that doesn't 't allow source points of different classes to be transported to the same larget point. Finally, we also add Joint Distribution Optimal Transportation (JDOT) (Courty et al. 2017a) method to our comparison that uses OT to align joint probability distributions and learns classifier for pscudo-labelled target data simultancously. All these basclines are evaluated against. the source classifier directly applied in the target domain (Base). Additionally, and show that our method compares favourably to deep DA hoth follow the evaluation protocol of Shen et al. (0088) and compare

---

Published in Transactions on Machine Learning Research (08/2023)
DANN
CORAL
WGRL
LaOT
Mean
67.67
90.76+4.39
92.74+3.52
93.82+5.55
p-value
0.09
0.2
0.33
Table 2: Average best accuracy for UDA against deep-based DA methods. Complete results are presented the Appendix.
Tasks
Base
SGW
COOTLP
COOT
LaOT
A-
83.0413.07
89.75=4.8
92.89-0.32
89.74410.01
91.86 (9110.911)
A-C
69.98=2.88
79.80=5.82
86.76=1.28
83.76++..0
81.12 (80.07+1.77)
A-W
30.49:3.96 93.76:+2.06
96.61+1.34
94.44+2.23
95.59
(92.92++1.47)
C-+A
83.09+2.94 78.37:5.08
67.28+1.02
89.6611.23
89.35 (88.51+1.4)
C+C
68.46=3.13 81.3115.009
67.28+1.19
81.95+1.79
82.72
(79.82+1.78)
C-W
81.66=+4.62
93.81+3.36
69.39+22.01
90.92+1.85
91.53
(88.3442.34)
W-A
84.59+3.4
11.63=
72.33+1.19
84.75+1.557
91.34
(88.92+:2.44)
W-C
4.660
75.25+6.13 13
63.51+0.78
77.3+3.7
81.75
(76.08+3.11)
W-W
32.83++3.42
94.00+11.13
77.49+22.6
95.42+1.39
94.24 (93.28+2.65)
p-value
<le-2
<0.05
0.05
0.73
Table 3: Classification results for semi-supervised HDA task. Bold and underlined scores present the best and the second best results.
the best achieved performance (using target labels) of our method against three deep-based baselines, namely: domain adversarial neural networks (DANN) (Ganin et al.) 2016). Deep Correlation Alignment (CORAL) [unn& & Saenko 2016) and WVssersstein-guidee Representation learning (WGRL) (Shen et al. 2018).
Results The obtained results are presented in Tables 1 and an illustrative example of the inner- working of our algorithm is given in Figure 3 (for other pairs, similar plots can be found in the Appendix). From the comparison with both shallow OT-based and deep DA methods,
can that LaOT is statistically par with them according to Wilcoxon signed-rank test calculated with respect to the best model. This performance is achieved despite the simplicity of our method, that similarly to CORAL and OT-IT, doesn't 't rely on adversarial training (DANN, WGRL), on structural constraints on the coupling matrix (OT-MM) or pscudo-labeling and joint distribution adaptation (JDOT).
4.2 Comparison to large-scale OT
Below, we compare our approach to stochastic solver proposed to solve OT for large-scalc applications in Seguy el al. (2018). In their paper, the authors parametrize the dual variables of regularized OT problem with neural networks (Alg. 1 with entropic or le regularization) and then use a neural network to approximate the barycentric mapping (Alg. 2) based on the neural duals. As one of the examples where their approach can be useful, the authors proposed to solve the UDA problem on three large-scale DA tasks: MNIST (M) (60000 samples) to USPS (U) (9298 samples), USPS (U) MNIST (M) and MNIST (M) to SVHN (S) (73212 samples). We use the setup of (Seuuy et al. (2018) and report the best accuracy of the INN classifier on target domain in Table [| From the obtained results, we can see that our algorithm is on par or better than OTDA approaches and stochastic solvers (Alg. 1 only) for large-scale regularized OT presented in Seguy ct al. 2018). This suggests that it can be also used to tackle largo-scale problems efficiently. Finally, we note that our method is slightly worse than the combination of Alg. and on M-U and S-M tasks. This improvement, however, comes at a price of heavy computational burden: the neural networks parametrizing
We couldn't reproduce the 'Source only" bascline results from Seguy al. 2018 For fair comparison, report the relative performance each algorithm with respect the "Source only" reported results Seguy al. 2018) for their algorithms and those reproduced by for LaQT.
10

---

Published in Transactions on Machine Learning Research (08/2023)
Method
M +
U
U
+
M
S + M
OT-IT
-4.72
+20.38
intractable
Alg. 1 with ent.
-4.63
+20.58
+4.54
Alg. 1 with &
-4.33
+20.50
+6.23
Alg. 1+2 with ent.
+4.45
+23.05
+6.78
Alg. 1+2 with l
-0.66
+23.53
+8.47
LaOT
-0.9
+25.31
+6.03
Table Comparison stochastic OT solver Seguy al. (2018).
the duals has the output size equal to the input dimension of the original space, which can make their learning prohibitive for
datasets. Also, the second step (Alg. 2) has been shown to not converge to the true Monge mapping in Korotin et al.] (2021). Our method embeds the data into low-dimensional space and doesn't require any additional steps to produce the Monge mapping which is nearly optimal (with an explicit control of the optimality given by A) in the embedding space.
A second large-scale OT experiment was conducted using the VisDA17 dataset (Peng et al. 2017). We reproduced the results of another recent OT-baseline, JUMBOT (Fatras et al.) 2021) that is based the unbalanced mini-batch OT implemented in spirit close to JDOT. The resulting model achieved 69.72% accuracy on VisDA using neural network classifier in the target domain. We extracted the source and target representations from JUMBOT and and finetuned them using LaOT. For a fair comparison we used the same parameterless classifier as opposed to JUMBOT which is using a neural network. LaOT improved the accuracy in the target domain with our classifier from 56.6% using JUMBOT's pre-aligned features 62.35% with LaOT.
4.3 Heterogeneous remi-supervised DA
In this experiment, we evaluate LaOT on the same dalasel but with source and larget. feature representations given by activations from GoogleNet (Szegedy et al.) 2015) and Decaf (Donahue et al. 2014h neural network architectures. In the OT context, aligning two such heterogencous datasets alleviated by using OT in incomparable spaces: first such contribution relies on the Maanov-Wssserstein distance (Yan et al., 2018 (SGW), while a more recent. method improving upon this latter used its generalization termed Co-Optimal Transport [Redko al. 2020) (COOT). We follow the protocol of Redko et al. (2020) where only the domains A, C and W were considered. To help guiding adaptation in this case, previous works commonly consider the semi-supervised setting with a handful of labelled examples in the target domain. In this evaluation, we set the number of such examples to 3 per class, ic, n! = 30. For all baselines, we use the hyper-parameters suggested by authors in the respective papers. As our method aligns datasets using a Monge mapping and not the coupling matrix used in Redko et al. (2020) to perform label propagation (Redko et al. 2019a). we present the results of SGW and our method with 3NN classifier, and use label propagation results for COOT only.
Results From Table we see that our method is statistically better than SGW and COOT with label propagation and is on par with COOT followed by 3NN classifier. As in the homogeneous setting, our method uses simple closed-form solution in the embedding space, contrary to simultaneous sample and feature alignment of COOT and pair-wisc matrices' alignment with conditional distribution matching of SGW. This further supports our clain about the fact that representation learning can alleviate the intrinsic complexity of aligning sigh--imensional probability measures by finding embeddings making the OT problem easier to solve.
4.4 Computational complexity
Solving the Kantorovich (JDOT) and Gromov-Wassersted Wasserstein problem (HDA) has cubic and quartic complexities respectively. In this work aim to show that those problems become easier to solve (linear in the number of
11

---

Published in Transactions on Machine Learning Research (08/2023)
Method
100
n
1000
π
10000
= 20000
LaOT
0.07s
0.11s
0.17s
0.23s
Kantorovich OT GW
0.007s
0.22s
48.62s
1338.33s
4.82s
207.43s
infeasible
infeasible
Table 5: Time needed to compute the OT map between source and target dataset for a different number of samples n from the MNIST-SVHN dataset.
Method LaOT
UDA
Embedding
OT map
Total
0.046s
0.016s
0.062s
M-S
Kantorovich OT LaOT
0.22s
48.62s
138.53s
0.003s
0.009s
0.012s
A-W
Kantorovich OT
0.026s
0.026s
LaOT
0.003s
0.009s
0.012s
AC-WD
GW
20.81s
20.81s
Table 6: Computation time for the UDA and HDA experiments, split into time to compute the embedding and time to compute the OT map. The superscripts G and D indicate GoogleNet and Decaf features, respectively.
samples) when finding the right representation for the supports of the input measures. As previously shown we are able to do this while maintaining similar or better DA performance.
Table shows the time needed to compute the OT map for a different number of samples for the MNIST-SVHN pair of datasets. Our method scales well compared to the Kantorovich OT (UDA) and Gromov-Wassersee OT (HDA) solutions. In Table 6 we compare the computational time for the UDA and HDA experiments. LaOT is capable of computing the OT map for the entire MNIST-SVHN dataset (n = 60000) in reasonable experiment (d = 4096, n < 1000), LaOT compares to
time. In the higher dimensional Kantorovich OT and is significantly faster than Gromov- Wasserstein OT.
5 Conclusion
In this paper, we proposed a novel contribution at the crossroads of computational OT and transfer learning. On the one hand, we introduced a learning framework that embeds the data from two distributions to a new representation space where we can explicitly calculate the Monge mapping between them. On the other hand, we showed how this learning framework, termed learning linearly alignable representations, can be used in both homogeneous and heterogeneous domain adaplation with strong theoretical guarantees and high competitive performance. Our work is a first contribution that aims at exploiting the simplest solution to the Monge problem in general d-dimensional spaces. In this work we concentrated on only one application of our general approach, mainly to showcase how its simplicity can bring both theoretical and empirical advantages in transfer learning. Our proposal, however, can be used many other ML problems where Monge mapping is already used such as in, for instance, GANs, were the use of sliced Wasserstein distance is known to reduce significantly the computational burden related to their training.
Limitations Our method is, like other state-of-the-art DA methods, subject to impossibility theorems (Ben- David et al. 2010) concerning the ability of DA methods to generalize across different domains. [en- David et al. 2010 first introduced a series of theorems stating that DA can fail even when the source and target distributions are perfectly aligned and the source error minimized. This discussion is centered around the error bound given Eq 1. in their paper,
(T( Rs(h) d(Sx, Tx)
min min(R (R) Rs(h))
NEH
where d the HAH divergence. The impossibility theorems state that minimizing only the observable terms Rs(h) d(Sx, Tx) does not allow full control over the target error due to the presence of the third term.
12

---

Published in Transactions on Machine Learning Research (08/2023)
This term cannot be estimated or minimized due to the fact that (h) is defined over T which denotes the joint distribution over inputs X and Y. For target domain, we do not have labels in UDA setting we have only to 3 labeled imputs per class (semi-supervised DA) making the consistent estimation of its distribution unlikely.
The work by Zhhoo et al. (019)) transposes the seminal results of Ben-David al. (2010) into the invariant.
representation learning framework. The paper discusses the impossibility theorems that apply also in feature space, concluding that estimating the target marginal label distribution is necessary to tackle these drawbacks but is impossible due to the above mentioned limitations. Recent work by SSooanov et al. (2021) shows that the effects of the impossibility thcorems can be alleviated by learning a shared, invariant representation space of the data. In their work, [tojanov et al. (2021) used two separate autoencoders with two feat transformation functions to achieve that effect. Such a shared latent space between two different embeddings is less sensitive to domain-specific variations, resulting in better generalization performance. In addition having two separate encoders, the authors of Stojanov al. (2021) also propose to take into account the domain label as additional knowledge when encoding the source and larget. samples in order to further reduce the sensitivity to domain specific variations.
Our proposed method shares the invariant representation learning approach to DA with Zhao et al. 2019 and is therefore subject to the same impossibility theorems. However, we also share architectural properties with the method proposed by SSooanov et al. (2021) and should therefore benefit from the same mechanisms that alleviate the implications of the impossibility theorems by Ben-David et al. (2010).
Acknowledgments
This work was partially funded by Business Finland through the Santtu project. The presented calculations were in part performed using computer resources within the Aalto University School Science "Science-IT" project.
References
Mokhtar Z. Alaya, Maxime Bérar, Gilles Gasso, and Alain Rakotomamonjy. Theoretical guarantees for bridging metric measure embedding and optimal transport. Neurocomputing, 468:416-444 1116-430,-- 2022.
David Alvarez-Melis and Tommi S. Jaakkola. Gromov-Wassersten Alignment of Word Embedding Spaces. In EMNLP, pp. 1881-1890, 2018.
Brandon Amos, Lei Xu, and J. Zico Kolter. Input convex neural networks. In ICML, pp. 146 155, 2017.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In ICML, pp. 214 223, 2017.
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Vaughan. A theory of learning from different domains. Machine Learning, 79:151-175, 2010.
Clément Bonet, Titouan Vayer, Nicolas Courty, François Septier, and Lucas Drumetz. Subspace detours meet wonsortwassoostcinn. Algorithms, 14(12):366, 2021.
Nicolas Bonneel, Julien Rabin, Gabriel Peyré, and Hanspeter Pfister. Sliced and radon wasserstein barycenters of mcasures. J. Math. Imaging Vis., (1()):22-45 2015.
Yann Brenier. Polar factorization and monotone rearrangement of vector-valued functions. (4):375-417, 1991.
Charlotte Bunne, David Alvarez-Melis, Andreas Krause, and Stefanic Jegelka. Learning Generative Models across Incomparable Spaces. Tn ICML, pp. 851-861, 2019.
N. Courty, R. Flamary, and D. Tuia. Domain adaptation with regularized optimal transport. In ECML PKDD, pp. 1-16, 2014.
Nicolas Courty, Rémi Flamary, Amaury Habrard, and Alain Rakotomamonjy, Joint distribution optimal transportation for domain adaptation. In NIPS, pp. 3730-33 3730-3739, 2017 7a.
13

---

Published in Transactions on Machine Learning Research (08/2023)
Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamonjy. Optimal transport for domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9):1853-1888)) 2017b.
Nicolas Courty, Rémi Flamary, and Mélanie Ducoffe. Learning wasserstein embeddings. In ICLR, 2018.
Bharath B. Damodaran, Benjamin Kellenberger, Rémi Flamary, Devis Tuia, and Nicolas Courty. Deepjdot: Deep joint distribution optinal transport for unsupervised domain adaptation. In European Conference in Computer Visions (ECCV), 2018.
Pinar Demetci, Rebecca Santorella, Björn Sandstede, William Stafford Noble, and Ritambhara Singh.
Gromov-wasserstiin optimal transport to align single-cell multi-omics data. RioRriv, 2020.
Ishan Deshpande, Ziyu Zhang, and Alexander G. Schwing. Generative modeling using the sliced wasserstein distance. In CVPR, pp. 3483 3491, 2018.
Ishan Deshpande, Yuan-Ting Hu, Ruoyu Sun, Ayis Pyrros, Nasir Siddiqui, Sanmi Koyejo, Zhizhen Zhao, David Forsyth, and Alexander G Schwing, Max-sliced wasserstein distance and its use for gans, In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10648-10656, 2019.
Sofien Dhouib, levgen Redko, Tanguy Kerdoncuff, Rémi Emonet, and Mare Sebban. A swiss army knife for minimax optimal transport. In ICML, pp. 2504-2513, 2020.
J. Donahuc, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and Darrell. Docaf: A deep convolutional activation feature for generic visual recognition. In ICML, 2014.
D. C. Dowson and B. V. Landau. The fréchot distance between multivariate normal distributions. Journal of Multivariate Analysis, (2(3):450-4555 1982.
Kilian Fatras, Thibault Séjourné, Rémi Flamary, and Nicolas Courty, Unbalanced minibatch optimal transport; applications to domain adaptation. In International Conference on Machine Learning, pp. 3186 3197. PMLR, 2021.
Sira Ferradans, Nicolas Papadakis, Julien Rabin, Gabriel Peyré, and Jean-François Aujol. Regularized discrete optimal transport. In Arjan Kuijper, Kristian Bredies, Thomas Pock, and Horst Bischof (eds.), Scale Space and Variational Methods in Computer Vision, pp. 428444888 -29, 2013.
Rémi Flamary, Karim Lounici, and André Ferrari. Concentration bounds for linear monge mapping cstimation and optimal transport domain adaptation. CoRR, abs/1905.10115, 2019.
Rémi Flamary, Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Lactitia Chapel, Adrien Corenñlos, Kilian Fatras, Nemo Fournicr, Lćo Gautheron, Nathalic T.H. Gayraud, Hicham Janati, Alain Rakotomamonjy, levgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica J. Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer. Pot: Python optimal transport. Journal of Machine Learning Research, 22(78):1-8, 2021.
Nicolas Fournier and Arnaud Guillin. On the rate of convergence in wasserstein distance of the empirical measure, 2013.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Vietor Lempitsky. Domain-adversarial training of neural networks. J. Mach. Learn. Res., (7(1):2096-2088 2016.
L. Kantorovich. On the translocation of masses. C.R. (Doklady) Acad. Sci. URSS (N.S.), 37:199-201, 1942.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiw:1412.698// 2014.
Matthicu Kirchmeyer, Alain Rakotomamonjy, Emmanuel de Bezenac, and patrick gallinari. Mapping conditional distributions for domain adaptation under generalized target shift. In International Conference Learning Representations, 2022.
14

---

Published in Transactions on Machine Learning Research (08/2023)
Nicholas I. Kolkin, Jason Salavon, and Gregory Shakhnarovich. Style transfer by relaxed optimal transport and sell-similarity. In CVPR, pp. 10051-10060, 2019.
Soheil Kolouri, Kimia Nadjahi, Unut Simsekli, Roland Badeau, and Gustavo K Rohde. Generalized sliced wasserstein distances. arXiv preprint arXii::192.004434 2019.
Alexander Korotin, Lingxiao Li, Aude Genevay, Justin Solomon, Alexander Filippov, and Evgeny Burnaev. Do neural optimal transport solvers work? A continuous wasserstein-2 benchmark. 2021.
Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. From word embeddings to document distances. Proceedings of the 32nd International Conference on Machine Learning, pp. 957-966 2015.
Charlotte Laclau, levgen Redko, Basarab Matei, Younès Bennani, and Vincent Brault. Co-clustering through optimal transport. In ICML, pp. 1955 1964, 2017.
Olivier Ledoit and Michael Wolf. Honey, shrunk the sample covariance matrix. The Journal of Portfolio Management, pp. 110-119, 2004.
Anton Mallasto, Karol Arndt, Markus Heinonen, Samuel Kaski, and Ville Kyrki. Affine transport for sim-to-real domain adaptation. CoRR,
11739, 2021.
Facundo Memoli. Gromov wasserstein distances and the metric approach to object matching. Foundations of Computational Mathematics, pp. 1 71, 2011.
Gaspard Monge. Mémoire sur la théorie des déblais et des remblais. Histoire de l'Académie Royale des Sciences, pp. 666-704, 1781.
Youssef Mroueh. Wasserstein style transfer. In AISTATS, pp. 842-852, 2020.
Boris Muzellec and Marco Cuturi. Subspace detours: Building transport plans that optimal on subspace projections. In NewrIPS, pp. 6914-6944 19255, 2019.
S. J. Pan and Q. Yang. Survey on Transfer Learning. JEEE Transactions on Knowledge and Data Engineering, 22(10):1345 1359, 2010.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advonces in Neural Information Processing Systems 32, pp. 8024 8035. 2019.
François-Pieed Pierre Paty and Marco Cuturi. Subspace robust wasserstein distances. In ICML, pp. 5072 5081, 2019.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 282202 2011.
Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv preprint NYo:1710.0695 2017.
Michaël Perrot, Nicolas Courty, Rémi Flamary, and Amaury Habrard. Mapping estimation for discrete optimal transport. In NIPS, pp. 4197 4205, 2016.
Gabriel Peyré and Marco Cuturi. Computational optimal transport. Foundations and Trends in Machine Learning, 11:355-600 607, 2019.
François Pitié and Anil C. Kokaram. The linear monge-kantororve linear colour mapping for example-basod colour transfer. In IEFE European Conference on Visual Media Production, 2007.
15

---

Published in Transactions on Machine Learning Research (08/2023)
Alain Rakotomamonjy, Rémi Flamary, Gilles Gasso, Mokhtar Z Alaya, Maxime Berar, and Nicolas Courty. Optimal transport for conditional domain matching and label shift. Machine Learning, 2021.
I. Redko, N. Courty, R. Flamary, and D. Tuia. Optimal transport for multi-source domain adaptation under target shift. In International Conference on Artificial Intelligence and Statistics (AISTAT), 2019a.
levgen Redko, Amaury Habrard, and Marc Sebban. Theoretical analysis of domain adaptation with optimal transport. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML/PKDD), pp. 737-753, 2017.
levgen Redko, Nicolas Courty, Rémi Flamary, and Devis Tuia. Optimal transport for multi-source domain adaptation under target shift. In AISTATS, pp. 849 858, 2019b.
levgen Redko, Emilie Morvant, Amaury Habrard, Marc Sebban, and Younès Bennani. Advances in Domain Adaplation Theory. Elsevier, 2019c.
Ivegen Redko, Titouan Vayer, Rémi Flamary, and Nicolas Courty. Co-optimal transport. In Neural Information Processing Systems (N) 2020.
Antoine Rolet, Marco Cuturi, and Gabriel Peyré. Fast dictionary learning with smoothed wasserstein loss. In AISTATS, pp. 630 638, 2016.
K. Saenko, B. Kulis, M. Fritz, and T. Darrell. Adapting visual category models to new domains. In ECCV, LNCS, pp. 213 226, 2010.
Vivien. Seguy, Bharath B. Damodaran, Remi Flamary, Nicolas Courly, Antoine Rolet, and Mathieu Blon- del. Large-scale optimal transport and mapping estimation. In International Conference Learning Representations (ICLR), 2018.
Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. Wasserstein distance guided representation learning for domain adaptation. In Sheila A. McIlraith and Kilian Q. Weinberger (ods.), AAAI-18, pp. 4058-4065, 2018.
Sidak Pal Singh, Andreas Hug, Aymeric Dieuleveut, and Martin Jaggi. Context mover's distance and barycenters: Optimal transport of contexts for building representations. In Proceedings of the Twenty Third International Conference Artificial Intelligence and Statistics, pp. 3437-3449, 2020.
Petar Stojanov, Zijian Li, Mingming Gong, Ruichu Cai, Jaime Carbonell, and Kun Zhang, Domain adaptation with invariant representation learning: What transformations to learn? In Advances in Neural Information Processing Systems, volume 34, pp. 24791 24803, 2021.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In ECCV Workshops, 2016.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR, pp. 1-9, 2015.
Laurens van der Maaton and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579 2605, 2008.
Jonathan Weed and Francis Bach. Sharp asymptotic and finitc-sample rates of convergence of empirical measures in wasserstein distance, 2017.
Karl Weiss, Taghi M. Khoshgoftaar, and DingDing Wang. A survey of transfer learning. Journal of Big Data, 3(1):9, 2016.
Garrett Wilson and Diane J. Cook. A Survey of Unsupervised Deep Domain Adaptation. [es, stati, 2019.
Renjun Xu, Pelen Liu, Liyan Wang, Chao Chen, and Jindong Wang. Reliable weighted optimal transport for unsupervised domain adaptation. In CVPR, pp. 4393 4402, 2020.
16

---

Published in Transactions on Machine Learning Research (08/2023)
Yuguang Yan, Wen Li, Hanrui Wu, Huaqing Min, Mingkui Tan, and Qingyao Wu. Semi-supervised optimal transport for heterogeneous domain adaptation. In IJCAT, pp. 2969-2999999 2018.
Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. In ICML, pp. 7523 7532, 2019.
Erheng Zhong, Wei Fan, Qiang Yang, Olivier Verscheure, and Jiangtao Ren. Cross validation framework to choose amongst models and datasets for transfer learning. In Machine Learning and Knowledge Discovery in Databases, pp. 547 562, 2010.
A Appendix
A.1 Proofs of theorems
Full proof of Theorem 3.1
Proof. From the definition of linearly alignable feature transformation functions, we deduce that 37 such that S -h. = T Given the assumption about the existence of mapping m, we have that for any h E H, Rga. (h) = R. o T- We then use Proposition from Flamary et al. (2019) to obtain the desired result by replacing the original source and larget distributions with their embedded counterparts.
]
Full proof of Theorem 3.2
Proof. Let ht E arg oR 01 (h, ft) + R RTisg- (h, fo). Then, we have that:
Rygu (h, f1) S Ryp (h, h²) + T (h", f{)
< Ri (1, + Re (h", ft) + RT (h, (1, 1()
< Re (t), + R (h, 1.) + 2M, W1 (T Te)))
< Rygi ( + RTisgn) (h, fa) + Rrisg1 (h' fa) + 2M W W (T(S8)) Txtt)
Big (li, + 2M, () T + min R RT (h, fi) + Brisy (h, fa) hEW
S Ryg. (hi", ) + 2M, W; S7 1, T + min Ryg (h, f() + R (h, fa)
NEW
S R (1, f2) + 222 ) min Ryp: (h, fo) + RTysy (h. fa).
The proof follows the common reasoning used to obtain DA learning bounds with the Wasserstein distance Redko et al. (2017): Shen et al. (2018). Line is obtained using Lemma 1 from Shen et al. (001)). Line is due to the Jensen inequality implying for all 0 < p < g, that W, S W,- is then completed by an upper-bound on the Wasserstein distance between T[S and T that was bounded in Mallasto et al. 22021 by tr(
A.1.1 Pseudo-code
We present the pseudo-code for our algorithm Algorithm 1 The algorithm describes the stochastic optimisation of our objective function in the coupled autoencoder setting, For a number of epochs we iterate over batches of training data from the source domain S and target domain T. We compute the gradients for source and target domain autoencoder model using the loss function presented in 6 Then the weights of both the source and target domain models are updated. The final output is two fully trained autocncoder models parameterised by wg and wT
(E)
(E)
17

---

Published in Transactions on Machine Learning Research (08/2023)
Algorithm Algorithm for learning linearly alignable represenlations. More details in section A.1.1
Input: S = [xf. T [x]]]=1 target labels [y]) if available, initial weights wgg wy number of epochs E, size of the latent embedding k, learning rate a, hyperparameter
Output: final weights w/²), mff
1. for t= 0 to E
2.
for (batchig, batchr) in cip(bacchsss, batchest)
3.
estimate Vag L(wss w V Vug LRo.. (batchig batchr) + ACLa (g- (batchs) (batchrr) ))
4.
estimate Vay Coy_(Ws WT ) FwyCRee (batchs, batchr) + ACLA (batchs)) dr (batchr))
(1)
5.
w's
((+1) = wg aV ((us w
(t)
6.
WT = wy a (wss WT
((41)
7.
return WT
A.2 Comparison of LaoT with linear Monge mapping on raw data
In Table we present an abalation study showing how promoting linear alignability affects the performance on DA task compared to applying linear Monge mapping on raw data directly (OT-Gauss). We can see that apart from two DA tasks, OT-Gauss method is always far below LaOT and even of the base classifier.
Tasks
Base
OT-Gauss
LaOT
A-C
84.77
83.35
86.02
(84.93-0..77
A
86.62
83.44
92.36 (88.85-2.55)
A-W
79.32
81.36
96.95 (92.33=2.83)
C->A
92.07
89.56
92.59 (90.73+1.01)
C-D
84.08
82.17
93.63 (89.87=1.55)
C-W
76.27
81.69
93.90 (88.07=2.27)
D
83.19
82.67
89.87 (86.96+0.56)
D+C
77.03
78.45
79.52 (76.5=0.87)
D-W
96.27
97.63
95.93 (94.07+1.07)
W
79.44
84.13
93.42 (90.16=0.74)
W-C
71.77
76.22
83.26 (75.57=1.52)
W-
96.18
97.45 (95.92.12.22
Table 7: Classification results for UDA task comparing LaOT and linear Monge mapping on the raw data (OT- Gaus). Bold and underlined scores present the best and the second best results. Baseline results reported from Courty al. 2017a)
A.3 Full comparison with deep UDA methods
Below, we provide full results for all pairs of Offic//Caltech dataset corresponding to the average results Table 8
We can see that our method remains efficient even when compared stronger baselines given by adversarial DA methods.
A.4 Illustration of the trade-off between data fidelity and linear alignability
In Figure we present the results obtained by best performing LaOT models when varying the À parameter in [0, 0.01, 0.05, 0.1, 0.5, 1]. The value of À = 0 correspond to the case when only data fidelity loss is minimizod and no alignment is forced between the two embeddings. As can be seen from this result, this leads to drastic loss in terms of accuracy, while other values of À lead to approximately the same results.
18

---

Published in Transactions on Machine Learning Research (08/2023)
Tasks
DANN
DeepCORAL
WGRL
LaOT
A-C
87.80
86.18
86.99
87.62
A-
82.46
91.23
93.68
98.09
A-W
77.81
90.53
89.47
99.32
C-AA
93.27
93.01
93.54
93.53
C-+D
91.23
89.47
94.74
96.18
C-W
89.47
92.63
91.58
97.97
D
84.70
85.75
91.69
92.07
D-C
82.11
85.37
90.24
83.17
D-W
98.95
97.89
97.89
98.64
W- 82.98
88.39
93.67
94.47
W+C
81.30
88.62
89.43
84.77
WDD
100
100
100
100
Table 8: Best accuracy results for UDA against deep- based DA methods. Baseline results are reported from
Shen et al. (2018).
A.5 Illustration of learned embeddings
In Figure we provide plots of embeddings obtained using tSNE van der Maaten & Hinton (2008) learned for UDA task with LaOT. We can see that LaOT does not explicitly align two domains but has an extra degree of flexibility allowing to learn potentially richer representations.
19

---

Published in Transactions on Machine Learning Research (08/2023)
caltech10
dslr
webcam
AcorSC)
Acsuney
Accuracy
0.
0.00
0.
0.11
30
caltech10 amazon
caltech10 > dslr
caltech10 webcam
70
Acturany
Accuracy
0.0
0.01
005
01
0.5
10
08
0.1
0.06
03
05
10
3.0
0.01
035
01
10
webcam amazon
webcam caltech10
webcam + dslr
50-
Acuro
8
Accuracy
Accuacy
30
10
dslr amazon
caltech10
webcam
30
Auareey
Accurac
0.0
0.50
Dda
0.1
4.5
34
0.
0.:1
0.6
03
0.3
1.0
200
0.0
02
05
14
Figure 4: Trado-off between linear alignability loss and data fidelity loss for optimal LaOT models achieving highest UDA performance.
A.6 Illustration of learning dynamics
In Figure t we provide illustration for learning dynamics of our method UDA tasks. From this, see that the accuracy of the linear classifier increases when the distance after the projection with the linear Monge map in the embedding space decreases. This is in line with what expect from the minimization of our objective function.
20

---

Published in Transactions on Machine Learning Research (08/2023)
amazon - caltech10
amazon dsir
amazon - webcam
Target caltech10 dsir
Target:
points
Target
caltech10 amazon
caltech10 + webcam
Source points
Target points
Source points
Target points
Source points
Target points
webcam amazon
webcam caltech10
webcam - dsir
Source points
Target points
Source points
Target points
Source points
Target points
dsir + amazon
dslr + caltech10
dslr - webcam
points
Target points
points
Target points
Source points
Target points
Figure 5: Visualizations of embeddings for different UDA tasks.
A.7 Comparison with invariant feature transformation learning
Finally, we compare our approach against invariant feature transformation learning where the source and target data are explicitly forced to be close in the embedding space. For this, simply set (x)(x) Ax b in equation [ and optimize it before. For the sake of clarity, to illustrate both the Icarned embeddings and the learning dynamics of LaOT and the invariant feature transformation approach. These results are presented Figure 7 From this plot, we distinctly see that LaOT allows for the embeddings maintain their own topology for each individual domain as
dslr - webcam
Accuracy of linear classifier
W5 after projert eecciin
take the task D-W
the left, yet they well aligned after the projection
Epochs
LaOT
Invariant
21

---

Published in Transactions on Machine Learning Research (08/2023)
caltehh
dsllr
webcam
Accuraty linear cleosiiee
Accuanry of linear classilfer
Accuraty liman
Epachs
Epochs
120
Epachs
caltech10 amazon
caltech10 dsir
caltech10 webcam
Acouracy lineer dlesitter
Accurtyy linear dlass dasssifeer
projection
Acturacy lnver
Epochs
Epochs
Epochs
webcam amazon
webcam caltech10
webcam dslr
Accuracy Slosiiier
Accunary of linearr classiffer
projection
projection
projection
Accuracy linwar
Epochs
Lpocht
Epochs
dsir amazon
dsir caltech10
dslr webcam
Accurscy of linear clessitter
Accuracy of linwar classifeer
projecttoon
projecttinn
W/ projection
Accurac of Uenaa
Epochs
Epochs
Epochs
Figure 6: Learning dynamics of our method on UDA tasks.
with the linear Monge mapping as seen on the right. Invariant feature transformation learning forces the embeddings to be close to each other in the embodding space but achieves less precise alignment of the data in the embedding space. In this particular case, both achieve good performance, yet LaOT manages to do it in fewer epochs due to the additional flexibility that it has that does not require it to perfectly aligu the two domains.
22

---

Published in Transactions on Machine Learning Research (08/2023)
dslr - webcam
webcam
w
Source points
Target points
Source e points
Target points
Transp. points
Target points
Figure 7: Comparison with invariant feature transformation learning. (left) embeddings learned with LaOT; (middle) embeddings learned with invariant feature transformation; (right) source and target data after the projection with linear Monge mapping in the embedding space. Upper right: learning dynamics comparing the two models.
23


Reviews:
[{"review_id": "75oYp8P2eB", "reviewer": "Reviewer_xgC1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strong points: \n\n- I would think this paper proposes a new insight in domain adaptation setting. By assuming the linear relation in the source and target representation, this paper does not need to learn the invariant representation.\n- Theoretical studies are conducted. \n- Experiments are conducted in various DA (domain adaptation) settings.\n- The paper is clearly written and well-organized. \n\n## Weak Points:\n\nUnfortunately, I do not think the method is *principled*. Several designed components deeply depend on many hidden assumptions without clear discussions. Moreover, due to the experiments design (only with one benchmark and simple digits dataset), these limitations are omitted in the paper. Based on these, I would suggest significant **major revisions**. \n\n## Discussions on weak points:\n\n- About linear relation assumptions.\n\nThe key point within the paper is by assuming the linear relation between the source and target random variable in the representation space. However, throughout reading the whole paper. This reviewer still feels quite confused about when this assumption holds true. \n\nAs we all know, the high-dimensional data are rather complex and impossible to have such structures. Then it could be quite natural to consider such an assumption in the representation space. However, this assumption deeply depends on the data distribution and learned representation functions $g_s$ and $g_t$. Could you possibly list sufficient assumptions (or just speculations) when this is indeed true? \n\nIf this does not hold true, the closed form does not make sufficient sense to me. In this case, it would be much better to learn $A$ and $b$ as an optimization problem to avoid the noise in the linear model. E.g, using gradient based approach to update $A$ and $b$ in an online manner.\n\nAnother limitation is about the estimation of $A$ and $b$ in a deep learning regime. In general, deep learning will use a mini-batch to learn the model, where we could not directly estimate these two parameters from the mini-batch, right? This should be a biased estimator (due to the challenging in computing variance matrix). How are you addressing this estimation issue in deep learning? \n\n- About Eq(5) and parameters in linear model\n\nThe closed form parameters are based on the Gaussian assumption in the source and target domain. This is hardly true for $S(x)$ and $T(x)$. Does this hold in the latent space $S(z)$ and $T(z)$? In general, representation space could have manifold assumptions via a sort of clusters. However, in general it should be multiple clusters (like a mixture of gaussian) rather than one cluster (like single gaussian).\n\nSure, one could still achieve this by assuming the gaussian distribution is rich enough. For example, high-dimensional with full-rank covariance matrix. However,  this will raise significant issues in estimating the parameter of gaussian distribution. I do not think the current approach could correctly estimate these parameters. \n\n\n- About the decoder part.\n\nDifferent from the ideas in learning invariant representation, this paper added an auxiliary task by considering a decoder and reconstruction loss. However, this part seems quite ad-hoc and lacks very clear support. (The theoretical analysis in Sec 3.2 could not explain the role of reconstruction). \n\nUsing decoders enables me to think about why not using variational autoencoders in your case. For example,\n\n- If the encoder outputs a distribution of gaussian of source(target) on Z.\n- We could directly use this information to compute the closed form W-distance. \n- Decoder reconstructed images.\n- We could optimize W-distance and VAE (source and target) to ensure the latent representation is matched. \n\nThis could avoid issues in estimating $A$ and $b$.  \n\n\n- About experiments\n\nI would suggest authors doing major revisions in improving the experimental sections. \n\n\n1. About motivations for only choosing the Office/ Caltech 10 dataset .  \n\n\n> “The reason to choose this particular dataset is two-fold: first, it was used to evaluate all other OT-based baselines in DA thus allowing for fair comparison with them; second, it still represents a benchmark with enough room for improvement. “\n\nI do not think these arguments are convincing. Indeed, this paper aims to propose a *practical* framework in OT based domain adaptation.  If experiments are only conducted in office/caltech10 dataset and simple digits datasets. We are still quite unaware about limitations of each component and its practical limitations. From my perspective, experimental evaluation should be comprehensive (by considering most common DA datasets) to show the strong and weak points of the proposed approach across different datasets. It does not necessarily compete with the SOTA methods, but it should clearly deliver a message: when the proposed works or fails.\n\n\n2. In Sec 4, Implementation details. \t\n\t \t \t \t\t\n> We use fully connected NNs with 1 hidden layer for gs , gt , decs , dect with ReLU activation function. \n\t\t\t\t\nI could not understand how 1 layer MLP could learn image representation. Is there something that I am missing?\n\n3. Sec 4.3 Heterogeneous semi-supervised DA \n\n> In this experiment, we evaluate LaOT on the same dataset but with source and target feature representations given by activations from GoogleNet Szegedy et al. (2015) and Decaf Donahue et al. (2014) neural network architectures.\n\nI do not think this experiment is indeed heterogeneous data distribution. Since we just adopted two similar data embedding functions from the same dataset to obtain separate datasets (with different dim). In general, Heterogeneous data should have quite different data-space. For example multi-modality problems in health, the EHR (electronic health record) and medical imaging data both contain the pathology information of a patient, where they have quite different data space. The example within the paper could be considered by DA with different data-dimensions.\n\n4. In general, if an auto-encoder based approach is adopted. It is suggested to visualize the reconstructed images to illustrate the results.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper considers an optimal transport based domain adaptation. Specifically, this paper proposed a novel assumption in the embedding space:\n\n$$ z_t = A z_s + b$$\n\nWhere $A$ and $b$ are estimated from a closed form by source and target data-embedding. \nSeveral theoretical studies are proposed. Experiments are conducted in Office/ Caltech10 dataset and digits dataset (MNIST, USPS and SVHN) with various domain adaptation settings. \n", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please check the discussion point on weak points. ", "broader_impact_concerns": "I have No Broader Impact concerns. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "OrZYSvN0Vu", "reviewer": "Reviewer_Ubdi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n- Proposed a novel OT-based approach for DA that utilizes the closed-form solution of the OT problem (Monge problem) through an affine mapping. The closed-form solution of the Monge mapping significantly reduces computational complexity compared to existing OT-based methods, making it computationally efficient.\n- Demonstrated the applicability of the proposed approach in both homogeneous and heterogeneous DA settings, showcasing its versatility.\n- Provided promising empirical results, validating the effectiveness of the proposed approach. The method outperforms or achieves comparable performance to other well-known baselines in DA, based on traditional OT and OT in incomparable spaces.\n\n**Weaknesses**:\n- One notable weakness of this paper lies in the assumption that generative modeling can reliably map both the source and target distributions to linearly alignable representations. However, the lack of theoretical evidence undermines the claim that generative modeling can consistently achieve this mapping for highly dissimilar distributions.\n- The experiments primarily focus on benchmark datasets, including office-caltech-10, MNIST-M, USPS, and SVHN. While these datasets provide a foundation for comprehensive experimental results, it would be beneficial for the authors to further validate their method on additional challenging public benchmarks, such as office-home and DomainNet. This would not only enhance the empirical support of the paper but also strengthen its applicability and generalizability.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces an innovative approach utilizing optimal transport (OT) for domain adaptation (DA). The authors suggest initially leverage generative modeling to discover a new data representation, enabling linear alignment of the source and target distributions. They then optimize an affine mapping to align the linearly alignable representations. Notably, the affine mapping can be formulated as an OT problem with closed-form solution, resulting in a computationally efficient embedding space compared to existing OT-based methods. The proposed approach demonstrates its effectiveness in both homogeneous and heterogeneous DA scenarios, surpassing or matching widely-used baselines while significantly reducing computational complexity.\n\nA distinctive contribution of this paper lies in its exploration of the Monge problem in general d-dimensional spaces, an aspect that has received limited attention previously. This exploration of the simplest solution to the Monge problem opens up possibilities beyond DA, extending its applicability to other machine learning tasks utilizing Monge mapping, such as GANs. The paper represents an advancement in both computational OT and transfer learning, supported by robust theoretical foundations and promising empirical results.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would suggest the authors to disclose more details regarding to following items.\n- The empirical experiments depicted in Figure 2 aim to illustrate the feasibility of transferring high-dimensional distributions to a lower-dimensional space and aligning them using an affine mapping. While these experiments present promising results, the origin of the original distributions remains ambiguous. It is crucial to specify whether the distributions stem from the same source, similar sources, or significantly distinct sources. Such clarification is pivotal in establishing the efficacy of the proposed approach.\n- The paper lacks clarity regarding the prediction methodology for the target data. It is unclear whether the authors propose using the source classifier directly or the conjunction of the inverse affine mapping with source prediction function. ", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "doTY1kAX3s", "reviewer": "Reviewer_bKqw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- Simple and scalable method\n- Theoretical guarantees\n\nWeaknesses\n- Lukewarm results ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes Linearly Alignable Optimal Transport (LaOT) for domain adaptation. The main idea is to project the source and target domain data into a low dimensional representation space that maintains most of the input specific information and then apply simple, closed-form, linear optimal transport. More specifically, for the first step, the authors propose to use auto-encoders whereas for the second the authors assume that the representation space of the encoders between the source and the target domain are linked through an invertible affine transformation. In this case, the optimal transport problem reduces to forming Gaussian approximations over the marginals of the representations at the source and target domain and then finding the linear map that minimises the quadratic cost Wasserstein distance between them. The main benefit is that in this case there is a simple solution for the linear map, which depends on the means and covariances of the source and target domain. In order to then find “linearly alignable” representations, the authors propose to jointly optimise the autoencoders, regularised by the quadratic cost Wasserstein distance between the “linearly transported” marginal of the source representations to the target marginal over the representations. In this way, the auto encoders are encouraged to produce representations where the linear map itself is optimal in aligning the two representation spaces. The benefit of this approach is the simplicity and scalability, as the OT happens in the representation space instead of the, usually, high-dimensional input space.\n\nThe authors provide both best, i.e., assuming that there is indeed a linear mapping that connects the representation spaces of the two domains, and worst case, i.e., a general guarantee about the method irrespective of the existence of an underlying linear map, learning bounds for their method, mainly building upon prior works. Experimentally, the authors separate their settings into unsupervised and homogeneous (i.e., source and target domains have the same support) domain adaptation and semi-supervised and heterogeneous (i.e., the target domain has some labels but the support between the source and the target domain is different) domain adaptation. In all of these cases, the authors show comparable, to some baselines, and better, to some other baselines, performance.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I am leaning on the positive side for this work; the method is simple, scalable and has some theoretical guarantees. However, having said that, the results are a bit lukewarm and some more experiments / comparisons could perhaps make this work more convincing. More specifically, it seems that while LaOT is better than some baselines it is not (statistically significant) better than others, e.g., Tables 1-4. For example, at Table 4 we see that LaOT actually has negative transfer on the M $\\rightarrow$ U task, whereas the baseline, at least in a specific configuration, has positive transfer on all cases. Furthermore, during the discussion of the results of Table 3 at page 11, the authors argue that LaOT allows for easier solutions to OT problem, however, this does not necessarily translate to improvements compared to, e.g., COOT.\n\nBesides that, some other requests / points for discussion are the following:\n- How does the results scale with network / decoder size / flexibility? One could argue that with flexible enough decoders, full invariance might suffice as the linear alignability could implicitly enforced in some intermediate layer. While in appendix A.7 there is a comparison with invariant feature transformations, from what I understand, the linear map is still there (instead of using the identity with no additional parameters) and the authors did not explore how the model size affects the conclusions.\n- During the discussion of the results at Table 4, the authors argue that the baseline method comes at a heavy computational burden due to the neural networks having an output size equal to the high-dimensional input. Isn’t this the case for LaOT as well, given that each domain decoder needs to reconstruct the original high-dimensional input? \n- How does the size of the batch affect the results at LaOT, given that empirical estimates for the Gaussian means and covariances are employed?\n- How is semi-supervised learning done in practice? Is an additional term added to eq. 7 that uses the representation which is linearly aligned?\n\nMinor things:\n- It will be better to use \\citep instead of \\cite when not directly referencing to a work. \n- The citation for the Adam optimiser is wrong.\n", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper has significantly improved during review process, the presentation is acceptable by all reviewers; and despite the restriction coming from fixed representation, the paper is still valuable to the community. This assumption lives with many recent results and going beyond it is not a straightforward thing to resolve, hence my overall vote is for the paper to be accepted.
Decision: Accept (poster)