Low-rank Optimal Transport: Approximation, Statistics and Debiasing
Anonymous Author(s)
AfﬁliationAddress email
Abstract
The matching principles behind optimal transport (OT) play an increasingly important role in machine learning, a trend which can be observed when OT is used todisambiguate datasets in applications (e.g. single-cell genomics) or used to improvemore complex methods (e.g. balanced attention in transformers or self-supervisedlearning).
To scale to more challenging problems, there is a growing consensus that
OT requires solvers that can operate on millions, not thousands, of points.
The lowrank optimal transport (LOT) approach advocated in Scetbon et al.
[2021] holdsseveral promises in that regard, and was shown to complement more establishedentropic regularization approaches, being able to insert itself in more complexpipelines, such as quadratic OT.
LOT restricts the search for low-cost couplings tothose that have a low-nonnegative rank, yielding linear time algorithms in cases ofinterest.
However, these promises can only be fulﬁlled if the LOT approach is seenas a legitimate contender to entropic regularization when compared on propertiesof interest, where the scorecard typically includes theoretical properties (statisticalbounds, relation to other methods) or practical aspects (debiasing, hyperparametertuning, initialization).
We target each of these areas in this paper in order to cementthe impact of low-rank approaches in computational OT.
Introduction
Optimal transport (OT) is used across data-science to put in correspondence different sets of observations.
These observations may come directly from datasets, or, in more advanced applications, depictintermediate layered representations of data.
OT theory provides a single grammar to describe andsolve increasingly complex matching problems (linear, quadratic, regularized, unbalanced, etc...),making it gain a stake in various areas of science such as as single-cell biology Schiebinger et al.[2019], Yang et al.
[2020], Demetci et al.
[2020], imaging Schmitz et al.
[2018], Heitz et al.
Zheng et al.
[ neuroscience Janati et al.
[2020], Koundal et al.
Regularized approaches to OT.
Solving OT problems at scale poses, however, formidable challenges.
The most obvious among them is computational: the Kantorovich [1942] problem on discretemeasures of size n is a linear program that requires O ( n 3 log n ) operations to be solved.
A secondand equally important challenge lies in the estimation of OT in high-dimensional settings, since itsuffers from the curse-of-dimensionality Fournier and Guillin [2015].
The advent of regularizedapproaches, such as entropic regularization [Cuturi, 2013], has pushed these boundaries thanks forfaster algorithms
[Chizat et al., 2020, Clason et al., 2021] and improved statistical aspects [Genevayet al., 2018a].
Despite these clear strengths, regularized OT solvers remain, however, costly as theytypically scale quadratically in the number of observations.
Scaling up using low-rank couplings.
While it is always intuitively possible to reduce the sizeof measures (e.g. using k -means) prior to solving an OT between them, a promising line of work proposes to combine both [Forrow et al., 2019, Scetbon et al., 2021, 2022].
Conceptually, theselow-rank approaches solve simultaneously both an optimal clustering/aggregation strategy with thecomputation of an effective transport.
This intuition rests on an explicit factorization of couplings intotwo sub-couplings.
This has several computational beneﬁts, since its computational cost becomeslinear in n if the ground cost matrix seeded to the OT problem has itself a low-rank.
While thesecomputational improvements, mostly demonstrated empirically, hold several promises, the theoreticalproperties of these methods are not yet well established.
This stands in stark contrast to the Sinkhornapproach, which is comparatively much better understood.
Our Contributions.
The goal of this paper is to advance our knowledge, understanding and practicalability to leverage low-rank factorizations in OT.
This paper provides ﬁve contributions, targetingtheoretical and practical properties of LOT: (i) We derive the rate of convergence of the low-rank OTto the true OT with respect to the non-nnegative rank parameter.
(ii)
We provide an upper-bound ofthe (statistical) sample complexity when estimating LOT using the plug-in estimator, and attain aparametric rate O ( p 1 /n )
that is independent of the dimension.
(iii)
We introduce a debiased versionof LOT: as the Sinkhorn divergence [Feydy et al., 2018], we show that it can interpolate betweenthe maximum mean discrepancy [Gretton et al., 2012] and OT, it is nonnegative and it metrizes theweak convergence.
(iv) We exhibit links between the bias induced by the low-rank factorization andclustering methods.
(v) We propose solutions to practical issues such as the tuning the step-length orthe choice of the initialization when applying the algorithm proposed in [Scetbon et al., 2021].
Notations.
We consider ( X , d X ) and ( Y , d Y ) two nonempty compact Polish spaces and we denote
M +1 ( X ) (resp.
M +1 ( Y ) )
the space of positive Radon probability measures on X (resp.
Y ).
For 57 all n  1 , we denote  n the probability simplex of size n and  ⇤ n the subset of  n of positivehistograms.
We write 1 n , (1 , . . .
, 1) T 2 R n
and we denote similarly k · k 2 the Euclidean norm andthe Euclidean distance induced by this norm depending on the context.
Background on Low-rank Optimal Transport
Let µ 2 M +1 ( X ) , ⌫ 2 M +1 ( Y ) and c : X ⇥ Y ! R + a nonnegative and continuous function.
The
Kantorovitch formulation of optimal transport between µ and ⌫ is deﬁned by
[Equation]
where the feasible set is the set of distributions over the product space X ⇥ Y with marginals µ and ⌫ :
[Equation]
with P 1# ⇡ (resp.
P 2# ⇡ ), the pushforward probability measure of ⇡ using the projection maps
P 1 ( x, y )
=
x (resp.
P 2 ( x, y ) =
y ).
When there exists an optimal coupling solution of (1) supportedon a graph of a function, we call such function a Monge map.
In the discrete setting, one canreformulate the optimal transport problem as a linear program over the space of nonnegative matricessatisfying the marginal constraints.
More precisely, let a and b be respectively elements of  ⇤ n and ⇤ m and let also X , { x 1 , . . .
, x n } and Y , { y 1 , . . . , y m } be respectively two subsets of X and Y .
By denoting µ a, X , P ni {+=
+} a i  x i and ⌫ b, Y ,
{+P mj =
+}
b j  y j the two discrete distributions associatedand writing C , [ c ( x i , y j )] i,j , the discrete optimal transport problem can be formulated as
OT c ( µ a, X , ⌫ b, Y ) =
min P 2 ⇧ a,b h C, P i where ⇧ a,b , { P 2 R n ⇥ m + s.t.
P 1 m = a, P T 1 n = b } .
Scetbon et al.
[2021] propose to constrain the discrete optimal transport problem to couplings thathave a low-nonnegative rank:
Deﬁnition 1.
Given M 2 R n ⇥ m + , the nonnegative rank of M is deﬁned by: rk + ( M ) ,min
[Equation]
Note that for any M 2 R n ⇥ m + , we always have that rk + ( M )  min( n, m ) .
For r  1 , weconsider the set of couplings satisfying marginal constaints with nonnegative-rank of at most r as⇧ a,b ( r ) , { P 2 ⇧ a,b , rk + ( P )  r } .
The discrete Low-rank Optimal Transport (LOT) problem is 78 deﬁned by:
[Equation]
To solve this problem, Scetbon et al.
[2021] show that Problem (3) is equivalent to [Equation]
[Equation]
where C 1 ( a, b, r ) , n [Equation] , R, g
[Equation]
⇥ ( [Equation] s.t. Q 1 r = a,
[Equation]
=
[Equation]
o and C 2 ( r ) , n
[Equation]
o . They propose to solve it using a mirror descent scheme and prove the non-asymptotic stationary convergence of their algorithm.
While Scetbon et al.
[2021] only focus on the discrete setting, we consider here its extension for arbitrary probability measures.
Following [Forrow et al., 2019], we deﬁne the set of rank- r couplings satisfying marginal constraints by:
[Equation]
This more general deﬁnition of LOT between µ 2 M +1 ( X ) and ⌫ 2 M +
( Y ) reads:
[Equation]
Note that this deﬁnition of LOT r,c is consistent as it coincides with the one deﬁned in (3) on discreteprobability measures.
Observe also that ⇧ r ( µ, ⌫ ) is compact for the weak topology and therefore theinﬁmum in (5) is attained.
See Appendix A for more details.
Approximation Error of LOT to original OT as a function of rank
Our goal in this section is to obtain a control of the error induced by the low-rank constraint whentrying to approximate the true OT cost.
We provide ﬁrst a control of the approximation error in thediscrete setting.
The proof is given in Appendix B.1.
Proposition 1.
Let n, m  2 , X , { x 1 , . .
, x n } ⇢ X , Y , { y 1 , . . .
, y m } ⇢ Y and a 2  ⇤ n and 89 b 2  ⇤ m .
Then for 2  r  min( n, m ) , we have that
[Equation]
Remark 1.
Note that this result improves the control obtained in [Liu et al., 2021], where they obtainthat | LOT r [Equation] µ [Equation]
OT
[Equation] X [Equation] ⌫
[Equation] Y [Equation] | .
k C k 1 p nm (min( n, m )  r ) as we have for any 92 z
[Equation] It is in fact possible to obtain another control of the approximation error by partitioning the spacewhere the measures are supported.
For that purpose let us introduce the notion of entropy numbers.
Deﬁnition 2.
Let ( Z , d ) a metric space, W ⇢ Z and k  1 an integer.
Then by denoting B Z ( z, " ) ,{ y 2 Z : d ( z, y )  " } , we deﬁne the k -th (dyadic) entropy number of W as
[Equation]
For example, any compact set W of R d admits ﬁnite entropy numbers, and by denoting R ,sup w 2 W k w k 2 , we have N k ( W , k · k 2 )  4 R/ 2 k/d .
We obtain next a control the approximation 99 error of LOT r,c to the true OT cost using entropy numbers (see proof in Appendix B.2).
Proposition 2.
Let µ 2 M +1 ( X ) , ⌫ 2 M +1 ( Y ) and assume that c is L -Lipschitz w.r.t.
x and y .
Then 101 for any r  1 , we have
[Equation]
This results in the following bound for the p -Wasserstein distance for any p  1 on R d .
Corollary 1.
Let d  1 , p  1 , X a compact subspace of R d and µ, ⌫ 2 M [-+
-]+1 ( X ) .
By denoting
R , sup x 2 X k x k 2 , we obtain that for any r  1 ,
[Equation]
As per the Proof of Proposition 2 we can provide a tighter control, assuming a Monge map exists.
Corollary 2.
Under the same assumptions of Proposition 2 and by assuming in addition that thereexists a Monge map solving OT c ( µ, ⌫ ) , we obtain that for any r  1 ,
[Equation]
When X = Y are a subspaces of R d , a sufﬁcient condition for a Monge map to exists is that either µor ⌫ is absolutely continuous with respect to the Lebesgue measure and that c is of the form h ( x  y )where h : X ! R + is a strictly convex function [Santambrogio, 2015, Theorem 1.17].
Therefore if 111 µ is absolutely continuous with respect to the Lebesgue measure, we obtain that for any r, p
[Equation]
Sample Complexity of LOT
We now focus on the statistical performance of the plug-in estimator for LOT.
In the followingwe assume that X = Y for simplicity.
Given µ, ⌫ 2 M +1 ( X ) , we denote the empirical measuresassociated ˆ µ n , 1 n
P n i =
X i and ˆ ⌫ n , 1 n
{+P n i =
 +} Y i , where ( X i , Y i ) n i =1 are sampled independentlyfrom µ ⌦ ⌫ .
We consider the plug-in estimator deﬁned as LOT r, c
(ˆ µ n , ˆ ⌫ n ) , and we aim at quantifyingthe rate at which it converges towards the true low-rank optimal transport cost LOT r,c ( µ, ⌫ ) .
Beforedoing so, in the next Proposition we show that this estimator is consistent on compact spaces.
Theproof is given in Appendix B.3.
Proposition 3.
Let r  1 and µ, ⌫ 2 M [-+
-]+1 ( X ) , then LOT r,c (ˆ µ n
[Equation]
Next we aim at obtaining the convergence rates of our plug-in estimator.
In the following Proposition,we obtain a non-asymptotic upper-bound of the statistical error.
See Appendix B.4 for the proof.
Proposition 4.
Let r  1 and µ, ⌫ 2 M [-+
-]+1 ( X ) .
Then, there exists a constant K r such that for any 124  > 0 and n  1 , we have, with a probability of at least 1  2  , that
[Equation]
This result shows that the estimation of LOT r,c is independent of the dimension and can be performedon general compact metric spaces.
Therefore, LOT r,c presents a clear statistical beneﬁt compared tounregularized OT which suffers from the curse of dimensionality [Dudley, 1969].
In addition, LOTcompares favorably to known results on entropic optimal transport.
The rate of entropy regularized OTdoes not depend on the ambient dimension with respect to n , but carries an exponential dependence indimension with respect to the regularization parameter "
[Mena and Niles-Weed, 2019].
By contrast,the term associated with the nonnegative rank r has no direct dependence on dimension.
Our next aim is to obtain an explicit rate with respect to r and n .
In Proposition 4, we cannot controlexplicitly K r in the general setting.
Indeed, in our proof, we obtain that K r , 14 / min i  ⇤ i where(  ⇤ i ) ri =1 2  ⇤ r are the weights involved in the decomposition of one optimal solution of the true
LOT r,c ( µ, ⌫ ) .
Therefore the control of K r requires additional assumptions on the optimal solutionsof LOT r,c ( µ, ⌫ ) .
In the following Proposition, we obtain an explicit upper-bound of the statisticalerror with respect to r and n in the asymptotic regime.
Proposition 5.
Let r  1 ,  > 0 and µ, ⌫ 2 M [-+
-]+1 ( X ) .
Then there exists a constant N r,  such that if 139 n  N r,  then with a probability of at least 1  2  , we have
[Equation]
Note that one cannot recover the result obtained in Proposition 5 from the one obtained in Propositionin 4 as we have that K r  14 r ! r ! + 1 + 1 .
In order to prove the above result, we use an extensionof the McDiarmid’s inequality when differences are bounded with high probability [Kutin, 2002].
See proof in Appendix B.5 for more details.
Debiased Formulation of LOT
We introduce here the debiased formulation of LOT r,c and show that it is able to distinguish twodistributions, metrize the convergence in law and can be used as a new objective in order to learndistributions.
We focus next on the debiasing terms involving measures with themselves LOT r,c ( µ, µ )in this new divergence, and show that they can be interpreted as deﬁning a new clustering methodgeneralizing k -means for any geometry.
5.1 On the Proprieties of the Debiased Low-rank Optimal Transport
When it comes to learn (or generate) a distribution in ML applications given samples, it is crucial toconsider a divergence that is able to distinguish between two distributions and metrize the convergencein law.
In general, LOT r,c ( µ, µ ) 6 = 0 and the minimum of LOT r,c ( ⌫ , µ ) with respect to ⌫ will notnecessarily recover µ .
In order to alleviate this issue we propose a debiased version of LOT r,c deﬁnedfor any µ, ⌫ 2 M +
( X ) as
[Equation]
Note that DLOT r,c ( ⌫ , ⌫ ) = 0 .
In the next Proposition, we show that, as the Sinkhorn divergence [Genevay et al., 2018b, Feydy et al., 2018], DLOT r,c interpolates between the Maximum Mean
Discrepancy (MMD) and OT.
See proof in Appendix B.6.
Proposition 6.
Let µ, ⌫ 2 M [-+
-]+1 ( X ) .
Let us assume that c is symmetric, then we have
[Equation]
If in addition we assume the c is Lipschitz w.r.t to x and y , then we have
[Equation]
( µ, ⌫ ) . Next, we aim at showing some useful properties of the debiased low-rank OT for machine learningapplications.
For that purpose, let us ﬁrst recall some deﬁnitions.
Deﬁnition 3.
We say that the cost c : X ⇥ X ! R + is a semimetric on X if for all x, x 0 2 X , 163 c ( x, x 0 )
=
c ( x 0 , x ) and c ( x, x 0 )
=
if and only if x = x 0 .
In addition we say that c has a negative typeif 8 n  2 , x 1 , . . .
, x n 2 X and ↵ 1 , . . .
, ↵ n 2 R such that
[-=
 α i = 0 , (cid:80) ni,j =
 α i α j c ( x i , x j ) ≤ 0 .-][Equation]
We say also that c has a strong negative type if for all µ, [-ν ∈ M +
-]⌫ [Equation]
1 ( X
[Equation] ⌫ ] ⌦ [ µ  ⌫ ] < 0 .
Note that if c has a strong negative type, then c has a negative type too.
For example, all Euclideanspaces and even separable Hilbert spaces endowed with the metric induced by their inner productshave strong negative type.
Also, on R d , the squared Euclidean distance has a negative type [Sejdinovicet al., 2013].
We can now provide stronger geometric guarantees for DLOT r,c .
In the next Proposition, we showthat for a large class of cost functions, DLOT r,c is nonnegative, able to distinguish two distributions,and metrizes the convergence in law.
The proof is given in Appendix B.7.
Proposition 7.
Let r  1 , and let us assume that c is a semimetric of negative type.
Then for all µ, ⌫ 2 M +1 ( X ) , we have that
[Equation]
In addition, if c has strong negative type then we have also that
[Equation]
where the convergence of the sequence of probability measures considered is the convergence in law.
Observe that when c has strong negative type, ⌫ ! DLOT r,c ( ⌫ , µ )
and it admits a unique globalminimizer at ⌫ = µ .
Therefore, DLOT r,c has desirable properties to be used as a loss.
[-Let r ≥ 1 and ( µ n ) n ≥ 
-]
5.2 Low-Rank Transport Bias and Clustering
We turn next to the debiasing terms appearing in DLOT and exhibit links between LOT and clusteringmethods.
Indeed, in the discrete setting, the low-rank bias of a probability measure µ deﬁned as
LOT k,c ( µ, µ ) can be seen as a generalized version of the k -means method for any geometry.
In thenext Proposition we obtain a new formulation of LOT k,c ( µ, µ ) viewed as a general clustering methodon arbitrary metric space.
See proof in Appendix B.8.
Proposition 8.
Let n  k  1 , X , { x 1 , . . .
, x n } ⇢ X and a 2  ⇤ n .
If c is a semimetric of negative 185 type, then by denoting C
= ( c ( x i , x j ))
i,j , we have that
[Equation]
Let us now explain in more details the link between (15) and k -means.
When X is a subspace of R d ,c is the squared Euclidean distance and a = 1 n , we recover exactly the k -means algorithm.
Corollary 3.
Let n  k  1 and X , { x 1 , . .
, x n } ⇢ R d .
We have that
[Equation]
In the general setting, solving LOT k,c ( µ a, X , µ a, X ) for a given geometry c , and a prescribed histrogram a offers a new clustering method where the assignment of the points to the clusters is determinedby the matrix Q ⇤ solution of (15).
Computing LOT: Adaptive Stepsizes and Better Initializations
We target in this section practical issues that arises when using [Scetbon et al., 2021, Algo.3] tosolve (4).
Scetbon et al.
[2021] propose to apply a mirror descent scheme with respect to the KullbackLeibler divergence which boils down to solve at each iteration k
the following convex problem:
[Equation]
(7) where ( Q 0 , R 0 , g 0 ) 2 C 1 ( a, b, r ) \ C 2 ( r ) , ⇠ k , ( ⇠ (1) k , ⇠ (2) k , ⇠ (3) k ) , ⇠ (1) k ,
Q k exp( [Equation] [Equation]
[Equation]
(
[Equation]
Q
[Equation]
diag(
[Equation]
g
[Equation]
[Equation]
(3) k , g k  exp(  k ! k /g 2 k )with [ ! k ] i , [ Q Tk CR k
i,i for all i 2 { 1 , . . .
, r } , KL( w , r ) ,
P i w i log( w i /r i ) and (  k ) k  0 is asequence of positive step sizes.
In the general setting, each iteration of their algorithm requires
O ( nmr ) operations and when the ground cost matrix C admits a low-rank factorization of the form
C = AB T where A 2 R n ⇥ d and B 2 R m ⇥ d with d ⌧ min( n, m ) , then the total complexity periteration becomes linear O (( n + m ) rd ) .
In the following we investigate two practical aspects of thealgorithm: the choice of the step sizes and the initialization.
Adaptive choice of  k .
Scetbon et al.
[2021] show experimentally that the choice of (  k ) k  0 does not impact the solution obtained upon convergence, but rather the speed at which it is attained.
Indeed the larger  k is, the faster the algorithm will converge.
As a result, their algorithm simply relies on a ﬁxed  schedule.
However, the range of admissible  depends on the problem considered and it may vary from one problem to another.
It may be of particular interest for practitioners to have a generic range of admissible values for  independently of the considered problem, in order to alleviate parameter tuning issues.
We propose to consider instead an adaptive choice of (  k ) k along iterations.
D’Orazio et al.
[2021], Bayandina et al.
[2018] have proposed adaptive mirror descent schemes where, at each iteration, the step-size is normalized by the squared dual-norm of the gradient.
Applying such a strategy in our case amounts to consider at each iteration
[Equation]
where the initial  > 0 is ﬁxed.
We recommend to set such as global  2 [1 , 10] , and observe thatthis range works whatever the problem considered.
[Equation]
where C X,Z = ( c ( x i , z j ))
i,j , C Y,Z
=
( c ( y i , z j ))
i,j , and H ( P ) =  P i,j P i,j (log( P i,j  1) .
Inpractice we ﬁx " = 1 / [-
-]10 and we then initialize LOT r,c using ( Q, R ) solution of (8) and g ,
Q T 1 r (= R T 1 r ) .
Note that ( Q, R, g ) is an admissible initialization and ﬁnding the centroids as wellas solving (8) requires O (( n + m ) r ) algebraic operations.
Therefore such initialization does notchange the total complexity of the algorithm.
In the general (non-Euclidean) case, we propose to initialize the algorithm by applying our generalizedk -means approach deﬁned in (15) on each input measure where we ﬁx the common marginal tobe g = 1 r /r .
More precisely, by denoting C X,X =
( c ( x i , x j ))
i,j and C Y,Y =
( c ( y i , y j ))
i,j , weinitialize the algorithm by solving:
[Equation]
Note that again the ( Q, R, g ) obtained is an admissible initialization and the complexity of solving (9)is of the same order as solving (4), thus the total complexity of the algorithm remains the same.
Experiments
In this section, we illustrate experimentally our theoretical ﬁndings and show how our initializationprovide practical improvements.
For that purpose we consider 3 synthetic problems and one realworld dataset to: (i) provide illustrations on the statistical rates of LOT r,c , (ii) exhibit the gradientﬂow of the debiased formulation DLOT
Statistical rates.
We aim at showing the statistical rates of the plug-in estimator of LOT r,c .
As
LOT r,c ( µ, µ ) 6 = 0 and as we do not have access to the this value given samples from µ , we consider or L ( µ ) , LOT r,c ( µ, ⌫ ) .
The ground cost is the squared Euclidean distance and we ﬁx r = 100 .
We consider 1000 samples from each distribution and and we plot the evolution of the probability measure obtained along the iterations of a gradient descent scheme.
We also display in green the vector ﬁeld in the descent direction.
We show that the debiased version allows to recover the target distribution while LOT r,c is learning a biased version with a low- instead the debiased version of the low-rank optimal transport, DLOT r,c .
In ﬁgure 1, we show thatthe empiricial rates match the theoretical bound obtained in Proposition 4.
In particular, we show thatthat these rates does not depend on the dimension of the ground space.
Gradient Flows using DLOT.
We illustrate here a practical use of DLOT for ML application.
Inﬁgure 2, we consider Y 1 , . . .
, Y n independent samples from a moon shape distribution in 2D, and bydenoting ˆ ⌫ n the empirical measure associated, we show the iterates obtained by a gradient descentscheme on the following optimization problem:
[Equation]
We initialize the algorithm using n = 1000 samples drawn from a Gaussian distribution.
We showthat the gradient ﬂow of our debiased version is able to recover the target distribution.
We alsocompare it with the gradient ﬂow of the biased version (LOT) and show that it fails to reproduce thetarget distribution as it is learning a biased one with a low-rank structure. , we consider the shortest path distance on the graph associated with the ground cost c ( x, y ) = 1  k ( x, y ) where k is a Gaussian kernel.
In the two ﬁrst problem (starting from the left), we ﬁx r = 2 , in the next three problem we ﬁx r = 3 and in the last one we ﬁx r = 4 .
We observe that the ﬂexibility of our method allows to recover the clustering for a well chosen ground cost.
Application to Clustering.
In this experiment we show some applications of the clustering methodinduced by LOT r,c .
In ﬁgure 3, we consider 6 datasets with different structure and we aim atrecovering the clusters using (15) for some well chosen costs.
We compare the clusters obtained when considering either the squared Euclidean cost (which amounts at applying the k -means) and theshortest-path distance on the data viewed as a graph.
We show that our method is able to recover theclusters for well chosen costs and therefore [2021] can be seen as a alternative in order to
Effect of the Initialization.
Our goal here is to show the effect of the initialization.
In ﬁgure 4, wedisplay the evolution of the cost as well as the value of the stopping criterion along the iterationsof the MD scheme solving (4) when considering different initialization.
Recall that the stoppingcriterion introduced in [Scetbon et al., 2021] is deﬁned for all k  1 by
[Equation]
))) , where (( Q k , R k , g k ))
k  0 is the sequence solution of (7).
We show that whatever the initializationchosen, the algorithm manages to converge to an efﬁcient solution if no stopping criterion is used.
However, the choice of the initialization may impact the termination of the algorithm as someinitialization might be too close to some spurious local minima.
We show also that the initializationwe propose in (8) and (9) are sufﬁciently far away from bad local minima and allow the algorithm toconverge directly toward the desired solution.
Conclusion
We assembled in this work theoretical and practical arguments to support low-rank factorizationsfor OT.
We have presented two controls: one concerning the approximation error to the true optimaltransport and another concerning the statistical rates of the plug-in estimator.
The latter is showed tobe independent of the dimension, which is of particular interest when studying OT in ML settings.
We have motivated further the use of LOT as a loss by introducing its debiased version and showedthat it possesses desirable properties: positivity and metrization of the convergence in law.
We havealso presented the links between the bias induced by such regularization and clustering methods,and studied empirically the effects of hyperparameters involved in the practical estimation of LOT.
The strong theoretical foundations provided in this paper motivate further studies of the empiricalbehaviour of LOT estimator, notably on ﬁnding suitable local minima and on improvements on theconvergence of the MD scheme using other adaptive choices for step sizes.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect
the
paper’s
contributions and scope?
[Yes] See Sections 3,4,5,6.
(b) Did you describe the limitations of your work?
[Yes] See Section 8.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sec-
tions 3,4,5.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix.3.
If you ran experiments...
[-Then by definition there exists for all k ∈ [ | 1 , r | ] ,( µ ( k ) n ) n ≥ 0 , ( ν ( k ) n ) n ≥ 0 and ( λ ( k ) n ) n ≥ 0 such that for all n ≥ 
-]
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 7.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[-ν ( S k, Y ) with the convention that 
-]
[Yes] See Section 7.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[-with the convention that 0 
-]
[Yes] See Section 7.
[-Let r ≥ 1 and µ, ν ∈ M +
-]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[-x ∈X ∈ M +
-]
[-i ≥ 
-]
[-i ≥ 
-]
[-In addition we have that for all n ≥ 
-]
[Yes]
[-Proposition. Let r ≥ 1 and µ, ν ∈ M +
 ( X ) .-](b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
[-Let r ≥ 1 , δ > 0 and µ, ν ∈ M +
-]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[-Let µ, ν ∈ M +
-]
[N/A]
[-Let r ≥ 1 and ( µ n ) n ≥ 
-]
If you used crowdsourcing or conducted research with human subjects...
[-(cid:80) r i =
 λ i µ ( i ) . Then for any sequence of probability measures ( µ n ) ≥ 0 such that µ n → µ , thereexist for all i ∈ [ | 1 , r | ] a sequence-](a) Did you include the full text of instructions given to participants and screenshots, if
[-By denoting ˜ µ (1) ≜ (cid:80) ri =
-]
applicable?
[-n | ˜ µ (1) n | = r (cid:88) i = µ ( i ) n where µ ( i ) n ≥ 0 and µ ( i ) n → λ i (cid:80) ri =
-]
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[-Therefore ( x ∗ i − y ∗ i ) T 1 n = 
-]
[N/A]