Low-rank Optimal Transport: Approximation, Statistics and Debiasing
Authors: Meyer Scetbon
marco cuturi
Venue: NeurIPS 2022 Accept (2022)
Keywords: Low-rank Optimal Transport:  Approximation
Statistics and Debiasing

Abstract: The matching principles behind optimal transport (OT) play an increasingly important role in machine learning, a trend which can be observed when OT is used to disambiguate datasets in applications (e.g. single-cell genomics) or used to improve more complex methods (e.g. balanced attention in transformers or self-supervised learning). To scale to more challenging problems, there is a growing consensus that OT requires solvers that can operate on millions, not thousands, of points. The low-rank optimal transport (LOT) approach advocated in \cite{scetbon2021lowrank} holds several promises in that regard, and was shown to complement more established entropic regularization approaches, being able to insert itself in more complex pipelines, such as quadratic OT. LOT restricts the search for low-cost couplings to those that have a low-nonnegative rank, yielding linear time algorithms in cases of interest. However, these promises can only be fulfilled if the LOT approach is seen as a legitimate contender to entropic regularization when compared on properties of interest, where the scorecard typically includes theoretical properties (statistical complexity and relation to other methods) or practical aspects (debiasing, hyperparameter tuning, initialization). We target each of these areas in this paper in order to cement the impact of low-rank approaches in computational OT.

Low-rank Optimal Transport: Approximation, Statistics and Debiasing
Meyer Scetbon CREST, ENSAE
Marco Cuturi Apple and CREST, ENSAE cuturi@apple.cmm com
meyer. wewtton@esseeee.ee......
Abstract
The matching principles behind optimal transport (OT) play increasingly impor- tant role in machine learning, a trend which can be observed when OT used to disambiguate datasets in applications (e.g. single-cell genomics) or used improve more complex methods (e.g. balanced attention in transformers or self-supervised learning). To scale more challenging problems, there a growing consensus that OT requires solvers that can operate on millions, not thousands. of points. The low- rank optimal transport (LOT) approach advocated in [Seeetonn et al. 2222 holds several promises in that regard, and was shown to complement more established entropic regularization approaches. being able to insert itself in more complex pipelines, such as quadratic OT. LOT restricts the search for low-cost couplings those that have a low-nonnegative rank, yielding linear time algorithms in cases of interest. However, these promises can only be fulfilled if the LOT approach is seen as a legitimate contender to entropic regularization when compared on properties of interest, where the scorecard typically includes theoretical properties (statistical complexity and relation to other methods) or practical aspects (debiasing, hyperparameter tuning, initialization). We target each of these areas in this paper in order to cement the impact of low-rank approaches in computational OT.
Introduction
Optimal transport (OT) is used across data-science to put in correspondence different sets of observa- tions. These observations may come directly from datasets, or, in more advanced upplications, depict intermediate layered representations of data. OT theory provides a single grammar to describe and
solve increasingly complex matching problems (linear, quadratic, regularized, unbalanced, etc...),
making it gain a stake in various areas of science such as as single-cell biology Schhebinger et al.
(2099] Yang al. 202(1) , Demetei al. 20200) imaging Schmitz al. 2018 Heitz al. 20200]
Zheng et al. 2020 or neuroscience Janati et al.
2020 Koundal et al.
2020
Regularized approaches to OT. Solving OT problems at scale poses, however, formidable chal-
lenges. The most obvious among them is computational: the Kantorovich [142]] problem on discrete measures of size is linear program that requires (n log n) operations to be solved. A second and equally important challenge lies in the estimation of OT in high-dimensional settings, since
suffers from the curse-of-dine dimensionnlltty FFurnier and Guillin 2015]. The advent of regularized approaches, such as entropic regularization Cuturri 2013 has pushed these boundaries thanks for faster algorithms (Scetbon and Cuturi, 2020
Chizat al. 2020 Clason al. 2021 and improved
statistical aspects [Genevay et al.) 201 Sa]. Despite these clear strengths, regularized OT solvers
remain, however, costly as they typically scale quadratically in the number of observations.
Scaling up OT using low-rank couplings. While is always intuitively possible to reduce the size of measures (e.g. using k-means) prior to solving an OT between them, promising line of Conceptually, these
work proposes to combine both [Forrow et al. 2019 Scetbon et al. 2021 2022
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

low-rank approaches solve simultaneously both an optimal clustering/aggregatinn strategy with the computation of an effective transport. This intuition rests on an explicit factorization of couplings into two sub-couplings. This has several computational benefits, since its computational cost becomes linear in n if the ground cost matrix seeded to the OT problem has itself a low-rank, While these computational improvements, mostly demonstrated empirically, hold several promises, the theoretical properties of these methods are not yet well established. This stands in stark contrast to the Sinkhorn approach, which is comparatively much better understood.
Our Contributions. The goal of this paper to advance our knowledge, understanding and practical ability to leverage low-rank factorizations in OT. This paper provides five contributions, targeting theoretical and practical properties of LOT: (i) We derive the rate of convergence of the low-rank OT to the true OT with respect to the non nnegative rank parameter. (ii) We make a first step towards a better understanding of the statistical complexity of LOT by providing an upper-hound of the statistical error, made when estimating LOT using the plug-in estimator; that upper- hpper-hound has a parametric rate (iii) We introduce debiased version of LOT: as the Sinkhorn divergence [Feydy et al.) 2018), we show that debiased LOT is nonnegative, metrizes the weak convergence, and that it interpolates between the maximum mean discrepancy [Gretton et al. 2012] and OT. (iv) We exhibit links between the bias induced by the low-rank factorization and clustering methods. (v) We propose practical strategies to tune the step-length and the initialization of the algorithm in (Seetbon et 20211.
O(/I/)) that is independent of the dimension.
Notations, We consider (X' dx) and ()', dyy two nonempty compact Polish spaces and we denote MI (X)) (resp. .M (J?)) the space of positive Radon probability measures on X' (resp. }). For all n 2 1, we denote A,, the probability simplex of size n and A; the subset of A, of positive histograms. We write 1, - (1,..., 1)T e R" and we denote similarly || lla the Euclidean norm and the Euclidean distance induced by this norm depending on the context.
2 Background on Low-rank Optimal Transport
Let Mi)),, E Mi ()) and e X' x y + R4 nonnegative and continuous function. The Kantorovitch formulation of optimal transport between μ and is defined by
/ Jxxy c(æ, y)dT(I; 8),
min OTo(u, <TI((M)))
(1)
where the feasible set is the set of distributions over the product space X' x y with marginals μ and v:
11( v) & { E M x y) s.t. PI#x = μ. P2# = [/
with Pr (resp. P2 ) the pushforward probability measure of using the projection maps Pi (x, y) = I (resp. P2 y) = y). When there exists an optimal coupling solution of ( supported graph of function,
call such function Monge map. In the discrete setting, one can
reformulate the optimal transport problem as a linear program over the space of nonnegative matrices satisfying the marginal constraints. More precisely, let a and b be respectively elements of A; and A and let also X é {fj...., and Y & {yI..... }m be respectively two subsets of X' and V. By demoting Ma,x A EE az,r, and V/,Y En= by the two discrete distributions associated and writing C' A [e(0,, y,),e, the discrete optimal transport problem can be formulated as
OTo 1b, 1),Y
min (C,P) where 11.,b A {P' E R 1 s.t. Plm a, 1",,, -b. (2)
Pell, At
Scetbon et al. 2021 propose to constrain the discrete optimal transport problem to couplings that have a now-nonneeative rank:
Definition 1. Given M E Raxm the nonnegative rank of M is defined by: rk4 (M) a minfg|M E'_ R2, Vi, (k(R)) 1, 0).
Note that for any M- E RTX=" we always have that (M) < min(n, m). For r 2 1, we consider the set of couplings satisfying marginal constaints with nonnegative-rank of at most r as (1),(()) & {P E H1 (P) < r). The discrete Low-rank Optimal Transport (LOT) problem is defined by:
LOT, LOT, A min (C, P). pell..((r)
(3)
2

---

To solve this problem, Scetbon et al. 2021 show that Problem B is equivalent to
min (Q. (Q..g)eCi(akarnnnnn
(C, Q diag(1/9)
(4)
A {(0) R, E R x x x RC S.I. Q² = R' = 9) They propose to solve it using mirror descent scheme and prove the non-asympiotic stationary convergence of their algorithm. While Scetboo et al. [202T] only focus on the discrete setting, we consider here its extension for arbitrary probability measures. Following Forrow et 2019]. we define the set of rank-r couplings satisfying marginal constraints by:
where Ci Ci b, A R0, E Rax x RR+x x (R+) )' s.t. Q1, a, Rl, =b and C2(r)
(1, & { 11(,,))) v): 3(4)=1 E Mi (m)) MI))", A A Σ App
This more general definition of LOT between μ E Mi and v E Mf reads:
inf / dx,
LOT r., rEll.
(5)
Note that this definition of LOT,,, r is consistent as coincides with the one defined in B on discrete probability measures. Observe also that (u, v/) is compact for the weak topology and therefore the infimum (5) is attained. See Appendix A for more details.
Approximation Error of LOT to original OT as a function of rank
Our goal in this section is to obtain control of the error induced by the low-rank constraint when trying to approximate the true OT cost. We provide first control of the approximation error in the discrete setting. The proof is given in Appendix B.1
Let n, 2 X A {II...., In) X,,, Y & ( 3m C and E A and b E Ai.. Then for 2 S S min m), we have that
Proposition 1.
o (Ma, vb, (e., va,Y)) S [|C| In(min((, m)/(r 1))
Remark 1. Note that this result improves the control obtained in [Luu et al. 2021y, where they obtain that T..OT,,. v) OT OT x, v,Y )l < ||C|l vnm((min(r) m) r) as we have for any 2,2' 2 1, In(2) In(z
=/
It is in fact possible obtain another control of the approximation error by partitioning the space where the measures are supported. For that purpose let us introduce the notion of entropy numbers.
Definition 2. Let (2, d) metric space, W C 2 and k 2 1 an integer. Then by denoting Bz Bz E) & E 2: d(z, v) I E), we define the k-tth (dyadic) entropy number of W as
NeW, d) A inffe s.s. 2 22t Z W c UC
For example, any compact set W of Rd admits finite entropy numbers, and by denoting R & supwew ||w||2, we have N4 (MV, || l1l) < 4R/22/d, We obtain next a control of the approximation error of LOT,,e the true OT cost using entropy numbers (see proof in Appendix B.2)
Proposition 2. Let MI (X),, , v Mf(()) and assume that e is L-Lipschitz w.t.. x and y. Then for any r 2 we have
[O, v) (TT(,, v)1 S 2L (10(( dx, (ogg (LVF () (D)
This results in the following bound for the p- Wasserstein distance for any p 2 on Rd. Corollary 1. Let d 1, p 2 X compact subspace of IRd and p.v E M (2). By denoting
R é supzex ||]]| we obtain that for any r 2
(
LOT (1T.. OT 1-115 (H, S 4dp pp/2d
3

---

As per the Proof of Proposition 2 we can provide a tighter control, assuming a Monge map exists.
Corollary 2. Under the same assumptions of Proposition and by assuming in addition that there exists a Monge map solving OT,(u, v), we obtain that for any 2 1,
LOTr,((uVV) - OT,(,, v\1 - (N
When X' ) are subspaces of R a sufficient condition for a Monge map to exists is that either j
or is absolutely continuous with respect to the Lebesgue measure and that c is of the form h(x - y)
where h X' - RR+ is strictly convex function [Santambrogio 2015, Theorem 1.17]. Therefore
is absolutely continuous with respect to the Lebesgue measure, we obtain for any r 2 1 and p > 1
AI - OT 1-11 (1, v), S 2dp (8R
rP/d
4 Sample Complexity of LOT
We now focus on the statistical performance of the plug-in estimator for LOT. In the following assume that X' )/ for simplicity. Given Jr, (MM(()) (X)), denote the empirical measures associated fi, - 1 Σn= 5x; and i) - 1 Ex= 5y, where (X,, Y.);== are sampled independently n from 2 v. We consider the plug-in estimator defined as LOT,, (An. Pr,,, and we aim at quantifying the rate at which converges towards the true low-rank optimal transport cost Lo,,,e 2). Before doing so, in the next Proposition we show that this estimator is consistent on compact spaces. The proof is given in Appendix B33
Proposition 3. Let 21 and jr, U (x),, then T,T,,, Pa)
LOT,, v) a.s.
a-++t
Next we aim at obtaining the convergence rates of our plug-in estimator. In the following Proposition, we obtain non-asymptotic upper-bound of the statistical error. See Appendix B.4 the proof.
Proposition 4. Letr 2 and μ,v v E Mi (X). Then, there exists constant K, such that for any 6 > 0 and n 2 1, we have with a probability of at least 1 26, that
/ _ Kellello n
log(40/5)
/r Follog400 (00/))
LOT 0n) s LOT,, + 11/lc//
This result is, to the best of our knowledge, the first attempt providing statistical control of low-rank optimal transport. We provide an upper-bound of the plug-in estimator which converges towards LOT,,e parametric rate and which is independent of the dimension on general compact metric spaces. While we fall short of providing a lower bound that could match that upper bound, and therefore provide complete statistical complexity result, we believe this result might provide first explanation on why, in practice. LOT, r.e displays better statistical properties than unregularized OT and its curse of dimensionality [Dudley 1969]. In addition, that upper bound compares favorably to known results on entropic optimal transport. The rate of entropy regularized OT does not depend on the ambient dimension with respect to n. but carries an exponential dependence in dimension with respect to the regularization parameter [Mena and Niles- Weed 2019]. By contrast, the term associated with the nonnegative rank in our bound has no direct dependence on dimension.
Our next aim is to obtain an explicit rate with respect to r and n. In Proposition g we cannot control explicitly K, in the general setting. Indeed, in our proof, we obtain that K,, 14/ min; X where (X)) A are the weights involved in the decomposition of one optimal solution of the true LOT,, v). Therefore the control of K, requires additional assumptions on the optimal solutions of LOT, (u, v). In the following Proposition, we obtain an explicit upper-bound of the plug-in estimator with respect to r and n in the asymptotic regime.
Proposition 5. Let r 2 1, ō > 0 and p., v E MM (X). Then there exists a constant Nr.6 such that if Nr.5 then with a probability of at least 1 28, we have
r + 77/1
log (0//))
(Pn Pn) < LOT,, v) + 11|ld|l

---

Note that one cannot recover the result obtained in Proposition firom the one obtained in Proposition
as we have that K, 14r
r+++2 - +30. In order prove the above result, we use an extension of
the McDiarmid's inequality when differences are bounded with high probability Kutin 2002 See proof in Appendix B.5 for more details.
5 Debiased Formulation of LOT
We introduce here the debiased formulation of LOTr,e and show that is able to distinguish two distributions, metrize the convergence in law and can be used as new objective in order to learn distributions. We focus next on the debiasing terms involving measures with themselves LOTr, μu) in this new divergence, and show that they can be interpreted as defining a new clustering method generalizing k-means for any geometry.
5.1 On the Proprieties of the Debiased Low-rank Optimal Transport
When it comes to learn (or generate) a distribution in ML applications given samples, it crucial to consider divergence that is able distinguish between two distributions and metrize the convergence in law. In general, LOT,, (,,,, (4) & 0 and the minimum of IOTr,,( μ1) with respect to will not necessarily recover μr. In order alleviate this issue propose debiased version of LOT,, r.c defined for any v.v E MM (X)
(L, & LOT,,c(u,2 1 LOT,, +
Note that DLOT, v) 0. In the next Proposition, we show that, as the Sinkhorn diver- gence Genevay et al. 2018b Feydy et al. 2018), DLOT,, interpolates between the Maximum Mean Discrepancy (MMD) and OT. See proof in Appendix [.66
Proposition 6. Let μ. E Mi (X)). Let assume that is symmetric, then we have
1 [
DLOT v)
-c(x, v) & dlu v)((,
2 x2
If in addition we assume the e is Lipschitz w.r.t to x and y, then we have DLOT (,(,,,
--+000 OT OTe(H,)) v).
Next, aim at showing some useful properties of the debiased low-rank OT for machine learning applications. For that purpose, let us first recall some definitions.
Definition 3. We say that the cost c X x X' R4 is semimetric on X if for all x, E X, x() = =c(x', x)) and c(r, = Of if and only if = x'_ In addition we say that has negative type vn 2 2, 21,..., En X and on,..., On E R such that Ei= a; 0. Σx,=1 0,a,c([1, F2)) O. We say also that e has a strong negative type if for all μ, M+ v] @ v] O.
= Sxx c(z, vddlu
Note that if c has a strong negative type, then c has a negative type too. For example, all Euclidean spaces and even separable Hilbert spaces endowed with the metric induced by their inner products have strong negative type. Also, on Rd, the squared Euclidean distance has negative type (Sejdinovic al. 2013
We can now provide stronger geometric guarantees for DLOT, r.c- the next Proposition, we show that for large class of cost functions, DI.T,,, is nonnegative, able to distinguish two distributions, and metrizes the convergence law. The proof is given in Appendix B.8
Proposition 7. Let 1, and let us assume that is a semimetric of negative type. Then for all Jr, E MT (2)), we have that
DLOT- (p, 0. In addition, if has strong negative type then we have also that DLOT v) = + P === V and
pn 7 4 DLOT,,c 11) +0
where the convergence of the sequence of probability measures considered is the convergence in law.
5

---

Observe that when has strong negative type, DLOT,,o (4, 2 and it admits unique global minimizer at // j., Therefore, DLOT,, has desirable properties be used as a loss. It is also worth noting that, in order to obtain the metrization of the convergence in law, we show the following Proposition. See proof Appendix B.7
Proposition 8. Let that pn 7 μ and Vn + with respect to the convergence in law. Then we have that
and (1n)a20 and (un)n20 two sequences of probability measures such
o. vn)) - LOTroo(u,
5.2 Low-Rank Transport Bias and Clustering
We turn next to the debiasing terms appearing in DLOT and exhibit links between LOT and clustering methods. Indeed, in the discrete setting, the low-rank bias of a probability measure js defined as LOT 14) can be seen as generalized version of the k-means method for any geometry. In the next Proposition we obtain a new formulation of L (M, (J) viewed as a general clustering method on arbitrary metric space. See proof in Appendix B.9
Proposition 9. Let type, then by denoling C = (c( F3y) ))3. we have that
1, X (II, (
X and E Ai If is semimetric of negative
LOTR,o xx,x) min (ddiag(1/""1)))
QERT .QI= a
(6)
Q
Let explain in more details the link between and k-means. When X' is subspace of R the squared Euclidean distance and a = 1n we recover exactly the k-means algorithm.
Corollary 3. Let 2 k 2 and X & II In) c RRd, We have that
LOT k. H (11,,, Ma,x)
min ΣΣ
Qiglaa 2gl12 s.t. E (0,1)xx Q1 =I,
Q.a1
In the general setting, solving LM.. for given geometry e, and prescribed histro- gram offers new clustering method where the assignment of the points to the clusters is determined by the matrix Q" solution of 6
Computing LOT: Adaptive Stepsizes and Better Initializations
We target in this section practical issues that arises when using [Scetbon et al. 2021
Algo.3| to
solve Scetbon et al. [021 propose to apply a mirror descent scheme with respect to the Kullback- Leibler divergence which boils down to solve at each iteration k 2 0 the following convex problem
using the Dykstra's Algorithm [Dykstra 1983
argmin KL(C,Ex) Rec11, (ECh(a.)r)nc((r)
where (Qu, Ro, 90) E C, n C2(r). & A ( (3)
Skk 5
(())), 56
Qx 1
exp(-74C1 CR. Rs diag(1/ge)) c.2 A R& exp(pxn Qn diag(1 diag(1/9))) S 2 ga O exp((nww//0 with [wn]: 2 1Q CRyji, for all rI. KL(w,r) & E, w; log(w,/r:) and (T))20 o is sequence of positive step sizes. In the general setting, each iteration of their algorithm requires ( operations and when the ground cost matrix C' admits a low-rank factorization of the form C AB" where A E Raxq and B E mx with < min(n, m), then the total complexity per iteration becomes linear O((n + m)rg). Note that for the squared Euclidean cost on Rdd, we have that g = d ++ In the following we investigate two practical aspects of the algorithm: the choice of the step sizes and the initialization.
A daptive choice of 7/k Scetbon et al. [021 show experimentally that the choice of (74) does not impact the solution obtained upon convergence, but rather the speed at which it is attained. Indeed the larger "/ is, the faster the algorithm will converge. result, their algorithm simply relies on fixed schedule. However, the range of admissible depends on the problem considered and it may

---

vary from one problem to another. Indeed, the algorithm might fail to converge as one needs to ensure at each iteration of the mirror descent scheme that the kernels & do not admit 0 entries in order to solve using the Dykstra's s Algorithm. Such a situation can occur when the terms involved in the exponentials become too large which may depend on the problem considered. Therefore, it may of particular interest for practitioners to have generic range of admissible values for independently of the considered problem, in order to alleviate parameter tuning issues. We propose to consider instead an adaptive choice of (TT))20 along iterations. D'Orazio et al. [2021), Bayandina et al. (2018) have proposed adaptive mirror descent schemes where, at each iteration, the step-size is normalized by the squared dual-norm of the gradient. Applying such a strategy in our case amounts to consider at each iteration
?
7 [[CCRiiag(1/)) CT Q -(Q"))))))))))/////) I
where the initial 7 > O is fixed. By doing so, we are able to guarantee a lower-bound of the exponential terms involved in the expression of the kernels & at each iteration and prevent them from having 0 entries. We recommend to set such as global y E [1, 10), and observe that this range works whatever the problem considered.
On the choice of the initialization. As LOT r.c is non-convex optimization problem, the ques- tion choosing an efficient initialization arises in practice, (Scethon et al. [022] show experimentally that the convergence of the algorithm does not depend on the initalization chosen if no stopping criterion is used. Indeed, their experimental findings support that only well behaved local minimas attractive. However, in practice one needs to use stopping criterion in order to terminate the algorithm. We do observe in many instances that using trivial initializers may result in spurious local minima, which trigger the stopping criterion early on and prevent the algorithm to reach good solution. Based on various experimentations, we propose to consider a novel initialization of the algorithm. Our initialization aims at being close to a well-behaved local minimum by clustering the input measures. When the measures are supported on Euclidean space, we propose to find r centroids (= of one of the two input discrete probability measures using k-means and solve the following convex barycenter problem:
min (ix,2, + (Cy,z, R3 - (Q) - (R) s.t. Q1 = =a R1, = Q" = 1TT,, Q.
where Cx,z = (c(xi, zz))s.j- Cr,z = (clys, 2,)),s, and H(P) = Eng (log( Pi, - 1). In practice we fix E = 1/10 and we then initialize LOT,,c using (Q, solution of ( and - 1² 1,). Note that (Q, R. g) is an admissible initialization and finding the centroids as well as solving D requires O((n + m)r) algebraic operations. Therefore such initialization does not change the total complexity of the algorithm.
In the general Mhon-Euclidean case, we propose to initialize the algorithm by applying our generalized k-means approach defined in (6) on each input measure where we fix the common marginal to be
= 1,/r. More precisely. by denoting Cx.x (c(x1, and Cy,y (c(yi, 4; ),j. we
initialize the algorithm by solving:
Q E
s.t. Q E Rank Q1 = a, Q² = 1,/r.
(10)
R. E argmiin(yy, (Crr,,,,a,iggL/L R²) s.t. Re RR R1, b, R 1, 1,/r.
It
Note that again the (Q, R. obtained is an admissible initialization and the complexity of solving of the same order as solving 4. thus the total complexity of the algorithm remains the same.
1o
Experiments
In this section, we illustrate experimentally our theoretical findings and show how our initialization provide practical improvements. For that purpose we consider synthetic problems and one real world dataset to: (i) provide illustrations on the statistical rates of LOT, r.. (ii) exhibit the gradient flow of the debiased formulation DLOT,, (iā) use the clustering method induced by LOT, and (iv) show the effect of the initialization. All experiments were run MacBook Pro 2019 laptop.
7

---

dimension = 2
dimension = 5
dimension = 7
dimension = 10
10a
10 1
10a
DLOT
10c
10-1
10-1
10-2
10-2
101
10
10
10
10
10
10
102
10
10
10'
10a
sample
sample
sample
sample
Upper Bound (r= 1)
DLOT:
DLOT: 5
DLOT: r= 100
Figure 1: In this experiment, we consider mixture of 10 anisotropic Gaussians supported on R and we plot the value of D between two independent empirical measures associated to this mixture when varying the number of samples and the dimension d for multiple ranks The ground cost considered the squared Euclidean distance. Note that LOT, (p, μ1) * and therefore we use DLOT, ( 14) instead to evaluate the rates. Each point has been obtained by repeating 10 times the experiment. We compare the empirical rates obtained with the theoretical one derived in Proposition for 1. We observe that our theoretical results match the empirical ones and, as expected, the rates do not depend on d.
DLOT, r=100
LOT. r=100
Figure 2: We compare the gradient flows (24)20 (in red) starting from Gaussian distribution, 14o- to a moon shape distribution (in blue), v, in 2D when minimizing either L(Hx) A DLOT,,, v) or L(n) & LOT,,e(p, v). The ground cost is the squared Euclidean distance and we fix r = 100. We consider 1000 samples from each distribution and and we plot the evolution of the probability measure obtained along the iterations of a gradient descent scheme. We also display in green the vector field in the descent direction. We show that the debiased version allows recover the target distribution while LOT, r.e is learning a biased version with a low-rank structure.
Statistical rates. We aim at showing the statistical rates of the plug-in estimator of LOT, As (0,, (a) * 0 and as we do not have access to this value given samples from μs, we consider instead the debiased version of the low-rank optimal transport, DLOT, r.c In figure [] we show that the empiricial rates match the theoretical bound obtained in Proposition 4 In particular, we show that that these rates does not depend on the dimension of the ground space. Note also that recover our theoretical dependence with respect to the rank r: the higher the rank, the slower the convergence.
Gradient Flows using DLOT. We illustrate here a practical use of DLOT for ML application. In figure 5. we consider ..,..., Yn independent samples from a moon shape distribution in 2D), and by denoting in the empirical measure associated, show the iterations obtained by gradient descent scheme on the following optimization problem:
.a....,,,
min DLOT DLOT,..(M,,,n...... Pn) XER*X2
We initialize the algorithm using n = 1000 samples drawn from Gaussian distribution. We show that the gradient flow of our debiased version is able recover the target distribution. We also compare it with the gradient flow of the biased version (LOT) and show that it fails to reproduce the is learning biased one with low-rank structure.
target distribution
Application Clustering. In this experiment we show some applications the clustering method induced by LOT,, In figure 3] we consider 6 datasets with different structure and we aim at recovering the clusters using (6) for some well chosen costs. We compare the clusters obtained when considering either the squared Euclidean cost (which amounts applying the k-means) and the

---

Figure 3: In this experiment, we draw 1000 samples from multiple distributions from the python package scikit-learn (Pedregosa et al. 2011] and we apply the method proposed in 4 for two different costs: in the top row we consider the squared Euclidean distance while in the bottom row, we consider the shortest path distance on the graph associated with the ground cost c(1, y) 1 k(x, y) where k is Gaussian kernel. the two first problem (starting from the left), we fix r 2, in the next three problem we fix 3 and in the last one we fix r 4. We observe that the flexibility of our method allows to recover the clustering for a well chosen ground cost.
0.5
LOT criterion: A
LOT cost
0.4
10-5
0.3
10"7
0.2
10-9
10
10
10
10a
10
10a
10
Operations
Operations
Inil: kmeans, 1=10
Inil: gkmeans, r=10
Init: bank_2. r=10
Init: random, r=10 Init: random, r-100
Init: kmeans, 1-100
Init: gkmeans, 1-100
Init: rank 2. r=100
Figure In this experiment, we consider the Newsgroup20 dataset [Pedregosa al. 2011 constituted of texts and we embed them into distributions in 50D using the same pre-processing steps as in [Cuturi et al. 2022]. We compare different initialization when applying the algorithm of (Scetbon et al. 2021 compare random texts viewed distributions for multiple choices of rank The times by sampling randomly multiple problems of similar size (0= 250 samples). We normalize the cost matrix by its maximum value in order to have comparable LOT cost. We consider 4 different initialization: the one using k- m-maans algorithm E the using the generalized k-means (TO), the rank-2 initialization [Scetbon et al. 202T] and a random initialization where Q, R and g are drawn from Gaussians. We compare both the cost value and the criterion value (A&) along the iterations of the MD scheme. Note that the curves obtained do not start at the same point in time as we start plotting the curves after obtaining the initial point which in some case requires operations (e.g. kmeans methods). First we observe that whatever the initialization considered, the algorithm converges toward the same value. In addition, we observe that both k-means and general k-means are able to initialize well the algorithm by avoiding bad local minima at initialization while the other initialization are close spurious local minima initialization.
ground cost considered in the squared Euclidean distance. repeat the experiments
algebraic
shortest-path distance the data viewed as graph. We show that method is able to recover the clusters on these settings for well chosen costs and therefore the proposed algorithm in (Setboon et al. 2021] can be seen new alternative in order to clusterize data.
Effect of the Initialization. Our goal here show the effect of the initialization. In figure we display the evolution of the cost as well as the value of the stopping criterion along the iterations of the MD scheme solving 4 when considering different initialization. The æ-axis corresponds to the total number of algebraic operations. This number is computed at each iteration of the outer loop of the algorithm proposed in [Seehhon et al. [202T] and is obtained by computing the complexity of all the operations involved in their algorithm reach it. We consider this notion of time instead of

---

CPU/GPU time as we do not want to be architecture/machine dependent. Recall also that the stopping criterion introduced in (Scetbon al., 2021] defined for all k 2 by
(KI( Rk,, 9x), (Qk-1, 11 + 1I[((K- R: (Qk, Rk, 9k)))
7
where ((Qk, Rik. (k)) is the sequence solution of First, show that whatever the initializa- tion chosen, the algorithm manages to converge to an efficient solution if no stopping criterion is used. However, the choice of the initialization may impact the termination of the algorithm some initialization might be too close to some spurious local minima. Indeed, the initial points obtained using a "rank 2" or random initialization which may trigger the stopping criterion too early and prevent the algorithm from continuing to run in order to converge towards attractive and well behaved local minimum. We show also that the initialization we propose in 9 and 10 are sufficiently far away from bad local minima and allow the algorithm to converge directly toward the desired solution.
be close spurious and non-attractive local minima,
The right figure of Fig.4 shows two main observations: (i) that the initial point obtained using a "rank 2" or random initialization can be close spurious and non-attractive local minima, which may trigger the stopping criterion too early and prevent the algorithm from continuing to run in order to converge towards attractive and well behaved local minimum.
(ii) When initialiazing the algorithm using kmeans methods, we show that our stopping criterion is a decreasing function of time meaning that the algorithm converges directly towards the desired solution.
Conclusion. We assembled in this work theoretical and practical arguments to support low-rank factorizations for OT. We have presented two controls: one concerning the approximation error the true optimal transport and another concerning the statistical rates of the plug-in estimator: The latter showed be independent of the dimension, which is of particular interest when studying OT in ML settings. We have motivated further the use of LOT as loss by introducing its debiased version and showed that possesses desirable properties: positivity and metrization of the convergence in law. We have also presented the links between the bias induced by such regularization and clustering methods, and studied empirically the effects of hyperparameters involved the practical estimation of LOT. The strong theoretical foundations provided in this paper motivate further studies of the empirical behaviour of LOT estimator, notably on finding suitable local minima and on improvements on the convergence of the MD scheme using other adaptive choices for step sizes.
This work was supported by a "Chaire d' d'excellence de I'IDEX Paris Saclay". The authors would also like to thank Gabriel Peyré and Jaouad Mourtada for enlightening conversa- tions on the topics discussed in this work.
10

---

References
Anastasia Bayandina, Pavel Dvurechensky, Alexander Gasnikov, Fedor Stonyakin, and Alexander Titov. Mirror descent and convex optimization problems with non-smooth inequality constraints. In Large-scale and distributed optimization, pages 181-213. Springer, 2018.
Lenaic Chizat, Pierre Roussillon, Flavien Léger, François-Xann Xavier Vialard. and Gabriel Peyré. Faster wasserstein distance estimation with the sinkhorn divergence. Advances Neural Information Processing Systems, 33, 2020.
Christian Clason, Dirk A. Lorenz, Hinrich Mahler, and Benedikt Wirth. Entropic regularization of continuous optimal transport problems. Journal of Mathematical Analysis and Applications, 494 (1):124432, 2021. ISSN 0022-247X. doi: https://doi.org//10. 101 j/a.////.aaa0000 2020. 124432.
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in neural information processing systems, pages 2292-23000 2013.
Marco Cuturi, Laetitia Meng-Papaxantie Yingtao Tian, Charlotte Bunne, Geoff Davis, and Olivier Teboul. Optimal transport tools (ott): A jax toolbox for all things wasserstein. arXiv preprint arXiv:2201.12324, 2022.
Pinar Demetci, Rebecca Santorella, Björn Sandstede, William Stafford Noble, and Ritambhara Singh. Gromov-wassersten optimal transport to align single-cell multi-omics data. BioRxiv, 2020.
Ryan D'Orazio, Nicolas Loizou, Issam Laradji, and Ioannis Mitliagkas. Stochastic mirror descent: Convergence analysis and adaptive variants via the mirror stochastic polyak stepsize. arXiv preprint 10.1522 2021.
Richard Mansfield Dudley. The speed of mean glivenko-cantelli convergence. The Annals of Mathematical Statistics, 40(1):40-50, 1969.
Richard L Dykstra. An algorithm for restricted least squares regression. Journal of the American Statistical Association, 78(384):837-843 1983.
Jean Feydy, Thibault Séjourné, François-Xave Vialard, Shun-Ichi Amari, Alain Trouvé, and Gabriel Peyré. Interpolating between optimal transport and mmd using sinkhorn divergences. arXiv preprint 110000/80000278: 2018.
Aden Forrow, Jan-Christian Hütter, Mor Nitzan, Philippe Rigollet, Geoffrey Schiebinger, and Jonathan Weed. Statistical optimal transport via factored couplings. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 2454 2465. PMLR, 2019.
Nicolas Fournier and Arnaud Guillin. On the rate of convergence in Wasserstein distance of the empirical measure. Probability Theory and Related Fields, 162(3-4):707-738, 2015.
Aude Genevay, Lénaic Chizat, Francis Bach, Marco Cuturi, and Gabriel Peyré. Sample complexity of sinkhorn divergences. arXiv preprint arXiv: 1810.0273 2018a.
Aude Genevay, Gabriel Peyre, and Marco Cuturi. Learning generative models with sinkhorn diver- gences. In Proceedings of the Twenty-Fing International Conference Artificial Intelligence and Statistics, Proceedings of Machine Learning Research. PMLR, 09-11 Apr 2018b
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(Mar):723-773 2012.
Matthicu Heitz, Nicolas Bonneel, David Coeurjolly, Marco Cuturi, and Gabriel Peyré. Ground metric learning graphs. Journal of Mathematical Imaging and Vision, pages 1-19, 2020.
Hicham Janati, Thomas Bazeille, Bertrand Thirion, Marco Cuturi, and Alexandre Gramfort. Multi- subject meg/ceg source imaging with sparse multi-task regression. Neurolmage, page 116847, 2020.
Leonid Kantorovich. On the transfer of masses (in russian). Doklady Akademii Nauk, 37(2):227-229, 1942.

---

Sunil Koundal, Rena Elkin, Saad Nadeem, Yuechuan Xue, Stefan Constantinou, Simon Sanggaard, Xiaodan Liu, Brittany Monte, Feng Xu, William Van Nostrand, al. Optimal mass transport with lagrangian workflow reveals advective and diffusion driven solute transport in the glymphatic system. Scientific reports, 10(1):1-18, 2020.
Samuel Kutin. Extensions to mediarmid's inequality when differences are bounded with high probability. Dept. Comput. Sci., Univ. Chicago, Chicago, IL, USA, Tech. Rep. TR-2002-04, 2002.
Weijie Liu, Chao Zhang, Nenggan Zheng, and Hui Qian. Approximating optimal transport via low-rank and sparse factorization. arXiv preprint 1xXiv:2111.06544 2021.
Gonzalo Mena and Jonathan Weed. Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem. Advances in Neural Information Processing Systems, 32, 2019.
Fabian Pedregosa, Gaël Varoquaux. Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathicu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 1:2825-28300 2011.
Filippo Santambrogio. Optimal transport for applied mathematicians. Birkäuser, NY, 55(58-63):94, 2015.
Meyer Scetbon and Marco Cuturi. Linear time sinkhorn divergences using positive features. In Advances in Neural Information Processing Systems, 2020.
Meyer Scetbon, Marco Cuturi, and Gabriel Peyré. Low-rank sinkhorn factorization, 2021.
Meyer Scethon, Gabriel Peyré, and Marco Cuturi. Linear-time gromov wasserstein distances using low rank couplings and costs. ICMI., 2022.
Geoffrey Schiebinger, Jian Shu, Marcin Tabaka, Brian Cleary, Vidya Subramanian, Aryeh Solomon, Joshua Gould, Siyan Liu, Stacie Lin, Peter Berube, al. Optimal-transpor gene expression identifies developmental trajectories in reprogramming. Cell, 176(4):928-)) 2019.
transport analysis of single-cell
Morgan Schmitz, Matthieu Heitz, Nicolas Bonneel, Fred Ngole, David Coeurjolly, Marco Cuturi, Gabriel Peyré, and Jean-Luc Starck. Wasserstein dictionary learning: Optimal transport-based unsupervised nonlinear dictionary learning. SIAM Journal on Imaging Sciences, 11()):643-77, 2018.
Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, and Kenji Fukumizu. Equivalence of distance-basee and based statistics hypothesis testing. The Annals of Statistics, pages 2263 1291, 2013.
Karren Dai Yang, Karthik Damodaran, Saradha Venkatachalapathy, Ali Soylemezoglu, GV Shiv- ashankar, and Caroline Uhler. Predicting cell lineages using autoencoders and optimal transport. PLoS compulational biology, 16(4):e 1007888, 2020.
Xinye Zheng, Jianbo Ye, James Z Wang, and Jia Li. Scott: Shape-location combined tracking with optimal transport. SIAM Journal on Mathematics of Data Science, 2(2):284 308, 2020.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes] See Sections 3
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
12

---

(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Sec-
tions 5
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 7
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Section
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section y
4. If you are using existing assets (c.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals. if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "iy1NtmBpI-", "reviewer": "Reviewer_hFtE", "summary": "This paper provides an enormous amount of theoretical analysis about low-rank OT: the convergence rate of low-rank OT to the true OT wrt rank parameter, sample complexity for estimating LOT; introduce debiased version of LOT which metrizes the weak convergence; bridge LOT with clustering methods. Practically, they propose a novel initialization to avoid the bad local minima.", "questions": "Question:\n\nFigure 2, which stage is the middle two plots of Figure 2 in gradient flow? They seem to be in a very late stage of convergence, but a weird phenomenon is that the gradient flows of both DLOT and LOT exceed the target distribution firstly, and then come back. Especially when looking at those green arrows, they firstly point outside the moon, then point inside the moon. I think if you solve gradient flow correctly, it will not have this \"exceed first and then pull back\" process.\n\nTypo: \n\nrow 142 delete \"in\"\n\nall the equation (6) is referenced as (16)", "limitations": ".", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThis paper provides the rigorous theoretical analysis of low-rank OT: \n\n- the convergence rate of low-rank OT to the true OT wrt rank parameter, \n\n- sample complexity for estimating LOT is dimensional independent; \n\n- introduce debiased version of LOT which metrizes the weak convergence;\n\n- LOT$(\\mu,\\mu)$ can be seen as a generalization of the k-means method\n\n- Practically, they also propose a novel initialization to avoid the bad local minima.\n\nWeakness:\n\n- The example in Figure 2 is too simple. Swiss roll or two moons in figure 3.2 is more convincing.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "NNhoScgngon", "reviewer": "Reviewer_W6vC", "summary": "This work advances the theory of low-rank factorizations for OT by studying the approximation error as a function of rank and the sample complexity of LOT. It additionally proposes the debiased formulation, DLOT, which is shown to interpolate between MMD and OT, and that it metrizes weak convergence. Additional connection to clustering is drawn and better practices of using adaptive stepsizes and better initializations are suggested. Experiments are done to support the claims in 2D synthetic examples as well as on the Newsgroup20 dataset.\n", "questions": "Please refer to my questions in the \"Strengths And Weaknesses\" section.", "limitations": "The author has addressed the limitations and future directions to take to advance LOT. Societal impact is not discussed but I don't think it's needed.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper is well-written and easy to follow. Although the contributions are a bit all over the place regarding LOT, they are clearly stated and adequately justified.\n\nOn the theory side, the paper provides comprehensive bounds for approximation error and sample complexity while improving the bounds from previous results (e.g. Liu et al. 2021). Then the debiased formulation of LOT is shown to exhibit desirable properties similar to Sinkhorn divergence. The connection to clustering is interesting since it is specific to the low-rank approximation, something that full-rank versions cannot do. While I do not find any of the results surprising or groundbreaking, they are solid and much needed for future research on LOT.\n\nThe experiments section is short but verifies part of the theory. There are some missing experiments justifying the adaptive choice of $\\gamma_k$ versus without adaptation. Parts of the figures and captions could be improved --- see detailed comments below.\n\nA central question I have regarding the practicality of LOT: Is the computational benefit of LOT worth the introduction of nonconvexity and spurious local minima? I would hope to see more experiments (at least empirically) on demonstrating the benefits gained by low-rank approximation and advice on which $r$ to choose. It seems to me that LOT is only efficient when the ground cost matrix admits a low-rank factorization. In what applications is such condition met?\n\nComments:\n- Line 99: \"obtain next a control the approximation\", missing \"of\"?\n- Line 126: sample complexity shows promises since it does not depend on dimension - but wouldn't $||c||_\\infty$ in Proposition 4 depend in the sense that in many applications the diameter of $\\mathcal{X}$ could increase exponentially in $d$? Also as discussed in the paragraph below Proposition 4, $K_r$ could go to infinity.\n- Line 187, Line 192: (15) does not exist. Do you mean (6)?\n- Line 238: \"we do not have access to the this\", no \"the\" here\n- Line 252: (15) should be (6)?\n- Figure 1: which $r$ is used for the upper bound curve?\n- Figure 4: what is the x-axis \"operations\"? Why do some curves not start at 0 on the x-axis?\n- Figure 4: what is the takeaway message from the right figure?\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Jxho_pci88", "reviewer": "Reviewer_Tiv3", "summary": "The optimal transport (OT) is becoming more and more prominent in machine learning field, however, traditional algorithm such as the linear program has a slow computational speed. In the last decade the entropy-regularized OT (EOT) was proposed and the speed has been  improved a lot. This work studies the low-rank OT (LOT) which is an algorithm proposed by Scetbon $\\textit{et al.}$ [2021] that has a promising linear time complexity by searching for the low-cost couplings with low-nonnnegative ranks. The rate of convergence and an dimension independent upper bound of the sample complexity are provided. Furthermore, a debiased version of LOT (DLOT) is proposed, ad the debiasing terms connect LOT to clustering methods. To improve the computation performance adaptive step size and better initializations are introduced, and the effectivenesses are empirically verified by experiments.", "questions": "1. The adaptive step size improves the convergence. The authors also suggest clipping the step size in the range of [1, 10] for most use cases. Could an explanation or evidence be provided to support this choice?\n\n2. In Fig. 1, the notations are confusing. Could the authors choose the variable names more carefully so they are consistent with previous sections? Eg. $n$ is number of samples here and in a few places, but somewhere else $n$ represents dimension. Also, is the dimension $d$ the same as stated in line 203 or is it the dimension of the marginal? \n\n3. In Fig. 1, it seems that the DLOT values of larger $r$ are higher among all $d$ cases. Is there an intuition or explanation?\n\n4. Several references of equations are mislabeled. Eg. in line 187, 192, and 252, in the main text the reference equations are (15) while in the supplements (16). Also, there is not eq. (15) in the main text. Please fix and make them consistent.\n\nMinor errors:  \n[1] Line 141-142: \"...one obtained in Proposition in 4...\", please remove a redundant \"in\".  \n[2] Line 244: A typo: \"...we show the iterates obtained by a gradient descent...\", should iterates be \"iterations\"?\n[3] Eq. (6): \"$Diag$\" should be \"diag\" instead.", "limitations": "There is no negative societal impact. The authors address the limitations.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThis paper extends the LOT work by Scetbon $\\textit{et al.}$ [2021] and studies the theoretical and practical properties of LOT deeply in several aspects. Such complete investigation of an algorithm is essential for bringing in a member into the computational OT family. This work also proposes interesting ideas such as linking the low-rank transport bias to the clustering method, which may inspire other applications and benefit the machine learning community.\n\nWeaknesses:\nThe naming for the variables and equations referencing labels are confusing, as a result, sometimes it is hard to follow. The clarity of this paper could be improved and the notations could be used in a more consistent way so the readers can understand the meanings of the plots and equations with less efforts. See questions below for more details.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "5NeA860eDt", "reviewer": "Reviewer_PXvB", "summary": "This papers is concerned with a model that approximates optimal transport (OT) using the low-rank coupling/matrices. This model in itself has been proposed a couple of years ago and this paper aims at answering important theoretical questions such as approximation error with respect to standard OT and statistical rates of estimation. They introduce a \"debiased\" version of their estimation, similar to the one proposed for entropic OT and make the link with clustering methods. Their theoretical work is also complemented with additional tricks for improving the numerical efficiency of the method.\n", "questions": "My main concern is about the results for statistical estimation. \n\nPropositions 4 and 5 is about a one-sided inequality, i.e. there is no absolute value in the left-hand side of Equation in Prop. 4, neither in Equation in Prop. 5. Note that a complete result on statistical estimation is really about both lower and upper bounds. However, the authors only give an upper bound, as written line 123.\nAlthough I think it is true, I do not see how to get a lower bound. It is likely I may have missed a result in the paper showing that it is a simple consequence, and it was maybe obvious for the authors, and it would be meaningful to include it.\n\nCan the authors clarify their result? \n\nAs a side remark, it is rather borderline practice that the authors pretend to have a complete result on statistical complexity. \nIndeed they write after proposition 4: \"This result shows that the estimation of LOTr,c is independent of the dimension and can be performed on general compact metric spaces.\"\nHowever the result in its current form is only partial and thus one cannot claim anything on statistical estimation. Indeed, the fluctuations of the opposite quantity may be much larger and dependent on the dimension. This can happen in practice. \n\nSo what am I misunderstanding here?\n\nOthers: \n- Is it possible to add the following result. If $\\mu_n \\to \\mu$ for the weak-* topology then $LOT(\\mu_n) \\to LOT(\\mu)$? \n- proof of proposition 1: the decomposition of pi line 423 in supplementary (btw, there is a typo there) should be explained a bit more. Is it a standard SVD?\n- typos in the supplementary material can be corrected, line 480, line 494.\n", "limitations": "no particular comments.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Although one can argue about the usefulness of the model studied by the authors, it is a good paper by its many theoretical contributions exploring the foundations of low-rank approximation of OT. Results are ranging from obvious, easy to non-trivial and the paper will be a reference for other research developments around this model.\nStrengths:\n- Paper is well written.\n- Several meaningful theoretical results.\n- So far I checked, the proofs are correct (I did not check proof of proposition 5).\n\nWeakness: \n- See my question below: Authors must address my question below, if not I'll revise my rating accordingly.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept