Title: Computational Guarantees for Doubly Entropic Wasserstein Barycenters
Venue: NeurIPS 2023
Accepted: yes
Keywords: Wasserstein barycenters

entropic penalization

optimal transport

Sinkhorn's algorithm
Abstract: We study the computation of doubly regularized Wasserstein barycenters, a recently introduced family of entropic barycenters governed by inner and outer regularization strengths. Previous research has demonstrated that various regularization parameter choices unify several notions of entropy-penalized barycenters while also revealing new ones, including a special case of debiased barycenters.  In this paper, we propose and analyze an algorithm for computing doubly regularized Wasserstein barycenters. Our procedure builds on damped Sinkhorn iterations followed by exact maximization/minimization steps and guarantees convergence for any choice of regularization parameters. An inexact variant of our algorithm, implementable using approximate Monte Carlo sampling, offers the first non-asymptotic convergence guarantees for approximating Wasserstein barycenters between discrete point clouds in the free-support/grid-free setting.

Computational Guarantees for Doubly Entropic Wasserstein Barycenters via Damped Sinkhorn Iterations

Abstract

We study the computation of doubly regularized Wasserstein barycenters, a recently introduced family of entropic barycenters governed by inner and outer regularization strengths. Previous research has demonstrated that various regularization parameter choices unify several notions of entropy-penalized barycenters while also revealing new ones, including a special case of debiased barycenters. In this paper, we propose and analyze an algorithm for computing doubly regularized Wasserstein barycenters. Our procedure builds on damped Sinkhorn iterations followed by exact maximization/minimization steps and guarantees convergence for any choice of regularization parameters. An inexact variant of our algorithm, implementable using approximate Monte Carlo sampling, offers the first nonasymptotic convergence guarantees for approximating Wasserstein barycenters between discrete point clouds in the free-support/grid-free setting.

1 Introduction

The Wasserstein distance between two probability distributions measures the least amount of effort needed to reconfigure one measure into the other. Unlike other notions of distances based solely on the numerical values taken by the distribution functions (e.g., the Kullback-Leibler divergence), the Wasserstein distance incorporates an additional layer of complexity by considering pairwise distances between distinct points, measured by some predetermined cost function. As a result, the Wasserstein distances can be seen to lift the geometry of the underlying space where the probability measures are defined to the space of the probability measures itself. This allows for a more thorough and geometrically nuanced understanding of the relationships between different probability measures, which proved to be a versatile tool of increasing importance in a broad spectrum of areas.

Given a collection of probability measures and an associated set of positive weights that sum to one, the corresponding Wasserstein barycenter minimizes the weighted sum of Wasserstein distances to the given measures. In the special case of two measures and the squared Euclidean cost function, Wasserstein barycenters concide with the notion of McCann's displacement interpolation introduced in the seminal paper [42]. The general case, encompassing an arbitrary number of measures, was first studied by Agueh and Carlier [1], where they also demonstrated a close link between Wasserstein barycenters and the multi-marginal optimal transport problem [29]. Recent years have witnessed an increasing number of applications of Wasserstein barycenters across various scientific disciplines. See, for instance, the following sample of works in economics [16, 11], statistics [8], image processing [50], and machine learning [19], among other areas. For further background and references we point the interested reader to the introductory surveys [47, 45] and the textbooks [58, 59, 54, 46, 28].

Despite their compelling theoretical characteristics, the computation of Wasserstein barycenters poses significant computational challenges, particularly in large-scale applications. While Wasserstein barycenters can be computed in polynomial time for fixed dimensions [3], the approximation of Wasserstein barycenters is known to be NP-hard [4]. Currently employed methods for approximating Wasserstein barycenters are predominantly based on space discretizations. Unfortunately, such strategies are only computationally practical for problems of relatively modest scale. Although there are a handful of grid-free techniques available for approximating Wasserstein barycenters (e.g., [18, 35, 23, 40]), we are not aware of any existing methods that provide bounds on computational complexity. One contribution of the present paper is to introduce a method that in some regimes can provably approximate Wasserstein barycenters without relying on space discretizations, but instead employing approximate Monte Carlo sampling.

More broadly, the difficulties associated with computation of the optimal transport cost has prompted the exploration of computationally efficient alternatives, leading to the consideration of regularized Wasserstein distances. Among these, the entropic penalty has emerged as one of the most successful in applications. The practical success of entropic penalization can be attributed to Sinkhorn's algorithm [56], which enables efficient and highly parallelizable computation, an algorithm that gained substantial traction in the machine learning community following the work of Cuturi [20]. It is worth noting that entropic Wasserstein distances are of intrinsic interest, beyond their approximation capabilities. Indeed, they hold a rich historical connection to the Schrödinger bridge problem [55, 60, 26], as highlighted in the recent surveys [39, 14]. Furthermore, they increasingly serve as an analytically convenient tool for studying the unregularized optimal transport problem (see, e.g., [38, 31, 27, 15]) and they underlie some favorable statistical properties that are currently under active investigation; see the works [43, 30, 24, 48, 52, 49] and the references therein.

Let us now define the entropic optimal transport cost. Consider two probability measures, µ and ν , both supported on X , and let c : X × X → [0 , ∞ ) be a cost function. The entropic Wasserstein distance with a regularization level λ > 0 is defined as

where Π( µ, ν ) denotes the set of probability measures on X ×X with marginal distributions equal to µ and ν , and KL( · , · ) is the Kullback-Leibler divergence. When λ → 0 , the regularized cost T λ ( µ, ν ) converges to the unregularized Wasserstein distance. Various properties of entropic optimal transport can be found in the recent lecture notes by Léonard [39].

To develop efficiently computable approximations for Wasserstein barycenters, a natural approach is to replace the unregularized Wasserstein cost with the minimizer of the weighted sum of entropyregularized costs. This method was first explored by Cuturi and Doucet [21] and it has gained additional traction in the recent years. There is some flexibility in the definition of (1), which arises from substituting the reference product measure µ ⊗ ν with alternatives such as the Lebesgue measure. Consequently, various notions of entropic barycenters have emerged in the literature, which can be unified through the following optimization problem:

Here ν 1 , . . . , ν k are the probability measures whose barycenter we wish to compute and w 1 , . . . , w k are positive weights that sum to one. The inner regularization strength is denoted by λ > 0 while τ > 0 is the outer regularization strength. The measure π ref is an arbitrary reference measure, the support of which dictates the support of the computed barycenter. For instance, if we take π ref to be a uniform measure on a particular discretization of the underlying space, we are dealing with a fixedsupport setup. On the other hand, letting π ref be the Lebesgue measure puts us in the free-support setup. We shall refer to the minimizer of (2) as the ( λ, τ ) -barycenter, which exists and is unique due to the strict convexity of the outer regularization penalty; however, uniqueness may no longer holds when τ = 0 .

The objective (2) was recently studied in [17]; it also appeared earlier in [5] for the special case τ ≥ λ , where stochastic approximation algorithms were considered for the computation of fixed-support barycenters. In [17, Section 1.3], it is discussed how various choices of ( λ, τ ) relate to Barycenters previously explored in the literature. To provide a brief overview, (0 , 0) are the unregularized

Wasserstein barycenters studied in [1]. Inner-regularized barycenters ( λ, 0) introduce a shrinking bias; this can be seen already when k = 1 , in which case the solution computes a maximum-likelihood deconvolution [53]. The ( λ, λ ) -barycenters were considered in [21, 7, 22, 10, 36]; they introduce a blurring bias. Likewise, blurring bias is introduced by the outer-regularized barycenters (0 , τ ) , studied in [9, 12]. The only case not covered via the formulation (2) appears to be the one of debiased Sinkhorn barycenters [51, 33], for which an algorithm exists but without computational guarantees. Of particular interest are the ( λ, λ/ 2) barycenters: the choice τ = λ/ 2 for smooth densities yields approximation bias of order λ 2 , while the choice τ = λ results in bias of order λ , which is significantly larger than λ 2 in the regimes of interest. This is a new notion of entropic barycenters that was unveiled in the analysis of [17]. We provide the first convergence guarantees for this type of barycenters.

The regularity, stability, approximation, and statistical sample complexity properties of ( λ, τ ) -barycenters were investigated in [17]. However, the question of obtaining non-asymptotic convergence guarantees for the computation of ( λ, τ ) -barycenters with arbitrary regularization parameters was not addressed therein. In particular, the ( λ, λ/ 2) case, which has stood out due to its compelling mathematical features, has not yet been addressed in the existing literature. This gap is addressed by the present paper; we summarize our contributions in the following section.

1.1 Contributions

The remainder of this paper is organized as follows: Section 2 provides the necessary background on entropic optimal transport and a particular dual problem of the doubly regularized entropic objective (2). Section 3 introduces a damped Sinkhorn iteration scheme and complements it with convergence guarantees. An approximate version of the algorithm together with convergence results and implementation details is discussed in Section 4. We summarize our key contributions:

Lemma 1, presented in Section 3, demonstrates that bounds on the dual suboptimality gap for the dual problem (8), defined in Section 2.2, can be translated into Kullback-Leibler divergence bounds between the ( λ, τ ) -barycenter and the barycenters corresponding to dual-feasible variables. This translation enables us to formulate all our subsequent results in terms of optimizing the dual objective (8).

In Section 3, we introduce a damped Sinkhorn scheme (Algorithm 1) that can be employed to optimize ( λ, τ ) -barycenters for any choice of regularization parameters. The damping factor min(1 , τ /λ ) accommodates the degrading smoothness properties of the dual objective (8) as a function of decreasing outer regularization parameter τ . The introduced damping of the Sinkhorn iterations is, in fact, necessary and it is one of our core contributions: undamped exact scheme can be experimentally shown to diverge as soon as τ < λ/ 2 .

The main result of this paper is Theorem 1 proved in Section 3. It provides convergence guarantees for Algorithm 1 with arbitrary choice of regularization parameters λ, τ > 0 . This, in particular, results in the first algorithm with guarantees for computing ( λ, λ/ 2) barycenters. For smooth densities, these barycenters incur a bias of order λ 2 in contrast to the predominantly studied ( λ, λ ) barycenters that incur bias of order λ .

In Section 4, we describe Algorithm 2, an extension of Algorithm 1 that allows us to perform inaccurate updates. We formulate sufficient conditions on the inexact updates oracle under which the errors in the convergence analysis do not accumulate. Section 4.1 details an implementation of this inexact oracle, based on approximate Monte Carlo sampling.

Theorem 2 proved in Section 4 furnishes convergence guarantees for Algorithm 2. When combined with the implementation of the inexact oracle described in Section 4.1, this yields a provably convergent scheme for a grid-free computation of entropic Wasserstein barycenters between discrete distributions, provided sufficient regularity on the domain X and the cost function c .

Appendix F complements our theoretical results with numerical experiments. Our simulations experimentally confirm the necessity of damping when τ < λ/ 2 . They also provide experimental support for the suggested damping factor in Algorithms 1 and 2.

2 Background and Notation

This section provides the background material on doubly regularized entropic Wasserstein barycenters and introduces the notation used throughout the paper. In the remainder of the paper, let X be a compact and convex subset of R d with a non-empty interior. Let P ( X ) denote the set of probability measures on X endowed with Borel sigma-algebra. Let c : X × X → [0 , ∞ ) be a cost function such that c ∞ ( X ) = sup x,x ′ ∈X c ( x, x ′ ) < ∞ . We denote by KL( · , · ) the Kullback-Leibler divergence, ‖ · ‖ TV is the total-variation norm, and ‖ f ‖ osc = sup x f ( x ) -inf x ′ f ( x ′ ) is the oscillation norm. Given two measures ν, ν ′ , the notation ν glyph[lessmuch] ν ′ denotes that ν is absolutely continuous with respect to the measure ν ′ ; in this case dν/dν ′ denotes the Radon-Nikodym derivative of ν with respect to ν ′ . Finally, throughout the paper w denotes a vector of k strictly positive elements that sum to one.

2.1 Entropic Optimal Transport

For any µ, ν ∈ P ( X ) define the entropy regularized optimal transport problem by

where KL is the Kullback-Leibler divergence and Π( µ, ν ) ⊆ P ( X ⊗ X ) is the set of probability measures such that for any γ ∈ Π( µ, ν ) and any Borel subset A of X it holds that γ ( A ×X ) = µ ( A ) and γ ( X × A ) = ν ( A ) .

Let E µ,ν λ : L 1 ( µ ) × L 1 ( ν ) → R be the function defined by

The entropic optimal transport problem (3) admits the following dual representation:

The solution is unique µ -almost everywhere up to a constant; we fix a particular choice

Likewise, we denote ψ φ = argmax ψ ∈ L 1 ( ν ) E µ,ν λ ( φ, ψ ) with the analogous expression to the one given above, interchanging the roles of φ and ψ . Then, the maximum in (4) is attained by any pair ( φ ∗ , ψ ∗ ) such that φ ∗ = φ ψ ∗ and ψ ∗ = ψ φ ∗ ; such a pair is said to solve the Schrödinger system and it is unique up to translations ( φ ∗ + a, ψ ∗ -a ) by any constant a ∈ R . The optimal coupling that solves the primal problem (3) can be obtained from the pair ( φ ∗ , ψ ∗ ) via the primal-dual relation

We conclude this section by listing two properties of functions of the form φ ψ . These properties will be used repeatedly throughout this paper. First, for any ψ we have

which means, in particular, that for any ψ we have

For any ψ define

The second property of interest is that for any ψ and any x, x ′ ∈ X it holds that

In particular, for any ψ we have

2.2 Doubly Regularized Entropic Barycenters

Let ν = ( ν 1 , . . . , ν k ) ∈ P ( X ) k be k probability measures and let w ∈ R k be a vector of positive numbers that sum to one. Given the inner regularization strength λ > 0 and the outer regularization strength τ > 0 , the ( λ, τ ) barycenter µ λ,τ ∈ P ( X ) of probability measures ν with respect to the weights vector w is defined as the unique solution to the following optimization problem:

where π ref ∈ P ( X ) is a reference probability measure.

We will now describe how to obtain a concave dual maximization problem to the primal problem (7), following along the lines of Chizat [17, Section 2.3], where the interested reader will find a comprehensive justification of all the claims made in the rest of this section.

First, using the semi-dual formulation of entropic optimal transport problem (5), we have, for each j ∈ { 1 , . . . , k }

Denote ψ = ( ψ 1 , . . . , ψ j ) ∈ L 1 ( ν ) . Then, we may rewrite the primal problem (7) by

Interchanging min and max , which is justified using compactness of X as detailed in [17], we obtain the dual optimization objective E ν ,w λ,τ : L 1 ( ν ) → R defined by

The infimum above is attained by the measure

To each dual variable ψ we associate the marginal measures ν j ψ ( dy ) defined for j = 1 , . . . , k by

Finally, we mention that the objective E ν ,w λ,τ is concave and for any ψ , ψ ′ it holds that

In particular, fixing any optimal dual variable ψ ∗ , for any ψ it holds using concavity of E ν ,w λ,τ that

This concludes our overview of the background material on ( λ, τ ) -barycenters.

3 Damped Sinkhorn Scheme

This section introduces a damped Sinkhorn-based optimization scheme (Algorithm 1) and provides guarantees for its convergence (Theorem 1). Before describing the algorithm, we make a quick detour to the following lemma, proved in Appendix A, which shows that the sub-optimality gap bounds on the dual objective (8) can be transformed into corresponding bounds on relative entropy between the ( λ, τ ) -barycenter and the barycenter associated to a given dual variable.

Lemma 1. Fix any λ, τ > 0 and ν , w . Let ψ ∗ be the maximizer of dual problem E ν ,w λ,τ and let µ ψ ∗ be the corresponding minimizer of the primal objective (7) . Then, for any ψ ∈ L 1 ( ν ) we have

We now turn to describing an iterative scheme that ensures convergence of the dual suboptimality gap to zero. Let ψ t be an iterate at time t . Then, we have

where

In particular, when optimizing the dual objective E ν,w λ,τ , every time the variable ψ t is updated, it automatically triggers the exact maximization/minimization steps defined in (11). It is thus a natural strategy to fix φ t and µ t and perform exact minimization on ψ , which can be done in closed form:

where ν j t denotes the marginal distribution ν j ψ t defined in (9). The update (12) performs a Sinkhorn update on each block of variables ψ j . Together, the update (12) followed by (11) results in the iterative Bregman projections algorithm introduced in [7]. In [36], it was shown that this scheme converges for the ( λ, λ ) -barycenters. The analysis of [36] is built upon a different dual formulation from the one considered in our work; this alternative formulation is only available when τ = λ [17, Section 2.3] and thus excludes the consideration of debiased barycenters ( λ, λ/ 2) .

We have observed empirically (see Appendix F) that the iterates of the iterative Bregman projections (i.e., the scheme of updates defined in (12) and (11)) diverge whenever τ < λ/ 2 . Indeed, decreasing the outer regularization parameter τ makes the minimization step in (11) less stable. As a result, the cumulative effect of performing the updates (12) and (11) may result in a decrease in the value of the optimization objective E ν ,w λ,τ .

One of the main contributions of our work is to show that this bad behaviour can be mitigated by damping the exact Sinkhorn updates (12). This leads to Algorithm 1 for which convergence guarantees are provided in Theorem 1 stated below.

Algorithm 1: Exact Damped Sinkhorn Scheme

Input: regularization strengths λ, τ > 0 , reference measure π ref ,number of iterations T and k marginal measures ν 1 , . . . , ν k with positive weights w 1 , . . . , w k such that ∑ k j =1 w j = 1 .

Set η = min(1 , τ /λ ) and initialize ( ψ j 0 ) = 0 for j ∈ { 1 , . . . , k } .

(a) φ j t ( x ) ←-λ log ∫ X exp(( ψ j t ( y ) -c ( x, y )) /λ ) ν j ( dy ) for j ∈ { 1 , . . . , k }

For t = 0 , 1 . . . , T -1 do

(b) V t ( x ) ← ∑ k j =1 w j φ j ( t )( x )

(d) µ t ( dx ) ← Z -1 t exp( -V t ( x ) /τ ) π ref ( dx )

(c) Z t ← ∫ exp( -V t ( x ) /τ ) dπ ref ( dx )

(e) dν j t dν j ( y ) ← ∫ exp ( φ j t ( x )+ ψ j t ( y ) -c ( x,y ) λ ) µ t ( dx ) for j ∈ { 1 , . . . , k } .

Return ( φ j T , ψ j T ) k j =1 .

(f) ψ j t +1 ( y ) ← ψ j t ( y ) -ηλ log dν j t dν j ( y ) for j ∈ { 1 , . . . , k } .

Theorem 1. Fix any λ, τ > 0 and ν , w . Let ψ ∗ be the maximizer of dual problem E ν ,w λ,τ . Let ( ψ t ) t ≥ 0 be the sequence of iterates generated by Algorithm 1. Then, for any t ≥ 1 it holds that

Our convergence analysis draws upon the existing analyses of Sinkhorn's algorithm [2, 25], which in turn are based on standard proof strategies in smooth convex optimization (e.g., [44, Theorem 2.1.14]). Concerning the proof of Theorem 1, the main technical contribution of our work lies in the following proposition proved in Appendix B.

Proposition 1. Consider the setup of Theorem 1. Then, for any integer t ≥ 0 it holds that

With Proposition 1 at hand, we are ready to prove Theorem 1.

Proof of Theorem 1. Denote δ t = E ν ,w λ,τ ( ψ ∗ ) -E ν ,w λ,τ ( ψ t ) . We would like to relate the suboptimality gap δ t to the increment δ t -δ t +1 . To do this, we will first show that the iterates ψ t have their oscillation norm bounded uniformly in t . Indeed, for any j ∈ { 1 , . . . , k } , any t ≥ 1 , and any y ∈ X we have

By (6), ψ φ j t has oscillation norm bounded by c ∞ ( X ) . Because ψ j 0 = 0 and η ∈ (0 , 1] , by induction on t it follows that ‖ ψ t ‖ osc ≤ c ∞ ( X ) for any t ≥ 0 . Combining the bound on the dual sub-optimality gap (10) with Pinsker's inequality yields

Using concavity of the square root function, Proposition 1 yields for any t ≥ 0

By Proposition 1, the sequence δ t is non-increasing. Hence, dividing the above equality by δ t δ t +1 yields

Telescoping the left hand side completes the proof.

4 Approximate Damped Sinkhorn Scheme

In this section, we extend the analysis of Algorithm 1 to an approximate version of the algorithm. Then, in Section 4.1, we describe how inexact updates may be implemented via approximate random sampling, thus enabling the computation of ( λ, τ ) -barycenters in the free-support setting with convergence guarantees.

Algorithm 2 describes an inexact version of Algorithm 1. It replaces the damped Sinkhorn iterations of Algorithm 1 via approximate updates computed by an approximate Sinkhorn oracle - a procedure that satisfies the properties listed in Definition 1.

Definition 1 (Approximate Sinkhorn Oracle) . An ε -approximate Sinkhorn oracle is a procedure that given any ψ and any index j ∈ { 1 , . . . , k } , returns a Radon-Nikodym derivative d ˜ ν j ψ dν j of a measure ˜ ν j ψ glyph[lessmuch] ν j that satisfies the following properties:

d ˜ ν j ψ dν j is strictly positive on the support of ν j ;

‖ ˜ ν j ψ -ν j ψ ‖ TV ≤ ε/ (2 c ∞ ( X )) ;

E Y ∼ ν j [ dν j ψ d ˜ ν j ψ ( Y )] ≤ 1 + ε 2 / (2 c ∞ ( X ) 2 ) ;

For any η ∈ [0 , 1] and any j ∈ { 1 , . . . , k } it holds that ‖ ψ j + ηλ log( d ˜ ν j ψ /dν j ) ‖ osc ≤ (1 -η ) ‖ ψ j ‖ osc + ηc ∞ ( X ) .

Algorithm 2: Approximate Damped Sinkhorn Scheme

Input: error tolerance parameter ε > 0 , a function 'ApproximateSinkhornOracle' satisfying properties listed in Definition 1, regularization strengths λ, τ > 0 , reference measure π ref ,number of iterations T and k marginal measures ν 1 , . . . , ν k with positive weights w 1 , . . . , w k such that ∑ k j =1 w j = 1 .

Set η = min(1 , τ /λ ) and initialize ( ψ j 0 ) = 0 for j ∈ { 1 , . . . , k } .

(a) d ˜ ν j t dν j ( y ) ← ApproximateSinkhornOracle( ν , λ, τ, ψ t , ε, j ) for j ∈ { 1 , . . . , k } .

For t = 0 , 1 . . . , T -1 do

(b) ψ j t +1 ( y ) ← ψ j t ( y ) -ηλ log d ˜ ν j t dν j ( y ) for j ∈ { 1 , . . . , k } .

Return ( φ j T , ψ j T ) k j =1 .

The following theorem shows that Algorithm 2 enjoys the same convergence guarantees as Algorithm 1 up to the error tolerance of the procedure used to implement the approximate updates. A noteworthy aspect of the below theorem is that the error does not accumulate over the iterations.

Theorem 2. Fix any λ, τ > 0 and ν , w . Let ψ ∗ be the maximizer of dual problem E ν ,w λ,τ . Let ( ˜ ψ t ) t ≥ 0 be the sequence of iterates generated by Algorithm 2 with the accuracy parameter ε ≥ 0 . Let T = min { t : E ν ,w λ,τ ( ψ ∗ ) -E ν ,w λ,τ ( ˜ ψ t ) ≤ 2 ε } . Then, for any t ≤ T it holds that

The proof of the above theorem can be found in Appendix C.

4.1 Implementing the Approximate Sinkhorn Oracle

In this section, we show that the approximate Sinkhorn oracle (see Definition 1) can be implemented using approximate random sampling when the marginal distributions ν j are discrete. To this end, fix

the regularization parameters λ, τ > 0 , the weight vector w , and consider a set of k discrete marginal distributions

where δ x is the Dirac measure located at x and ν j ( y j l ) is equal to the probability of sampling the point y j l from measure ν j . We denote the total cardinality of the support of all measures ν j by

Fix any ψ ∈ L 1 ( ν ) . Suppose we are given access to n i.i.d. samples X 1 , . . . , X n from a probability measure µ ′ ψ that satisfies

Then, for j = 1 , . . . , k and l = 1 , . . . , m j consider

and for any parameter ζ ∈ (0 , 1 / 2] define

We claim that ˜ ν j implements the approximate Sinkhorn oracle with accuracy parameter of order O ( ε 1 / 4 µ ) provided that n is large enough. This is shown in the following lemma, the proof of which can be found in Appendix D.

Lemma 2. Fix any δ ∈ (0 , 1) and consider the setup described above. With probability at least 1 -δ , for each j ∈ { 1 , . . . , k } it holds simultaneously that the measure ˜ ν j defined in (13) satisfies all the properties listed in Definition 1 with accuracy parameter

The above lemma shows that a step of Algorithm 2 can be implemented provided access to i.i.d. sampling from some measure µ ′ ψ close to µ ψ in total variation norm, where ψ is an arbitrary iterate of Algorithm 2. The remainder of this section is dedicated to showing that this can be achieved by sampling via Langevin Monte Carlo.

Henceforth, fix π ref to be the Lebesgue measure on X , which corresponds to the free-support barycenters setup. Then, for any ψ we have

where 1 X is equal to one on X and zero everywhere else. It follows by (6) that ‖ V ψ ‖ osc ≤ c ∞ ( X ) /τ . Further, let diam X = sup x,x ′ ∈X ‖ x -x ′ ‖ 2 . By the convexity of X , the uniform measure on X satisfies the logarithmic Sobolev inequality (LSI) with constant diam( X ) 2 / 4 (cf. [37]). Hence, by the Holley-Stroock perturbation argument [32], the measure µ ψ satisfies LSI with constant at most exp(2 c ∞ ( X ) /τ ) diam( X ) 2 / 4 < ∞ .

It is well-established that Langevin Monte Carlo algorithms offer convergence guarantees for approximate sampling from a target measure subject to functional inequality constraints provided additional conditions hold such as the smoothness of the function V ψ . However, such guarantees do not directly apply to the measure µ ψ due to its constrained support. Instead, it is possible to approximate µ ψ arbitrarily well in total variation norm by a family of measures ( µ ψ ,σ ) σ> 0 (see Appendix E for details) supported on all of R d . Tuning the parameter σ allows us to trade-off between the approximation quality of µ ψ ,σ and its LSI constant. Crucially, standard sampling guarantees for Langevin Monte Carlo (e.g., [57]) apply to the regularized measures µ ψ ,σ , which leads to provable

guarantees for an implementation of Algorithm 2, thus furnishing the first convergence guarantees for computation of Wasserstein barycenters in the free support setup; see Theorem 3 stated below.

The above approximation argument applies to any cost function c that is Lipschitz on X and exhibits quadratic growth at infinity. For the sake of simplicity, we consider the quadratic cost c ( x, y ) = ‖ x -y ‖ 2 2 . The exact problem setup where we are able to obtain computational guarantees for free-support barycenter computation via Langevin Sampling is formalized below.

Problem Setting 1. Consider the setting described at the beginning of Section 4.1. In addition, suppose that

the reference measure π ref ( dx ) = 1 X dx is the Lebesgue measure supported on X (freesupport setup);

it holds that X ⊆ B R = { x ∈ R d : ‖ x ‖ 2 ≤ R } for some constant R < ∞ ;

the cost function c : R d × R d → [0 , ∞ ) is defined by c ( x, y ) = ‖ x -y ‖ 2 2 ;

for any ψ we have access to a stationary point x ψ of V ψ over X .

The last condition can be implemented in polynomial time using a first order gradient method. For our purposes, this condition is needed to obtain a good initialization point for the Unadjusted Langevin Algorithm following the explanation in [57, Lemma 1]; see Appendix E for further details.

We now proceed to the main result of this section, the proof of which can be found in Appendix E. The following theorem provides the first provably convergent method for computing Wasserstein barycenters in the free-support setting. We remark that a stochastic approximation argument of a rather different flavor used to compute fixed-support Wasserstein barycenters (for τ ≥ λ ) has been previously analyzed in [5].

Theorem 3. Consider the setup described in Problem Setting 1. Then, for any confidence parameter δ ∈ (0 , 1) and any accuracy parameter ε > 0 , we can simulate a step of Algorithm 2 with success probability at least 1 -δ in time polynomial in

In particular, an ε -approximation of the ( λ, τ ) -Barycenter can be obtained within the same computational complexity.

Comparing the above guarantee with the discussion following the statement of Lemma 2, we see an additional polynomial dependence on ( Rd -1 / 4 ) d (note that for R ≤ d 1 / 4 this term disappears). We believe this term to be an artefact of our analysis appearing due to the approximation argument described above. Considering the setup with R ≤ d 1 / 4 , the running time of our algorithm depends exponentially in R 2 /τ .

glyph[negationslash]

We conclude with two observations. First, since approximating Wasserstein barycenters is generally NP-hard [4], an algorithm with polynomial dependence on all problem parameters does not exist if P = NP . Second, notice that computing an ε approximation of ( λ, τ ) -Barycenter can be done in time polynomial in ε -1 . This should be contrasted with numerical schemes based on discretizations of the set X , which would, in general, result in computational complexity of order ( R/ε ) d to reach the same accuracy.

5 Conclusion

Weintroduced algorithms to compute doubly regularized entropic Wasserstein barycenters and studied their computational complexity, both in the fixed-support and in the free-support settings. Although a naive adaptation of the usual alternate maximization scheme from [7] to our setting leads to diverging iterates (at least for small values of τ ), our analysis shows that it is sufficient to damp these iterations to get a converging algorithm.

While we have focused on the problem of barycenters of measures, we note that the idea of entropic regularization is pervasive in other applications of optimal transport. There, the flexibility offered by the double entropic regularization may prove to be useful as well, and we believe that our damped algorithm could be adapted to these more general settings.

References

M. Agueh and G. Carlier. Barycenters in the Wasserstein space. SIAM Journal on Mathematical Analysis , 43(2):904-924, 2011.

J. Altschuler, J. Niles-Weed, and P. Rigollet. Near-linear time approximation algorithms for optimal transport via sinkhorn iteration. Advances in neural information processing systems , 30, 2017.

J. M. Altschuler and E. Boix-Adsera. Wasserstein barycenters can be computed in polynomial time in fixed dimension. The Journal of Machine Learning Research , 22(1):2000-2018, 2021.

J. M. Altschuler and E. Boix-Adsera. Wasserstein barycenters are np-hard to compute. SIAM Journal on Mathematics of Data Science , 4(1):179-203, 2022.

M. Ballu, Q. Berthet, and F. Bach. Stochastic optimization for regularized Wasserstein estimators. In International Conference on Machine Learning , pages 602-612. PMLR, 2020.

H. H. Bauschke and P. L. Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spaces . Springer, 2017.

J.-D. Benamou, G. Carlier, M. Cuturi, L. Nenna, and G. Peyré. Iterative bregman projections for regularized transportation problems. SIAM Journal on Scientific Computing , 37(2):A1111A1138, 2015.

E. Bernton, P. E. Jacob, M. Gerber, and C. P. Robert. On parameter estimation with the Wasserstein distance. Information and Inference: A Journal of the IMA , 8(4):657-676, 2019.

J. Bigot, E. Cazelles, and N. Papadakis. Data-driven regularization of Wasserstein barycenters with an application to multivariate density registration. Information and Inference: A Journal of the IMA , 8(4):719-755, 2019.

J. Bigot, E. Cazelles, and N. Papadakis. Penalization of barycenters in the Wasserstein space. SIAM Journal on Mathematical Analysis , 51(3):2261-2285, 2019.

G. Carlier and I. Ekeland. Matching for teams. Economic theory , pages 397-418, 2010.

G. Carlier, K. Eichinger, and A. Kroshnin. Entropic-Wasserstein barycenters: Pde characterization, regularity, and clt. SIAM Journal on Mathematical Analysis , 53(5):5880-5914, 2021.

P. Cattiaux, A. Guillin, and L.-M. Wu. A note on talagrand's transportation inequality and logarithmic sobolev inequality. Probability theory and related fields , 148:285-304, 2010.

Y. Chen, T. T. Georgiou, and M. Pavon. Stochastic control liaisons: Richard sinkhorn meets gaspard monge on a Schrodinger bridge. Siam Review , 63(2):249-313, 2021.

S. Chewi and A.-A. Pooladian. An entropic generalization of caffarelli's contraction theorem via covariance inequalities. arXiv preprint arXiv:2203.04954 , 2022.

P.-A. Chiappori, R. J. McCann, and L. P. Nesheim. Hedonic price equilibria, stable matching, and optimal transport: equivalence, topology, and uniqueness. Economic Theory , pages 317-354, 2010.

L. Chizat. Doubly regularized entropic Wasserstein barycenters. arXiv preprint arXiv:2303.11844 , 2023.

S. Cohen, M. Arbel, and M. P. Deisenroth. Estimating barycenters of measures in high dimensions. arXiv preprint arXiv:2007.07105 , 2020.

N. Courty, R. Flamary, A. Habrard, and A. Rakotomamonjy. Joint distribution optimal transportation for domain adaptation. Advances in neural information processing systems , 30, 2017.

M. Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems , 26, 2013.

M. Cuturi and A. Doucet. Fast computation of Wasserstein barycenters. In International conference on machine learning , pages 685-693. PMLR, 2014.

M. Cuturi and G. Peyré. Semidual regularized optimal transport. SIAM Review , 60(4):941-965, 2018.

C. Daaloul, T. L. Gouic, J. Liandrat, and M. Tournus. Sampling from the Wasserstein barycenter. arXiv preprint arXiv:2105.01706 , 2021.

E. del Barrio and J.-M. Loubes. The statistical effect of entropic regularization in optimal transportation. arXiv preprint arXiv:2006.05199 , 2020.

P. Dvurechensky, A. Gasnikov, and A. Kroshnin. Computational optimal transport: Complexity by accelerated gradient descent is better than by sinkhorn's algorithm. In International conference on machine learning , pages 1367-1376. PMLR, 2018.

S. Erlander and N. F. Stewart. The gravity model in transportation analysis: theory and extensions , volume 3. Vsp, 1990.

M. Fathi, N. Gozlan, and M. Prod'homme. A proof of the caffarelli contraction theorem via entropic regularization. Calculus of Variations and Partial Differential Equations , 59:1-18, 2020.

A. Figalli and F. Glaudo. An Invitation to Optimal Transport, Wasserstein Distances, and Gradient Flows . European Mathematical Society, 2021.

W. Gangbo and A. ´ Swi˛ ech. Optimal maps for the multidimensional monge-kantorovich problem. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences , 51(1):23-45, 1998.

A. Genevay, L. Chizat, F. Bach, M. Cuturi, and G. Peyré. Sample complexity of sinkhorn divergences. In The 22nd international conference on artificial intelligence and statistics , pages 1574-1583. PMLR, 2019.

I. Gentil, C. Léonard, and L. Ripani. About the analogy between optimal transport and minimal entropy. In Annales de la Faculté des sciences de Toulouse: Mathématiques , volume 26, pages 569-600, 2017.

R. Holley and D. W. Stroock. Logarithmic sobolev inequalities and stochastic ising models, 1986.

H. Janati, M. Cuturi, and A. Gramfort. Debiased sinkhorn barycenters. In International Conference on Machine Learning , pages 4692-4701. PMLR, 2020.

H. Janati, B. Muzellec, G. Peyré, and M. Cuturi. Entropic optimal transport between unbalanced gaussian measures has a closed form. Advances in neural information processing systems , 33: 10468-10479, 2020.

A. Korotin, L. Li, J. Solomon, and E. Burnaev. Continuous Wasserstein-2 barycenter estimation without minimax optimization. arXiv preprint arXiv:2102.01752 , 2021.

A. Kroshnin, N. Tupitsa, D. Dvinskikh, P. Dvurechensky, A. Gasnikov, and C. Uribe. On the complexity of approximating Wasserstein barycenters. In International conference on machine learning , pages 3530-3540. PMLR, 2019.

J. Lehec. The langevin monte carlo algorithm in the non-smooth log-concave case. arXiv preprint arXiv:2101.10695 , 2021.

C. Léonard. From the Schrödinger problem to the monge-kantorovich problem. Journal of Functional Analysis , 262(4):1879-1920, 2012.

C. Léonard. A survey of the Schrödinger problem and some of its connections with optimal transport. arXiv preprint arXiv:1308.0215 , 2013.

J. v. Lindheim. Simple approximative algorithms for free-support Wasserstein barycenters. Computational Optimization and Applications , pages 1-34, 2023.

A. Mallasto, A. Gerolin, and H. Q. Minh. Entropy-regularized 2-wasserstein distance between gaussian measures. Information Geometry , 5(1):289-323, 2022.

R. J. McCann. A convexity principle for interacting gases. Advances in mathematics , 128(1): 153-179, 1997.

G. Mena and J. Niles-Weed. Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem. Advances in Neural Information Processing Systems , 32, 2019.

Y. Nesterov et al. Lectures on convex optimization , volume 137. Springer, 2018.

V. M. Panaretos and Y. Zemel. Statistical aspects of Wasserstein distances. Annual review of statistics and its application , 6:405-431, 2019.

V. M. Panaretos and Y. Zemel. An invitation to statistics in Wasserstein space . Springer Nature, 2020.

G. Peyré, M. Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends® in Machine Learning , 11(5-6):355-607, 2019.

A.-A. Pooladian and J. Niles-Weed. Entropic estimation of optimal transport maps. arXiv preprint arXiv:2109.12004 , 2021.

A.-A. Pooladian, V. Divol, and J. Niles-Weed. Minimax estimation of discontinuous optimal transport maps: The semi-discrete case. arXiv preprint arXiv:2301.11302 , 2023.

J. Rabin, G. Peyré, J. Delon, and M. Bernot. Wasserstein barycenter and its application to texture mixing. In Scale Space and Variational Methods in Computer Vision: Third International Conference, SSVM 2011, Ein-Gedi, Israel, May 29-June 2, 2011, Revised Selected Papers 3 , pages 435-446. Springer, 2012.

A. Ramdas, N. García Trillos, and M. Cuturi. On Wasserstein two-sample testing and related families of nonparametric tests. Entropy , 19(2):47, 2017.

P. Rigollet and A. J. Stromme. On the sample complexity of entropic optimal transport. arXiv preprint arXiv:2206.13472 , 2022.

P. Rigollet and J. Weed. Entropic optimal transport is maximum-likelihood deconvolution. Comptes Rendus Mathematique , 356(11-12):1228-1235, 2018.

F. Santambrogio. Optimal transport for applied mathematicians. Birkäuser , 2015.

E. Schrödinger. Sur la théorie relativiste de l'électron et l'interprétation de la mécanique quantique. In Annales de l'institut Henri Poincaré , volume 2, pages 269-310, 1932.

R. Sinkhorn. Diagonal equivalence to matrices with prescribed row and column sums. The American Mathematical Monthly , 74(4):402-405, 1967.

S. Vempala and A. Wibisono. Rapid convergence of the unadjusted langevin algorithm: Isoperimetry suffices. Advances in neural information processing systems , 32, 2019.

C. Villani. Topics in Optimal Transportation . American Mathematical Society, 2003.

C. Villani et al. Optimal transport: old and new . Springer, 2009.

A. G. Wilson. The use of entropy maximising models, in the theory of trip distribution, mode split and route split. Journal of transport economics and policy , pages 108-126, 1969.

A Proof of Lemma 1

To simplify the notation throughout this proof, for each j ∈ { 1 , . . . , k } denote φ j = φ ψ j . We have

Observe that for any x ∈ X it holds that

Hence,

where in the final expression we have applied the Donsker-Varadhan variational principle (i.e., convexconjugate duality between KL-divergence and cumulant generating functions); therein, the supremum runs over probability measures µ absolutely continuous with respect to µ ψ , and it is attained by µ defined as

That is, the supremum in (15) is attained by µ = µ ψ ∗ . Hence, the identity (14) becomes

E

ν

,w

λ,τ

(

ψ

)

E

ν

,w

λ,τ

(

ψ

)

-

where the final inequality follows by noting that for each j the optimality of the pair (( φ ∗ ) j , ( ψ ∗ ) j ) for the entropic optimal transport dual objective E µ ψ ∗ ,ν j λ implies that

The proof of Lemma 1 is complete.

∗

B Proof of Proposition 1

Recall that for any non-negative integer t we have

where Z t is the normalizing constant defined by

With the notation introduced above, we have

Hence,

Therefore, to prove Proposition 1 it suffices to show that the inequality

holds for any t ≥ 0 . We will complete the proof of Proposition 1 using the following lemma, the proof of which is deferred to the end of this section.

Lemma 3. Let ( ψ t ) t ≥ 0 be any sequence of the form

where for j ∈ { 1 , . . . , k } , (∆ j t ) t ≥ 0 is an arbitrary sequence of strictly positive functions and η = min(1 , τ /λ ) . Then, for any t ≥ 0 it holds that

To complete the proof of Proposition 1, we will apply the above lemma with ∆ j t = dν j dν j t . Indeed, we have

By (16), the proof of Proposition 1 is complete.

B.1 Proof of Lemma 3

We will break down the proof with the help of the following lemma, the proof of which can be found in Section B.2.

Lemma 4. For any sequence ( ψ t ) t ≥ 0 and any t ≥ 0 it holds that where φ t -

Observe that the sequence ( ψ t ) t ≥ 0 of the form stated in Lemma 3 satisfies, for any for any j ∈ { 1 , . . . , k } and any t ≥ 0 , where

Hence, by Lemma 4 we have

We split the remaining proof into two cases: τ ≥ λ and τ < λ .

The case τ ≥ λ . When τ ≥ λ , we have max(1 , λ/τ ) = 1 and η = min(1 , τ /λ ) = 1 . Thus, (17) yields

This completes the proof of Lemma 3 when τ ≥ λ .

The case τ < λ . For j ∈ { 1 , . . . , k } and any x ∈ X define the measure ρ x by

By the definition of φ j t , we have

In particular, ρ x is a probability measure. Hence, (17) can be rewritten as

Because λ/τ > 1 , the function x ↦→ x λ/τ is convex. Applying Jensen's inequality to the conditional expectation and using the fact that ηλ/τ = 1 , it follows that

By the definition of ν j ψ t we have

Interchanging the order of integration in (18) and plugging in the above equation yields

This completes the proof of Lemma 3.

B.2 Proof of Lemma 4

To simplify the notation, denote Z t = Z ψ t . Let x ∈ X and t ≥ 0 . We have µ t glyph[lessmuch] µ t +1 with the Radon-Nikodym derivative dµ t +1 /dµ t given by

Multiplying both sides by Z t +1 /Z t and taking expectations with respect to µ t yields

In the case τ < λ , the proof is complete by the Artihmetic-Geometric mean inequality (recall that w j > 0 for j = 1 , . . . , k and ∑ k j =1 w j = 1 ). On the other hand, if τ ≥ λ then x ↦→ x λ/τ is concave. Hence, it follows that

where the final step follows via the Arithmetic-Geometric mean inequality. This completes the proof of Lemma 4.

C Proof of Theorem 2

For every t ≥ 0 and j ∈ { 1 , . . . , k } , let ˜ ν j t be the distribution returned by the approximate Sinkhorn oracle that satisfies the properties listed in Definition 1. We follow along the lines of proof of Theorem 1.

First, we will establish an upper bound on the oscillation norm of the iterates ˜ ψ t . Indeed, by the property four in Definition 1 we have

Let ˜ δ t = E ν ,w λ,τ ( ψ ∗ ) -E ν ,w λ,τ ( ˜ ψ t ) be the suboptimality gap at time t . Using the concavity upper bound (10) and the property two in Definition 1 we have

Combining the property three stated in the Definition 1 with Lemma 3 we obtain

Provided that ˜ δ t ≥ 2 ε it holds that

Let T be the first index such that ˜ δ T +1 < 2 ε and set T = ∞ if no such index exists. Then, the above equation is valid for any t ≤ T . In particular, repeating the proof of Theorem 1, for any t ≤ T we have

which completes the proof of this theorem.

D Proof of Lemma 2

The first property - the positivity of the probability mass function of ˜ ν j - is immediate from its definition.

To simplify the notation, denote in what follows

With this notation, recall that

The above is a sum of n non-negative random variables bounded by one with expectation

It follows by Hoeffding's inequality and the union bound that with probability at least 1 -δ the following holds for any j ∈ { 1 , . . . , k } and any l ∈ { 1 , . . . , m j } :

In particular, the above implies that

Notice that the above bound can be made arbitrarily close to m j ε µ by taking a large enough n and a small enough ζ . This proves the second property of Definition 1.

To prove the third property, observe that

This concludes the proof of the third property.

It remains to prove the fourth property of Definition 1. Observe that for any y, y ′ we have

Nowobserve that for any a, b > 0 the function g : [0 , ∞ ) → (0 , ∞ ) defined by g ( x ) = ( x + a ) / ( x + b ) is increasing if a < b and decreasing if a ≥ b . Thus, g is maximized either at zero or at infinity. It thus follows that

This proves the claim and completes the proof of this lemma.

E Proof of Theorem 3

The purpose of this section is to show how sampling via Langevin Monte Carlo algorithm yields the first provable convergence guarantees for computing barycenters in the free-support setup (cf. the discussion at the end of Section 2.2). In particular, we provide computational guarantees for implementing Algorithm 2.

A measure µ is said to satisfy the logarithmic Sobolev inequality (LSI) with constant C if for all sufficiently smooth functions f it holds that

To sample from a measure µ ( dx ) = exp( -f ( x )) dx supported on R d , the unadjusted Langevin Monte Carlo algorithm is defined via the following recursive update rule:

The following Theorem is due to Vempala and Wibisono [57, Theorem 3].

Theorem 4. Let µ ( dx ) = exp( -f ( x )) dx be a measure on R d . Suppose that µ satisfies LSI a with constant C and that f has L -Lipschitz gradient with respect to the Euclidean norm. Consider the sequence of iterates ( x k ) k ≥ 0 defined via (19) and let let ρ k be the distribution of x k . Then, for any ε > 0 , any η ≤ 1 8 L 2 C min { 1 , ε 4 d } , and any k ≥ 2 C η log 2KL( ρ 0 ,µ ) ε , it holds that

Thus, LSI on the measure µ provides convergence guarantees on KL( ρ k , µ ) . It is shown in [57, Lemma 1] how to initialize the iterate x 0 so that KL( ρ 0 , µ ) scales linearly with the ambient dimension d up to some additional terms. The final condition described in Problem Setting 1 ensures that (by [57, Lemma 1]) for any σ > 0 , the initialization scheme x 0 ∼ N ( x ψ , I d ) for the Langevin algorithm (19) satisfies

where L σ is the smoothness constant of V ψ /τ +dist( x, X ) / (2 σ 2 ) (see Lemma 5) and µ ψ ,σ is the probability measure defined in (20).

To implement the approximate Sinkhorn oracle described in Definition 1, we can combine Lemma 2 with approximate sampling via Langevin Monte Carlo; note that by Pinsker's inequality, KullbackLeibler divergence guarantees provide total variation guarantees which are sufficient for the application of Lemma 2. Therefore, providing provable convergence guarantees for Algorithm 2 amounts to proving that we can do arbitrarily accurate approximate sampling from distributions of the form

Here 1 X is the indicator function of X , ψ is an arbitrary iterate generated by Algorithm 2, and we consider the free-support setup characterized via the choice π ref ( dx ) = 1 X dx .

Notice that we cannot apply Theorem 4 directly because the measure µ ψ defined above has constrained support while Theorem 4 only applies for measures supported on all of R d . Nevertheless, we will show that the compactly supported measure µ ψ can be approximated by a measure µ ψ ,σ , where the parameter σ will trade-off LSI constant of µ ψ ,σ against the total variation norm between the two measures. To this end, define

The following lemma, proved at the end of this section, collects the main properties of the measure µ ψ ,σ .

Lemma 5. Consider the setup described in Problem Setting 1. Let ψ be any iterate generated by Algorithm 2 and let µ ψ ,σ be the distribution defined in (20) . Then, the measure µ ψ ,σ satisfies the following properties:

For any σ ∈ (0 , 1 / 4] it holds that

Let V σ ( x ) = exp( -V ψ ( x ) /τ -dist( x, X ) 2 / (2 σ 2 )) ; thus µ ψ ,σ ( dx ) = exp( -V σ ( x )) dx . The function V σ has L σ -Lipschitz gradient where

The measure µ ψ ,σ satisfies LSI with a constant C σ = poly( R, exp( R 2 /τ ) , L σ ) .

Above, the notation C = poly( x, y, z ) denotes a constant that depends polynomially on x, y and z . With the above lemma at hand, we are ready to prove Theorem 3.

Proof of Theorem 3. Let ψ be an arbitrary iterate generated via Algorithm 2. We can simulate a step of approximate Sinkhorn oracle with accuracy ε via Lemma 2 (with ζ = ε/ 4 ) in time poly( n, m, d ) provided access to n = poly( ε -1 , m, log( m/δ )) samples from any distribution µ ′ ψ such that

To find a choice of µ ′ ψ satisfying the above bound, consider the distribution

Let C σ and L σ be the LSI and smoothness constants of the distribution µ ψ ,σ provided in Lemma 5. By Theorem 4, it suffices to run the Langevin algorithm (19) for poly( ε -1 , m, d, C σ , L σ ) number of iterations to obtain a sample from a distribution ˜ µ ψ ,σ such that

In particular, by the triangle inequality for the total variation norm, the choice µ ′ ψ = ˜ µ ψ ,σ satisfies (21). This finishes the proof.

E.1 Proof of Lemma 5

To simplify the notation, denote µ = µ ψ , µ σ = µ ψ ,σ , V ( x ) = V ψ ( x ) /τ , and V σ ( x ) = V ( x ) /τ + dist( x, X ) 2 / (2 σ 2 ) .

Total variation norm bound. With the above shorthand notation, we have

and

We have

We thus need to upper bound Z σ . Let Vol( A ) be the Lebesgue measure of the set A , let ∂A denote the boundary of A , and let A + B = { a + b : a ∈ A,b ∈ B } be the Minkowski sum of sets A and B . Using the facts that for each j ∈ { 1 , . . . , k } we have sup y ∈X ψ j ( y ) ≤ c ∞ ( X ) ≤ 4 R 2 and that

X ⊆ B R = { x : ‖ x ‖ 2 ≤ R } we have

Bounding ( R + x ) d -1 ≤ 2 d -1 R d -1 +2 d -1 x d -1 and computing the integrals results in

Using the assumption σ ≤ 1 / 4 and using the bound Γ( d ) ≥ ( d/ 2) d/ 2 we can further simplify the above bound to

which completes the proof of the total variation bound.

Lipschitz constant of the gradient. Recall that for any any j ∈ { 1 , . . . , d } we have

Denote ˜ φ j ( x ) = φ j ( x ) -1 2 ‖ x ‖ 2 2 . Fix any x, x ′ and define g ( t ) = ˜ φ j ( x +( x ′ -x ) t ) . Then, for any t ∈ [0 , 1] we have

where

and Y j ∈ R d × m j is the matrix whose l -th column is equal to the vector y j l .

Because ˜ ψ j is concave, the bound (22) shows that φ j is 1 + 1 λ m j 4 R 2 -smooth.

Combining the above with the fact that the convex function dist( x, X ) has 1 -Lipschitz gradient [6, Proposition 12.30] proves the desired smoothness bound on the function V σ .

LSI Constant bound. The result follows, for example, by applying the sufficient log-Sobolev inequality criterion stated in [13, Corollary 2.1, Equation (2.3)], combined with the bound (22). The exact constant appearing in the log-Sobolev inequality can be traced from [13, Equation (3.10)].

F Numerical Experiments

In this section, we numerically validate our main theoretical results presented in Theorems 1 and 2. We empirically demonstrate the necessity of damping Sinkhorn iterations when τ < λ/ 2 . In addition,

we examine empirical convergence rates of Algorithms 1 and 2 (for a specific implementation of the approximate Sinkhorn oracle described below) in a simulation setup comprised of isotropic Gaussian measures. 1

Simulation setup. Let µ ∗ λ,τ denote the optimal ( λ, τ ) -barycenter; that is, µ ∗ λ,τ is the unique solution to the optimization problem (2). Let ( µ t ) t ≥ 0 denote the iterates of either Algorithm 1 or Algorithm 2. Then, the dual objective sub-optimality gap bounds of Theorems 1 and 2 combined with Lemma 1 establish the convergence of KL( µ ∗ λ,τ , µ t ) to zero as t goes to infinity.

To numerically compute KL( µ ∗ λ,τ , µ t ) we need to know the true ( λ, τ ) -barycenter µ ∗ λ,τ . The only setup where µ ∗ λ,τ admits a known closed-form expression is when the marginal measures ν j are isotropic Gaussians with identical variance. In particular, it was shown in [17, Proposition 3.4] that for ν j = N ( m j , σ 2 I d ) and for any non-negative weights ( w j ) k j =1 that sum to one, we have

Hence, in all the simulations performed in this section, we let ν j = N ( m j , σ 2 I d ) for some m j ∈ R d and σ 2 > 0 . While this simulation setup allows for exact computations of the divergence measure KL( µ ∗ λ,τ , µ t ) , there are two primary limitations in our experimental design. First, the theoretical results in this paper are proved under boundedness assumptions on the marginal measures, while Gaussian distributions are unbounded. Second, our simulations do not cover the free-support discrete point clouds setup investigated in Section 4.1 concerning the approximate Sinkhorn oracle implementable via Monte Carlo sampling.

Implementation of Algorithm 1. When the marginals ν j are Gaussian measures (not necessarily isotropic) and when ψ j 0 = 0 , then Sinkhorn updates admit a closed-form expression that result in quadratic Sinkhorn potentials ψ j , φ j and Gaussian measure µ t . In particular, the iterates of Algorithm 1 can be written as

for some matrices A t , C t , Σ t ∈ R d × d and vectors b t , d t , e t ∈ R d that can be computed using explicit recursive expressions (see, e.g., [34, 41]).

Implementation of Algorithm 2. We implement approximate Sinkhorn oracle (Definition 1) as follows. Our updates maintain the property that the potentials ψ j and φ j are quadratic functions of the form described in Equation (23). At every iteration t , we replace each matrix A j t (for j = 1 , . . . , k ) by performing the transformation A j t ↦→ A j t -N j t , where N j t is a random positive-definite matrix with trace equal to ε . In our simulations, each N j t is obtained by drawing an independent d × d matrix U with i.i.d. N (0 , 1) entries and setting N j t = ε · U glyph[latticetop] U/ trace( U glyph[latticetop] U ) . The parameter ε controls the approximation error of the implemented approximate Sinkhorn oracle.

Simulation results. We now comment on the main findings in our numerical simulations.

In Figure 1, we consider the toy setup k = 1 , ν 1 = N (0 , 1) . Figure 1a shows that undamped iterates explode when τ < λ/ 2 (note the absence of the blue line due to the explosion of iterates when λ/τ = 2 . 1 ). Figure 1b investigates the critical case τ ≈ λ/ 2 , noticing a sharp phase transition in the convergence behavior. Figure 1c demonstrates that damping removes the bad behavior. Moreover, the damping factor suggested in our work yields the fastest convergence among the tested five different damping factor choices.

In Figure 2, we investigate an undamped inexact algorithm described above. As suggested by Theorem 2, the iterates converge up to some level governed by the accuracy parameter ε of the inexact Sinkhorn oracle.

In Figure 3, we perform the same simulations as in Figure 2, but this time with damping. We observe again that damping helps significantly in the critical case τ ≈ λ/ 2 , yielding a much faster convergence rate. The second row in Figure 3 considers the overdamped case, where we observe slightly slower convergence than that with the optimal choice of the damping parameter.

In Figure 4, investigate the effect of different levels of damping in the noisy setup (Algorithm 2). First, observe that the undamped algorithm ( η = 1 ) explodes in the noisy case with τ = λ/ 2 , in contrast to the zero-noise setup reported in Figure 1a. Moreover, we find that overdamping might have a negative effect on attained accuracy: larger damping factors lead to lower accuracy at convergence in noisy setups.

0

20

40

60

80

100

Number of iterations

t

-

70

-

60

-

50

-

40

-

30

-

20

-

10

0

10

log KL(

µ

∗

λ,τ

||

µt

)

λ/τ

= 2

.

1

λ/τ

= 2

.

0

λ/τ

= 1

.

5

λ/τ

= 1

.

0

λ/τ

= 0

.

5

Algortihm 1 without damping (

η

= 1)

0

50

100

150

200

250

300

350

400

Number of iterations

t

-

40

-

30

-

20

-

10

0

log KL(

µ

∗

λ,τ

||

µt

)

λ/τ

= 2

.

003

λ/τ

= 2

.

002

λ/τ

= 2

.

0

λ/τ

= 1

.

99

λ/τ

= 1

.

95

Algortihm 1 without damping (

η

= 1)

Phase transition around

λ/τ

= 2

0

20

40

60

80

100

Number of iterations

t

-

60

-

50

-

40

-

30

-

20

-

10

0

log KL(

µ

∗

λ,τ

||

µt

)

η

= 1

.

0

η

= 0

.

85

η

= 0

.

7

η

= 0

.

5

η

= 0

.

25

Algortihm 1 with damping (

λ

= 10

-

3

, λ/τ

= 2

.

0)

0

20

40

60

80

100

Number of iterations

t

-

14

-

12

-

10

-

8

-

6

-

4

-

2

0

2

log KL(

µ

∗

λ,τ

||

µt

)

λ/τ

= 2

.

0

0

20

40

60

80

100

Number of iterations

t

-

14

-

12

-

10

-

8

-

6

-

4

-

2

0

2

λ/τ

= 1

.

5

0

20

40

60

80

100

Number of iterations

t

-

14

-

12

-

10

-

8

-

6

-

4

-

2

0

2

λ/τ

= 0

.

5

Algortihm 2 without damping (

λ

= 10

-

3

)

ε

= 0

.

001

ε

= 0

.

0001

ε

= 0

.

00001

0

20

40

60

80

100

-

14

-

12

-

10

-

8

-

6

-

4

-

2

0

2

log KL(

µ

∗

λ,τ

||

µt

)

λ/τ

= 2

.

0

, η

= 0

.

5

0

20

40

60

80

100

-

14

-

12

-

10

-

8

-

6

-

4

-

2

0

2

λ/τ

= 1

.

5

, η

= 0

.

667

0

20

40

60

80

100

-

14

-

12

-

10

-

8

-

6

-

4

-

2

0

2

λ/τ

= 0

.

5

, η

= 1

.

0

20

40

60

80

100

Number of iterations

t

-

14

-

12

-

10

-

8

-

6

-

4

-

2

0

2

log KL(

µ

∗

λ,τ

||

µt

)

λ/τ

= 2

.

0

, η

= 0

.

25

0

20

40

60

80

100

Number of iterations

t

-

14

-

12

-

10

-

8

-

6

-

4

-

2

0

2

λ/τ

= 1

.

5

, η

= 0

.

333

0

20

40

60

80

100

Number of iterations

t

-

14

-

12

-

10

-

8

-

6

-

4

-

2

0

2

λ/τ

= 0

.

5

, η

= 0

.

5

Algortihm 2 with damping (

λ

= 10

-

3

)

ε

= 0

.

001

ε

= 0

.

0001

ε

= 0

.

00001

0

20

40

60

80

100

Number of iterations

t

-

6

-

5

-

4

-

3

-

2

-

1

0

1

log KL(

µ

∗

λ,τ

||

µt

)

λ/τ

= 2

.

0

0

20

40

60

80

100

Number of iterations

t

-

6

-

5

-

4

-

3

-

2

-

1

0

1

λ/τ

= 1

.

5

0

20

40

60

80

100

Number of iterations

t

-

6

-

5

-

4

-

3

-

2

-

1

0

1

λ/τ

= 0

.

5

Algortihm 2 with damping (

λ

= 10

-

3

)

η

= 1

.

0

η

= 0

.

85

η

= 0

.

7

η

= 0

.

5

η

= 0

.

25

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents an algorithm (damped Sinkhorn) and theoretical convergence guarantees for computing doubly regularized Wasserstein barycenters. The concept of doubly entropic Wasserstein barycenters extends the single entropic regularized barycenters by introducing an additional level of regularization. This addition allows for the two regularization terms to counterbalance each other, leading to a debiasing effect under the right conditions. The authors demonstrate that the damped Sinkhorn algorithm can be implemented using an approximate Sinkhorn oracle. The latter can be evaluated using Monte Carlo sampling, providing a complete computational pipeline with theoretical guarantees.\n\nSTRENGTHS:\nThe paper tackles a complex and highly relevant problem in optimal transport and computational geometry: computing Wasserstein barycenters.\n\nThe convergence guarantees are rigorous and well-explained, enhancing the value of the proposed algorithm.\n\nThe paper connects the algorithmic problem to practical computation, demonstrating how the proposed methods can be implemented using Monte Carlo sampling. This gives a complete end-to-end view of the problem, which is valuable.\n\nWEAKNESSES:\nThe paper has some minor errors and inconsistencies, such as missing weights in Equation (2) and inconsistent notations (lack of boldface in $\\psi^*$) in Lemma 1 and Theorem 1. There is a typo in line 203 (“is can”). In Definition 1, both $\\widehat\\nu$ and $\\widetilde\\nu$ are used.\n\nIn the discussion above Lemma 2, the claim about achieving accuracy close to $\\sqrt{\\varepsilon_\\mu}$ is misleading as the bound seems to be larger than $(m_j \\varepsilon_\\mu)^{1/4}$ based on the first two terms in the bound.\n\nQUESTIONS:\nThe LSI constant derived using Holley--Stroock perturbation seems impractical due to its large value. Are there any more practical situations where this constant could be more manageable?\n\nWhat can be said about the convergence of the algorithm for an unbounded cost, e.g., the quadratic cost over $\\mathbb R^d$?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n-----\nEDIT : 6 --> 8\n-----\n\n\nThis paper builds on [15] and considers the recently introduced model of _doubly regularized entropic Wasserstein Barycenters_ which, given a set of measures $\\nu^1,\\dots, \\nu^k$, weights $(w_j)_j$ (non-negative and sum to $1$), a reference measure $\\pi$, and two smoothing parameters $\\lambda,\\tau > 0$, consider the problem\n\n$$ \\text{ minimize } \\mu \\mapsto \\tau \\mathrm{KL}(\\mu,\\pi) + \\sum_{j=1}^k w_j T_\\lambda(\\mu, \\nu_j), $$\n\nwhere\n\n$$ T_\\lambda(\\mu,\\nu) = \\inf_\\gamma \\braket{c,\\gamma} + \\lambda \\mathrm{KL}(\\gamma,\\mu\\otimes\\nu). $$\n\nDepending on the choice of the parameters $\\lambda,\\tau$, this problem encompasses many standard formulations for Wasserstein barycenters and their variations (Schrödinger barycenters, etc.). \n\nThis problem is deeply analyzed in [15] from a theoretical perspective (existence, uniqueness, smoothness of solutions, etc.), which concludes on the need to elaborate on the computational complexity of these $(\\lambda,\\tau)$-barycenters. This is the purpose of the current paper, which has two (three) main contributions to that respect : \n1. They provide a theoretical _damped Sinkhorn algorithm_ (which can been seen as an adaptation of _Iterative Bregman Projections_ of [6]) for which they prove a $O(t^{-1})$ convergence rate toward the global minimizer of the above functional ($t$ being the number of steps). In a nutshell, the trick consists of replacing the \"learning rate\" $\\lambda$ in the standard Sinkhorn algorithm by the quantity $\\min(\\lambda,\\tau)$. \n2. They provide a  _approximated_ algorithm for which an implementation is possible, provided one has access to an \"ApproximatedSinkhornOracle\", and for which convergence rate are accessible,\n3. (2bis) They provide an example of such a practical Approximated Sinkhorn Oracle based on random sampling techniques. \n\n\n### References\n[15] : (Doubly regularized Entropic Wasserstein Barycenters, Chizat, 2023)\n\nSTRENGTHS:\nFirst, the paper is overall very well written. It manages to remain concise without sacrificing the mathematical rigor, and basically every line/equation is interesting, which is highly appreciated. \n\nI also believe that its theoretical contributions are fairly significant and quite insightful and clearly speak in favor of adding more _outer regularization_ in OT-related problems. \n\nThe proofs have been investigated (with the exception of those of Lemma 2 and 3) and no major flaw was detected; the clarity of the presentation make them easy to parse while not being trivial.\n\nWEAKNESSES:\n# Major issue \n\n**The paper feels somewhat incomplete.** This is my main (and almost single) issue with the work (which undoubtedly has great potential). In a nutshell, this work is dedicated to the elaboration of provably converging **practical** algorithms to compute $(\\lambda,\\tau)$-barycenters; hence having absolutely no numerical illustration of such algorithms is somewhat surprising---even more given that the implementation seems to exist ($\\ell$199, \"we have observed empirically\"; why not showcasing it?). \n\nTypically, the divergence of the standard Sinkhorn algorithm (when $\\tau < \\lambda/2$) and the convergence of the damped one would have been nice to observe (especially given that the former is only empirical). Similarly, the paper mentions both in the abstract and in the conclusion that the approach works for _both fixed and free-support_ models (which boils down to the choice of the reference measure $\\pi$), but this is (almost) not developed in the work. In my opinion, it is fascinating to have a globally converging way to approximate Wasserstein barycenter, and showcasing empirical situations where the proposed approach provides much better results than the \"naive\" one (say, IBP). \n\nSimilarly, the work mentions at several occasions the edge case $\\tau = \\lambda/2$ (because of the powerful statistical properties showcased in [15]), but never leverage this setting as far as I can tell. I do not know if there is something specific to say from the algorithmic perspective, but here as well, I feel like this could have been a nice occasion to numerically showcase the debiasing effect (for instance). \n\nDo not get me wrong: I believe that the current contributions of the paper are of interest. But in my opinion, there are few elements that are missing to make the paper complete. \n\n# Minor issues and other remark/suggestions\n\nNote: this are not actual criticism that are expected to be addressed during the discussion period immediatly, but rather comments that may hopefully be useful for the authors. \n\n- The hyperlinks are broken (probably due to the split of the pdf between the main body and the supplementary material). It may be nice to fix that for the camera ready version (which implies, I guess, to compile the supplementary material independently. Note that you can include the main body in the supplementary, which makes it \"self contained\" and quite handy for reviewers/readers). \n- [typo] In Eq. (2), I think that the $(w_j)_j$ are missing. This occurs in other places as well (Eq 8, def of $\\mu_\\psi, Z_\\phi$). \n- [typo] The parameters $\\nu,w, \\lambda,\\tau$ for the objective function $E$ are sometime missing (e.g. in Eq 10). This can be slightly confusing given the proximity with the expectation operator $\\mathbf{E}$. \n- [typo] $\\ell$193, $\\nu_j^t$ should be $\\nu_t^j$ I guess. \n-  [typo] In definition 1, the notation $\\hat{\\nu}$ and $\\tilde{\\nu}$ are both used (in my understanding, they denote the same quantity). \n- [typo] $\\ell$232, I think it should be \"the proof _of its convergence_ is deferred...\". \n- [suggestion] That's trivial but it may be handy to write in $\\ell$215 that  $\\delta_{t}-\\delta_{t+1} = E_{\\lambda,\\tau}^{\\nu,w} (\\phi_{t+1}) - E_{\\lambda,\\tau}^{\\nu,w} (\\phi_{t}) \\geq 0$ by Prop 1, making the sequence of error non-increasing (instead of saying it in $\\ell$222). \n- [suggestion] Lemma 1 is never referenced after being stated (only once in the contributions section). While it is of course obvious, it may be worth adding a short sentence after Theorem 1 saying something like \"Theorem 1 together with Lemma 1 implies the convergence of $\\mu_t$ toward $\\mu_{\\lambda,\\tau}$.\n- [typo] $\\ell$430, I think that $\\Delta_t^j$ should be defined without the $\\log$.\n\nQUESTIONS:\n# Main questions\n\nThese are questions/suggestion I would like to see addressed during the discussion period. Given what I have written above, they naturally rely on (synthetic) numerical experiments that may hopefully shed some light on the strengths and weaknesses of the proposed approach.\n\n1. Is there a computational price to pay for the _damped_ Sinkhorn? Namely, from what is written in the paper, I assume that the \"naive\" Sinkhorn algorithm (with $\\eta = 1$ no matter $\\tau$) does converge when $\\tau \\in [\\lambda/2 , \\lambda]$ (at least empirically). If so, (i) does it provide (empirically) the same output (asymptotically in $t$) as the output of the damped algorithm? (ii) if yes, is it faster? (I would expect that it is, since the damped algorithm somewhat slow the gradient descent). \n2. Can you elaborate on $\\pi_{\\mathrm{ref}}$ be the Lebesgue measure encode \"the free support case\"? If my understanding is correct, this forces $\\mu$ to have a density, which is not exactly what \"free-support\" refers to in the work of Cuturi and Doucet (2014) (but I agree that their terminology can be discussed as well as they fix the cardinality of the support). But typically, if the $\\nu^j$ are discrete, their $(0,0)$-barycenter is discrete as well (Carlier & Ekeland, 2015) and in that case, it may be hard for $\\mu_t$ to concentrate. From [15, Thm 3.2] we may expect a $\\lambda^2$ proximity between the exact Wasserstein barycenter and the $(\\lambda,\\lambda/2)$ one, but as far as I can tell this theorem is proved only in the case where the measures have densities. So I would be interested to see if/how much things fail in the discrete setting (now that it is possible to implement it!). \n\n\n# Minor questions\n\nThese are minor questions that I'm asking because I am interested in the work, but I would not be offended if the authors do not address them during the discussion period. \n\n- From my reading of the proofs, I missed the point where using the damped algorithm is crucial. I think that I have a correct line-by-line understanding of the proof, but not a global vision; could you tell me what would fail if one tries to run the same proof but with $\\eta = 1$? I guess it appears in the proof of Lemma 4 (case $\\tau < \\lambda$), which I did not investigated; is it possible to have a global picture of why it is required?\n- In the way Theorem 2 is currently stated, nothing is said once the condition on $T$ is reached (assuming $T < \\infty$). Is it clear that once the condition is reached, the gap remains below $2\\epsilon$? As far as I can tell by checking the proof, it is not sure that the sequence $\\tilde{\\delta}(t) - \\tilde{\\delta}(t+1)$ is non-increasing (once the criterion is reached), may it happen that the gap increases again in an uncontrolled way? (i.e. at time $T$ we are below $2\\epsilon$, but at $T+1$ we suddenly get something much worse and we have to keep running the gradient descent again) I think that (if such thing can happen) this can be avoided by simply checking the variations of the objective value.\n\nLIMITATIONS:\nI do not identify specific limitation of potential negative societal impact specific to this work. \n\nFor the former point, I guess that additional numerical experiments may showcase some limitations of the work, that would be worth discussing then (without diminishing the contributions of the work).", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-29"}

comments: SUMMARY:
The paper has proposed a computational algorithm for computing the newly developed regularized Wasserstein barycenters in [Chizat,2023] via optimizing the duality of the primal problem. The paper also characterised the convergence of algorithms on both exact and approximated algorithms which are the main contributions of the paper.

STRENGTHS:
The algorithms and their convergence guarantee for computing doubly entropic Wasserstein barycenters are significant contributions in terms of technical aspects. The paper is easy to follow as the related results have been well presented to make use of the derivations of the proposed algorithms.

WEAKNESSES:
There are some aspects the paper can be improved:
- A section for numerical experiments/illustrations for both synthetic and real-world data will help to demonstrate the results of the two main algorithms and their convergence algorithms.
- Algorithms should link to the text updating equations. For instance, in Alg. 1, step 2(a) computes the left part of the equation (11) while step 2(b-d) computes the right part of that equation, authors break down into multiple lines of code without clear purposes which usually make readers harder to follow.
- In line 199, "We have observed empirically that the iterates of the iterative Bregman projections (i.e., the scheme of updates (12), (11)) diverge whenever \tau < \lamba<=2", it is not clear where the results with the iterative Bregman projections presented.

QUESTIONS:
As the doubly entropic Wasserstein barycenter may have some interesting properties for some specific type of data if the author can discuss further in terms of theoretical and experimental aspects that would be great.

LIMITATIONS:
No empirical result provided.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an algorithm for solving the doubly regularized Wasserstein barycenter problem for probability measures that corresponds to adding an inner regularization based on the entropy penalty appearing in the Wasserstein distance term, and an outer regularization appearing at the level of the Wasserstein barycenter problem, based on the KL divergence. The algorithm comes with convergence guarantees, depending on inner and outer regularization parameters.The algorithm is then modified to approximately solve the doubly regularized Wasserstein barycenter problem, but with non-asymptotic convergence guarantees.\n\nSTRENGTHS:\nThis article is very well written, proposing an algorithm for a recently introduced notion of regularization for the Wasserstein barycenter, unifying several proposals in the literature. The proposed algorithm does not rely on space discretization (prohibitive in high dimensions). The proofs are accurate, well documented and complete. The most important contribution lies in the approximate damped Sinkhorn scheme, which allows for non-asymptotic convergence guarantees on the value of the dual objective.\n\nWEAKNESSES:\nThe idea of dumping an algorithm is not really innovative, but it stabilizes the algorithm and provides convergence guarantees for any positive values of the parameters $(\\lambda,\\tau)$.\n\nA result on the distance between barycenters obtained via the exact and approximate damped Sinkhorn scheme could have been interesting. In particular, the addition of experiments on the barycenters obtained via both algorithms would strengthen the paper.\n\nFinally, an intuition on the approximate Sinkhorn oracle (Definition 1) is important in my opinion, as the properties listed in Definition 1 are quite precise. An example of a Radon-Nikodym derivative that does not satisfy these properties could be added.\n\nQUESTIONS:\nProbability measures are supported on a compact convex subset of $\\mathbb{R}^d$, could this be relaxed?\n\nIn practice, how does the number of support points of the discrete barycenter is chosen? More generally, how do you choose $\\pi_{ref}$?\n\nLIMITATIONS:\nAn experimental (and if possible theoretical) comparison of the barycenters obtained by the exact and approximate proposed algorithms, as well as a comparison with Bregman iterative projections algorithm for particular choices of regularization parameters would, in my opinion, improve this article.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a study on the computation of doubly regularized Wasserstein barycenters, a recently introduced family of entropic barycenters with inner and outer regularization strengths. The authors build upon previous research, which has shown that different choices of regularization parameters unify various entropy-penalized barycenter concepts, including debiased barycenters.\n\nThe proposed algorithm for computing doubly regularized Wasserstein barycenters combines damped Sinkhorn iterations with exact maximization/minimization steps, ensuring convergence for any choice of regularization parameters. Additionally, the authors introduce an inexact variant of the algorithm that utilizes Sinkhorn Oracle definition, providing non-asymptotic convergence guarantees for approximating Wasserstein barycenters between discrete point clouds in the free-support/grid-free setting. The authors highlight that while a straightforward adaptation of the alternate maximization scheme leads to diverging iterates for small values of $\\tau$, their analysis demonstrates that damping these iterations is sufficient to achieve convergence.\n\nSTRENGTHS:\n1. Clear structure, good mathematical exposition, and solid theoretical results. \n2. Detailed proofs to support their claims, demonstrating a strong theoretical foundation\n\nWEAKNESSES:\n1. Lack of numerical experiments and empirical evaluations. \n2. While the paper discusses computational complexity, further analysis and discussion on the scalability and efficiency of the algorithm could provide a more comprehensive understanding of its practical implications.\n\nQUESTIONS:\nI have the following questions/comments for the authors:\n\n1. Can the authors provide more motivation for using the definition of Sinkhorn Oracle in Section 4, Approximate Damped Sinkhorn Scheme? It would be helpful to understand the rationale behind this choice and how it contributes to the proposed algorithm.\n2. In Section 2.2, Doubly Regularized Entropic Barycenters, could the optimization part (Line 172 - 184) be made more clear and detailed? This section seems to be the foundation for the updates in your main contribution algorithm. Providing additional explanations and elaboration would enhance the understanding of the optimization process. \n3. Algorithm 1 is presented without a thorough mathematical derivation. It would be beneficial to include a more detailed derivation or explanation of the algorithm to aid readers in understanding the underlying principles and steps involved.\n\nLIMITATIONS:\nWhile the paper provides theoretical guarantees and analyses, incorporating visualizations and benchmarking against alternative approaches would enhance the practical significance of the proposed algorithm", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}