SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbf {E} _ {Y \sim \nu^ {j}} \left[ \frac {\nu_ {\psi} ^ {j} (Y)}{\widetilde {\nu} _ {\psi} ^ {j} (Y)} \right] = \mathbf {E} _ {Y \sim \nu^ {j}} \left[ \frac {\widehat {\nu} _ {\psi} ^ {j} (Y)}{\widetilde {\nu} _ {\psi} ^ {j} (Y)} + \frac {\nu_ {\psi} ^ {j} (Y) - \widehat {\nu} _ {\psi} ^ {j} (Y)}{\widetilde {\nu} _ {\psi} ^ {j} (Y)} \right] \\ \leq \mathbf {E} _ {Y \sim \nu^ {j}} \left[ \frac {1}{1 - \zeta} + \frac {\nu_ {\psi} ^ {j} (Y) - \widehat {\nu} _ {\psi} ^ {j} (Y)}{\widetilde {\nu} _ {\psi} ^ {j} (Y)} \right] \\ \leq \mathbf {E} _ {Y \sim \nu^ {j}} \left[ 1 + \frac {\zeta}{1 - \zeta} + \frac {\left| \nu_ {\psi} ^ {j} (Y) - \widehat {\nu} _ {\psi} ^ {j} (Y) \right|}{\widetilde {\nu} _ {\psi} ^ {j} (Y)} \right] \\ \leq 1 + \frac {\zeta}{1 - \zeta} + \frac {1}{\zeta} \| \nu_ {\boldsymbol {\psi}} ^ {j} (Y) - \widehat {\nu} _ {\boldsymbol {\psi}} ^ {j} (Y) \| _ {\mathrm {T V}} \\ \leq 1 + 2 \zeta + \frac {1}{\zeta} \left(m _ {j} \varepsilon_ {\mu} + m _ {j} \sqrt {\frac {2 \log \left(\frac {2 m}{\delta}\right)}{n}}\right). \\ \end{array}
$$
This concludes the proof of the third property.
It remains to prove the fourth property of Definition 1. Observe that for any  $y, y'$  we have
$$
\begin{array}{l} \left(\psi^ {j} (y) - \eta \lambda \log \frac {\widetilde {\nu} ^ {j} (y)}{\nu^ {j} (y)}\right) - \left(\psi^ {j} (y ^ {\prime}) - \eta \lambda \log \frac {\widetilde {\nu} ^ {j} (y ^ {\prime})}{\nu^ {j} (y ^ {\prime})}\right) \\ = \left(\psi^ {j} (y) - \psi^ {j} \left(y ^ {\prime}\right)\right) + \eta \lambda \log \left(\frac {\zeta + (1 - \zeta) \frac {1}{n} \sum_ {i = 1} ^ {n} K ^ {j} \left(X _ {i} , y ^ {\prime}\right)}{\zeta + (1 - \zeta) \frac {1}{n} \sum_ {i = 1} ^ {n} K ^ {j} \left(X _ {i} , y\right)}\right) \\ = \left(\psi^ {j} (y) - \psi^ {j} \left(y ^ {\prime}\right)\right) + \eta \lambda \log \left(\frac {\frac {\zeta}{1 - \zeta} + \frac {1}{n} \sum_ {i = 1} ^ {n} K ^ {j} \left(X _ {i} , y ^ {\prime}\right)}{\frac {\zeta}{1 - \zeta} + \frac {1}{n} \sum_ {i = 1} ^ {n} K ^ {j} \left(X _ {i} , y\right)}\right) \\ \leq \left(\psi^ {j} (y) - \psi^ {j} (y ^ {\prime})\right) + \eta \lambda \log \left(\frac {\frac {\zeta}{1 - \zeta} + \exp \left(\frac {c _ {\infty} (\mathcal {X}) + \psi^ {j} (y ^ {\prime}) - \psi^ {j} (y)}{\lambda}\right) \frac {1}{n} \sum_ {i = 1} ^ {n} K ^ {j} (X _ {i} , y)}{\frac {\zeta}{1 - \zeta} + \frac {1}{n} \sum_ {i = 1} ^ {n} K ^ {j} (X _ {i} , y)}\right). \\ \end{array}
$$
Now observe that for any  $a, b > 0$  the function  $g:[0,\infty) \to (0,\infty)$  defined by  $g(x) = (x + a) / (x + b)$  is increasing if  $a < b$  and decreasing if  $a \geq b$ . Thus,  $g$  is maximized either at zero or at infinity. It thus follows that
$$
\begin{array}{l} \eta \lambda \log \left(\frac {\frac {\zeta}{1 - \zeta} + \exp \left(\frac {c _ {\infty} (\mathcal {X})}{\lambda}\right) \frac {1}{n} \sum_ {i = 1} ^ {n} K ^ {j} (X _ {i} , y)}{\frac {\zeta}{1 - \zeta} + \frac {1}{n} \sum_ {i = 1} ^ {n} K ^ {j} (X _ {i} , y)}\right) \\ \leq \left\{ \begin{array}{l l} \eta c _ {\infty} (\mathcal {X}) - \eta (\psi^ {j} (y) - \psi^ {j} (y ^ {\prime})) & \text {i f} \exp \left(\frac {c _ {\infty} (\mathcal {X})}{\lambda}\right) \geq 1 \\ 0 & \text {o t h e r w i s e .} \end{array} \right. \\ \end{array}
$$
This proves the claim and completes the proof of this lemma.
E Proof of Theorem 3
The purpose of this section is to show how sampling via Langevin Monte Carlo algorithm yields the first provable convergence guarantees for computing barycenters in the free-support setup (cf. the discussion at the end of Section 2.2). In particular, we provide computational guarantees for implementing Algorithm 2.
A measure  $\mu$  is said to satisfy the logarithmic Sobolev inequality (LSI) with constant  $C$  if for all sufficiently smooth functions  $f$  it holds that
$$
\mathbf {E} _ {\mu} \left[ f ^ {2} \log f ^ {2} \right] - \mathbf {E} _ {\mu} \left[ f ^ {2} \right] \log \mathbf {E} _ {\mu} \left[ g ^ {2} \right] \leq 2 C \mathbf {E} _ {\mu} \left[ \| \nabla f \| _ {2} ^ {2} \right].
$$
To sample from a measure  $\mu(dx) = \exp(-f(x)) dx$  supported on  $\mathbb{R}^d$ , the unadjusted Langevin Monte Carlo algorithm is defined via the following recursive update rule:
$$
x _ {k + 1} = x _ {k} - \eta \nabla f (x _ {k}) + \sqrt {2 \eta} Z _ {k}, \quad \text {w h e r e} \quad Z _ {k} \sim \mathcal {N} (0, I _ {d}). \tag {19}
$$
The following Theorem is due to Vempala and Wibisono [57, Theorem 3].
Theorem 4. Let  $\mu(dx) = \exp(-f(x))dx$  be a measure on  $\mathbb{R}^d$ . Suppose that  $\mu$  satisfies LSI with constant  $C$  and that  $f$  has L-Lipschitz gradient with respect to the Euclidean norm. Consider the sequence of iterates  $(x_k)_{k \geq 0}$  defined via (19) and let  $\rho_k$  be the distribution of  $x_k$ . Then, for any  $\varepsilon > 0$ , any  $\eta \leq \frac{1}{8L^2C} \min\{1, \frac{\varepsilon}{4d}\}$ , and any  $k \geq \frac{2C}{\eta} \log \frac{2KL(\rho_0, \mu)}{\varepsilon}$ , it holds that
$$
\mathrm {K L} (\rho_ {k}, \mu) \leq \varepsilon .
$$
Thus, LSI on the measure  $\mu$  provides convergence guarantees on  $\mathrm{KL}(\rho_k,\mu)$ . It is shown in [57, Lemma 1] how to initialize the iterate  $x_0$  so that  $\mathrm{KL}(\rho_0,\mu)$  scales linearly with the ambient dimension  $d$  up to some additional terms. The final condition described in Problem Setting 1 ensures that (by [57, Lemma 1]) for any  $\sigma >0$ , the initialization scheme  $x_0\sim \mathcal{N}(x_\psi ,I_d)$  for the Langevin algorithm (19) satisfies
$$
\mathrm {K L} (\rho_ {0}, \mu_ {\psi , \sigma}) \leq \frac {c _ {\infty} (\mathcal {X})}{\tau} + \frac {d}{2} \log \frac {L _ {\sigma}}{2 \pi},
$$
where  $L_{\sigma}$  is the smoothness constant of  $V_{\psi} / \tau + \mathrm{dist}(x, \mathcal{X}) / (2\sigma^2)$  (see Lemma 5) and  $\mu_{\psi, \sigma}$  is the probability measure defined in (20).
To implement the approximate Sinkhorn oracle described in Definition 1, we can combine Lemma 2 with approximate sampling via Langevin Monte Carlo; note that by Pinsker's inequality, Kullback-Leibler divergence guarantees provide total variation guarantees which are sufficient for the application of Lemma 2. Therefore, providing provable convergence guarantees for Algorithm 2 amounts to proving that we can do arbitrarily accurate approximate sampling from distributions of the form
$$
\mu_ {\psi} (d x) \propto \mathbb {1} _ {\mathcal {X}} (x) \exp (- V _ {\psi} (x) / \tau) d x, \quad \text {w h e r e} \quad V _ {\psi} (x) = \sum_ {j = 1} ^ {k} w _ {j} \phi_ {\psi^ {j}} ^ {j} (x).
$$
Here  $\mathbb{1}_{\mathcal{X}}$  is the indicator function of  $\mathcal{X}$ ,  $\psi$  is an arbitrary iterate generated by Algorithm 2, and we consider the free-support setup characterized via the choice  $\pi_{\mathrm{ref}}(dx) = \mathbb{1}_{\mathcal{X}}dx$ .
Notice that we cannot apply Theorem 4 directly because the measure  $\mu_{\psi}$  defined above has constrained support while Theorem 4 only applies for measures supported on all of  $\mathbb{R}^d$ . Nevertheless, we will show that the compactly supported measure  $\mu_{\psi}$  can be approximated by a measure  $\mu_{\psi,\sigma}$ , where the parameter  $\sigma$  will trade-off LSI constant of  $\mu_{\psi,\sigma}$  against the total variation norm between the two measures. To this end, define
$$
\mu_ {\psi , \sigma} = \propto \exp (- V _ {\psi} (x) / \tau - \operatorname {d i s t} (x, \mathcal {X}) ^ {2} / (2 \sigma^ {2})) d x, \quad \text {w h e r e} \quad \operatorname {d i s t} (x, \mathcal {X}) = \inf  _ {y \in \mathcal {X}} \| x - y \| _ {2}. \tag {20}
$$
The following lemma, proved at the end of this section, collects the main properties of the measure  $\mu_{\psi,\sigma}$ .
Lemma 5. Consider the setup described in Problem Setting 1. Let  $\psi$  be any iterate generated by Algorithm 2 and let  $\mu_{\psi, \sigma}$  be the distribution defined in (20). Then, the measure  $\mu_{\psi, \sigma}$  satisfies the following properties:
1. For any  $\sigma \in (0, 1/4]$  it holds that
$$
\| \mu_ {\psi} - \mu_ {\psi , \sigma} \| _ {\mathrm {T V}} \leq 2 \sigma \exp \left(\frac {8 R ^ {2}}{\tau}\right) \left[ \left(4 R d ^ {- 1 / 4}\right) ^ {d - 1} + 1 \right].
$$
2. Let  $V_{\sigma}(x) = \exp(-V_{\psi}(x)/\tau - \mathrm{dist}(x, \mathcal{X})^2/(2\sigma^2))$ ; thus  $\mu_{\psi, \sigma}(dx) = \exp(-V_{\sigma}(x)) dx$ . The function  $V_{\sigma}$  has  $L_{\sigma}$ -Lipschitz gradient where
$$
L _ {\sigma} = \frac {1}{\tau} + \frac {1}{\tau \lambda} 4 R ^ {2} \max _ {j} m _ {j} + \frac {1}{\sigma^ {2}}.
$$
3. The measure  $\mu_{\psi, \sigma}$  satisfies LSI with a constant  $C_{\sigma} = \mathrm{poly}(R, \exp(R^2 / \tau), L_{\sigma})$ .
Above, the notation  $C = \mathrm{poly}(x,y,z)$  denotes a constant that depends polynomially on  $x,y$  and  $z$ . With the above lemma at hand, we are ready to prove Theorem 3.
Proof of Theorem 3. Let  $\psi$  be an arbitrary iterate generated via Algorithm 2. We can simulate a step of approximate Sinkhorn oracle with accuracy  $\varepsilon$  via Lemma 2 (with  $\zeta = \varepsilon /4$ ) in time  $\mathrm{poly}(n,m,d)$  provided access to  $n = \mathrm{poly}(\varepsilon^{-1},m,\log (m / \delta))$  samples from any distribution  $\mu_{\psi}^{\prime}$  such that
$$
\left\| \mu_ {\psi} ^ {\prime} - \mu_ {\psi} \right\| _ {\mathrm {T V}} \leq \frac {\varepsilon^ {2}}{1 6 m}. \tag {21}
$$
To find a choice of  $\mu_{\psi}^{\prime}$  satisfying the above bound, consider the distribution
$$
\mu_ {\psi , \sigma} \quad \mathrm {w i t h} \quad \sigma = \frac {\varepsilon^ {2}}{3 2 m} \cdot \left(2 \exp \left(\frac {8 R ^ {2}}{\tau}\right) \left[ \left(4 R d ^ {- 1 / 4}\right) ^ {d - 1} + 1 \right]\right) ^ {- 1}.
$$
Let  $C_{\sigma}$  and  $L_{\sigma}$  be the LSI and smoothness constants of the distribution  $\mu_{\psi, \sigma}$  provided in Lemma 5. By Theorem 4, it suffices to run the Langevin algorithm (19) for  $\mathrm{poly}(\varepsilon^{-1}, m, d, C_{\sigma}, L_{\sigma})$  number of iterations to obtain a sample from a distribution  $\widetilde{\mu}_{\psi, \sigma}$  such that
$$
\| \widetilde {\mu} _ {\boldsymbol {\psi}, \sigma} - \mu_ {\boldsymbol {\psi}, \sigma} \| _ {\mathrm {T V}} \leq \frac {\varepsilon^ {2}}{3 2 m}.
$$
In particular, by the triangle inequality for the total variation norm, the choice  $\mu_{\psi}^{\prime} = \widetilde{\mu}_{\psi ,\sigma}$  satisfies (21). This finishes the proof.
To simplify the notation, denote  $\mu = \mu_{\psi}, \mu_{\sigma} = \mu_{\psi, \sigma}$ ,  $V(x) = V_{\psi}(x) / \tau$ , and  $V_{\sigma}(x) = V(x) / \tau + \mathrm{dist}(x, \mathcal{X})^2 / (2\sigma^2)$ .
Total variation norm bound. With the above shorthand notation, we have
$$
\mu (d x) = \mathbb {1} _ {\mathcal {X}} Z ^ {- 1} \exp (- V (x)) d x, \quad \mathrm {w h e r e} \quad Z = \int_ {\mathcal {X}} \exp (- V (x)) d x
$$
$$
\mu_ {\sigma} (d x) = (Z + Z _ {\sigma}) ^ {- 1} \exp (- V _ {\sigma} (x)) d x, \quad \mathrm {w h e r e} \quad Z _ {\sigma} = \int_ {\mathbb {R} ^ {d} \setminus \mathcal {X}} \exp (- V _ {\sigma} (x)) d x.
$$
$$
\begin{array}{l} \| \mu - \mu_ {\sigma} \| _ {\mathrm {T V}} = \int_ {\mathbb {R} ^ {d} \setminus \mathcal {X}} (Z + Z _ {\sigma}) ^ {- 1} \exp (- V _ {\sigma} (x)) d x + \int_ {\mathcal {X}} | (Z + Z _ {\sigma}) ^ {- 1} - Z ^ {- 1} | \exp (- V (x)) d x \\ = \frac {2 Z _ {\sigma}}{Z + Z _ {\sigma}} \leq \frac {2 Z _ {\sigma}}{Z} \leq 2 \exp \left(\frac {c _ {\infty} (\mathcal {X})}{\tau}\right) Z _ {\sigma} \leq 2 \exp \left(\frac {4 R ^ {2}}{\tau}\right) Z _ {\sigma}. \\ \end{array}
$$
We thus need to upper bound  $Z_{\sigma}$ . Let  $\operatorname{Vol}(A)$  be the Lebesgue measure of the set  $A$ , let  $\partial A$  denote the boundary of  $A$ , and let  $A + B = \{a + b : a \in A, b \in B\}$  be the Minkowski sum of sets  $A$  and  $B$ . Using the facts that for each  $j \in \{1, \dots, k\}$  we have  $\sup_{y \in \mathcal{X}} \psi^j(y) \leq c_{\infty}(\mathcal{X}) \leq 4R^2$  and that
$\mathcal{X}\subseteq \mathcal{B}_R = \{x:||x||_2\leq R\}$  we have
$$
\begin{array}{l} Z _ {\sigma} = \int_ {\mathbb {R} ^ {d} \backslash \mathcal {X}} \exp (- V _ {\sigma} (x)) d x \\ \leq \exp \left(\frac {4 R ^ {2}}{\tau}\right) \int_ {\mathbb {R} ^ {d} \backslash \mathcal {X}} \exp \left(- \frac {\operatorname {d i s t} (x , \mathcal {X})}{2 \sigma^ {2}}\right) d x \\ = \exp \left(\frac {4 R ^ {2}}{\tau}\right) \int_ {0} ^ {\infty} \operatorname {V o l} \left(\partial \left(\mathcal {X} + \mathcal {B} _ {x}\right)\right) \exp \left(- \frac {x ^ {2}}{2 \sigma^ {2}}\right) d x \\ \leq \exp \left(\frac {4 R ^ {2}}{\tau}\right) \int_ {0} ^ {\infty} \operatorname {V o l} \left(\partial \mathcal {B} _ {R + x}\right) \exp \left(- \frac {x ^ {2}}{2 \sigma^ {2}}\right) d x \\ = \exp \left(\frac {4 R ^ {2}}{\tau}\right) \frac {\pi^ {d / 2}}{\Gamma (d / 2)} \int_ {0} ^ {\infty} (R + x) ^ {d - 1} \exp \left(- \frac {x ^ {2}}{2 \sigma^ {2}}\right) d x. \\ \end{array}
$$
Bounding  $(R + x)^{d - 1} \leq 2^{d - 1} R^{d - 1} + 2^{d - 1} x^{d - 1}$  and computing the integrals results in
$$
\begin{array}{l} \| \mu - \mu_ {\sigma} \| _ {\mathrm {T V}} \leq 2 \exp \left(\frac {8 R ^ {2}}{\tau}\right) \frac {\pi^ {d / 2}}{\Gamma (d / 2)} 2 ^ {d - 1} \left[ R ^ {d - 1} \sigma \frac {\sqrt {\pi}}{2} + 2 ^ {d / 2 - 1} \Gamma (d / 2) \sigma^ {d} \right] \\ \leq 2 \sigma \exp \left(\frac {8 R ^ {2}}{\tau}\right) \left[ \frac {(2 R) ^ {d - 1}}{\Gamma (d / 2)} + (4 \sigma) ^ {d - 1} \right]. \\ \end{array}
$$
Using the assumption  $\sigma \leq 1/4$  and using the bound  $\Gamma(d) \geq (d/2)^{d/2}$  we can further simplify the above bound to
$$
\left\| \mu - \mu_ {\sigma} \right\| _ {\mathrm {T V}} \leq 2 \sigma \exp \left(\frac {8 R ^ {2}}{\tau}\right) \left[ \left(4 R d ^ {- 1 / 4}\right) ^ {d - 1} + 1 \right],
$$
which completes the proof of the total variation bound.
Lipschitz constant of the gradient. Recall that for any any  $j \in \{1, \ldots, d\}$  we have
$$
\phi^ {j} (x) - \frac {1}{2} \| x \| _ {2} ^ {2} = - \lambda \log \left(\sum_ {l = 1} ^ {n _ {j}} \exp \left(\frac {\psi^ {j} \left(y _ {l} ^ {j}\right) - \frac {\| y _ {l} ^ {j} \| _ {2} ^ {2}}{2} + \langle x , y _ {l} ^ {j} \rangle}{\lambda}\right) \nu^ {j} \left(y _ {l} ^ {j}\right)\right).
$$
Denote  $\widetilde{\phi}^j (x) = \phi^j (x) - \frac{1}{2}\| x\| _2^2$ . Fix any  $x,x^{\prime}$  and define  $g(t) = \widetilde{\phi}^{j}(x + (x^{\prime} - x)t)$ . Then, for any  $t\in [0,1]$  we have
$$
g ^ {\prime \prime} (s) = - \frac {1}{\lambda} \operatorname {V a r} _ {L \sim \rho_ {t}} \left[ \left(Y ^ {j} \left(x ^ {\prime} - x\right)\right) _ {L} \right] \geq - \frac {1}{\lambda} \| x - x ^ {\prime} \| _ {2} ^ {2} m _ {j} 4 R ^ {2}, \tag {22}
$$
$$
\rho_ {t} (l) \propto \nu (y _ {l} ^ {j}) \exp \left(\frac {\psi^ {j} (y _ {l} ^ {j}) - \frac {\| y _ {l} ^ {j} \| _ {2} ^ {2}}{2} + \langle x + t (x ^ {\prime} - x) , y _ {l} ^ {j} \rangle}{\lambda}\right)
$$
and  $Y^{j} \in \mathbb{R}^{d \times m_{j}}$  is the matrix whose  $l$ -th column is equal to the vector  $y_{l}^{j}$ .
Because  $\widetilde{\psi}^j$  is concave, the bound (22) shows that  $\phi^j$  is  $1 + \frac{1}{\lambda} m_j 4R^2$ -smooth.
Combining the above with the fact that the convex function  $\mathrm{dist}(x, \mathcal{X})$  has 1-Lipschitz gradient [6, Proposition 12.30] proves the desired smoothness bound on the function  $V_{\sigma}$ .
LSI Constant bound. The result follows, for example, by applying the sufficient log-Sobolev inequality criterion stated in [13, Corollary 2.1, Equation (2.3)], combined with the bound (22). The exact constant appearing in the log-Sobolev inequality can be traced from [13, Equation (3.10)].
In this section, we numerically validate our main theoretical results presented in Theorems 1 and 2. We empirically demonstrate the necessity of damping Sinkhorn iterations when  $\tau < \lambda /2$ . In addition,
we examine empirical convergence rates of Algorithms 1 and 2 (for a specific implementation of the approximate Sinkhorn oracle described below) in a simulation setup comprised of isotropic Gaussian measures.
Simulation setup. Let  $\mu_{\lambda,\tau}^{*}$  denote the optimal  $(\lambda,\tau)$ -barycenter; that is,  $\mu_{\lambda,\tau}^{*}$  is the unique solution to the optimization problem (2). Let  $(\mu_t)_{t\geq 0}$  denote the iterates of either Algorithm 1 or Algorithm 2. Then, the dual objective sub-optimality gap bounds of Theorems 1 and 2 combined with Lemma 1 establish the convergence of  $\mathrm{KL}(\mu_{\lambda,\tau}^{*},\mu_t)$  to zero as  $t$  goes to infinity.
To numerically compute  $\mathrm{KL}(\mu_{\lambda,\tau}^{*}, \mu_t)$  we need to know the true  $(\lambda, \tau)$ -barycenter  $\mu_{\lambda,\tau}^{*}$ . The only setup where  $\mu_{\lambda,\tau}^{*}$  admits a known closed-form expression is when the marginal measures  $\nu^j$  are isotropic Gaussians with identical variance. In particular, it was shown in [17, Proposition 3.4] that for  $\nu^j = N(m_j, \sigma^2 I_d)$  and for any non-negative weights  $(w_j)_{j=1}^k$  that sum to one, we have
$$
\mu_ {\lambda , \tau} ^ {*} = N \left(\sum_ {j = 1} ^ {k} w _ {j} m _ {j}, \xi^ {2} I _ {d}\right), \quad \mathrm {w h e r e} \quad \xi^ {2} = \frac {\left(\sigma^ {2} + \sqrt {(\sigma^ {2} - \lambda) ^ {2} + 4 \sigma^ {2} \tau}\right) ^ {2} - \lambda^ {2}}{4 \sigma^ {2}}.
$$
Hence, in all the simulations performed in this section, we let  $\nu^{j} = N(m_{j},\sigma^{2}I_{d})$  for some  $m_j\in \mathbb{R}^d$  and  $\sigma^2 >0$ . While this simulation setup allows for exact computations of the divergence measure  $\mathrm{KL}(\mu_{\lambda ,\tau}^{*},\mu_{t})$ , there are two primary limitations in our experimental design. First, the theoretical results in this paper are proved under boundedness assumptions on the marginal measures, while Gaussian distributions are unbounded. Second, our simulations do not cover the free-support discrete point clouds setup investigated in Section 4.1 concerning the approximate Sinkhorn oracle implementable via Monte Carlo sampling.
Implementation of Algorithm 1. When the marginals  $\nu^j$  are Gaussian measures (not necessarily isotropic) and when  $\psi_0^j = 0$ , then Sinkhorn updates admit a closed-form expression that result in quadratic Sinkhorn potentials  $\psi^j, \phi^j$  and Gaussian measure  $\mu_t$ . In particular, the iterates of Algorithm 1 can be written as
$$
\psi_ {t} ^ {j} (y) = \frac {1}{2} y ^ {\top} A _ {t} ^ {j} y - y ^ {\top} b _ {t} ^ {j} + \operatorname {c o n s t}, \quad \phi_ {t} ^ {j} (y) = \frac {1}{2} x ^ {\top} C _ {t} ^ {j} x - x ^ {\top} d _ {t} ^ {j} + \operatorname {c o n s t}, \quad \mu_ {t} = N \left(e _ {t}, \Sigma_ {t}\right) \tag {23}
$$
for some matrices  $A_{t}, C_{t}, \Sigma_{t} \in \mathbb{R}^{d \times d}$  and vectors  $b_{t}, d_{t}, e_{t} \in \mathbb{R}^{d}$  that can be computed using explicit recursive expressions (see, e.g., [34, 41]).
Implementation of Algorithm 2. We implement approximate Sinkhorn oracle (Definition 1) as follows. Our updates maintain the property that the potentials  $\psi^j$  and  $\phi^j$  are quadratic functions of the form described in Equation (23). At every iteration  $t$ , we replace each matrix  $A_t^j$  (for  $j = 1, \dots, k$ ) by performing the transformation  $A_t^j \mapsto A_t^j - N_t^j$ , where  $N_t^j$  is a random positive-definite matrix with trace equal to  $\varepsilon$ . In our simulations, each  $N_t^j$  is obtained by drawing an independent  $d \times d$  matrix  $U$  with i.i.d.  $N(0,1)$  entries and setting  $N_t^j = \varepsilon \cdot U^\top U / \mathrm{trace}(U^\top U)$ . The parameter  $\varepsilon$  controls the approximation error of the implemented approximate Sinkhorn oracle.
Simulation results. We now comment on the main findings in our numerical simulations.
In Figure 1, we consider the toy setup  $k = 1, \nu^{1} = N(0,1)$ . Figure 1a shows that undamped iterates explode when  $\tau < \lambda / 2$  (note the absence of the blue line due to the explosion of iterates when  $\lambda / \tau = 2.1$ ). Figure 1b investigates the critical case  $\tau \approx \lambda / 2$ , noticing a sharp phase transition in the convergence behavior. Figure 1c demonstrates that damping removes the bad behavior. Moreover, the damping factor suggested in our work yields the fastest convergence among the tested five different damping factor choices.
In Figure 2, we investigate an undamped inexact algorithm described above. As suggested by Theorem 2, the iterates converge up to some level governed by the accuracy parameter  $\varepsilon$  of the inexact Sinkhorn oracle.
<sup>1</sup>The code for reproducing the simulation results is available at https://github.com/TomasVaskevicius/doubly-entropic-barycenters.
In Figure 3, we perform the same simulations as in Figure 2, but this time with damping. We observe again that damping helps significantly in the critical case  $\tau \approx \lambda /2$ , yielding a much faster convergence rate. The second row in Figure 3 considers the overdamped case, where we observe slightly slower convergence than that with the optimal choice of the damping parameter.
In Figure 4, investigate the effect of different levels of damping in the noisy setup (Algorithm 2). First, observe that the undamped algorithm  $(\eta = 1)$  explodes in the noisy case with  $\tau = \lambda /2$ , in contrast to the zero-noise setup reported in Figure 1a. Moreover, we find that overdamping might have a negative effect on attained accuracy: larger damping factors lead to lower accuracy at convergence in noisy setups.
(a) Explosion for  $\lambda /\tau >2$
(b) Phase transition at  $\lambda /\tau = 2$
(c) Damping helps.
Figure 1: Simulations for Algorithm 1 with  $k = 1$ ,  $\nu^1 = N(0,1)$ .
Figure 2: Setup:  $k = 3$ ,  $w = (1/3, 1/3, 1/3)$ ,  $\nu^1 = \nu^2 = \nu^3 = N(0, I_{10})$ . The effect of noise for the inexact algorithm.
Figure 3: Setup:  $k = 3$ ,  $w = (1/3, 1/3, 1/3)$ ,  $\nu^1 = \nu^2 = \nu^3 = N(0, I_{10})$ . The effect of noise for the inexact algorithm. Second row overdamps.
Figure 4: Setup:  $k = 3$ ,  $w = (1/3, 1/3, 1/3)$ ,  $\nu^1 = \nu^2 = \nu^3 = N(0, I_{10})$ . The effect of damping for the inexact algorithm.