SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left(\sqrt {2 \pi \epsilon}\right) ^ {D} \exp \left(\int_ {\mathcal {Y}} \frac {f ^ {*} (y)}{\epsilon} \mathcal {N} (y | x, \epsilon I) d y\right) \geq \left(\sqrt {2 \pi \epsilon}\right) ^ {D} \exp \left(\int_ {\mathcal {Y}} \frac {\beta + \alpha \| y \| ^ {2}}{\epsilon} \mathcal {N} (y | x, \epsilon I) d y\right) = \tag {24}
$$
$$
\left(\sqrt {2 \pi \epsilon}\right) ^ {D} \exp \left(\frac {\beta + \alpha \left(\| x \| ^ {2} + \epsilon D\right)}{\epsilon}\right), \tag {25}
$$
where in transition to line (24) we used the Jesnsen's inequality and  $\alpha, \beta \in \mathbb{R}$  are some constants for which  $f^{*}(\cdot) \geq \beta + \alpha \| \cdot \|^{2}$ . They exist since  $f^{*} \in \mathcal{C}_{2}(\mathcal{Y})$ . Indeed, there exist  $\tilde{\alpha}, \tilde{\beta}: |f^{*}(\cdot)| \leq \tilde{\beta} + \tilde{\alpha} \| \cdot \|^{2} \Rightarrow f^{*}(\cdot) \geq -\tilde{\beta} - \tilde{\alpha} \| \cdot \|^{2}$ , and we set  $\alpha = -\tilde{\alpha}, \beta = -\tilde{\beta}$ . In turn, line (25) uses the explicit formula for the second moment of  $\mathcal{N}(y|x,\epsilon I)$ . We use (25) to upper bound the first term in (23):
$$
\int_ {\mathcal {X}} \left[ - \epsilon \log Z _ {x} \right] d \mathbb {P} _ {0} (x) \leq \int_ {\mathcal {X}} \left[ \right. - \epsilon \log \left( \right.\left\{\left(\sqrt {2 \pi \epsilon}\right) ^ {D} \exp \left(\frac {\beta + \alpha \| x \| ^ {2} + \alpha \epsilon D}{\epsilon}\right)\right\}\left. \right] d \mathbb {P} _ {0} (x) =
$$
$$
- \frac {\epsilon D}{2} \log (2 \pi \epsilon) - \beta - \alpha \epsilon D - \alpha \int_ {\mathcal {X}} \| x \| ^ {2} d \mathbb {P} _ {0} (x).
$$
It remains to note that the last value is finite, since  $\mathbb{P}_0\in \mathcal{P}_2(\mathcal{X})$  by the assumption.
Proof of Corollary 3.4. We note that  $\frac{d\pi^{*}(y|x)}{dy} \propto \exp \left(\frac{f^{*}(y) - \frac{1}{2}\|x - y\|^{2}}{\epsilon}\right)$ . Therefore,
$$
\exp \left(\frac {f ^ {*} (y)}{\epsilon}\right) \propto \frac {d \pi^ {*} (y | x)}{d y} \exp \left(\frac {1}{2 \epsilon} \| x - y \| ^ {2}\right) \propto \frac {d \pi^ {*} (y | x)}{d y} \cdot \left[ \mathcal {N} (y | x, \epsilon I) \right] ^ {- 1}. \tag {26}
$$
By comparing (26) with (11), we see that  $\exp \left(\frac{f^{*}(y)}{\epsilon}\right)$  indeed coincides with the Schrödinger potential  $\phi^{*}(y)$ . Formula (12) for the optimal drift follows from [38, Proposition 4.1] $^3$ .
Proof of Corollary 3.5. First, we prove that constructed  $\mathbb{P}_1 \stackrel{\mathrm{def}}{=} \pi_1^*$  actually has finite entropy. This is needed to ensure that the assumptions of [38, Proposition 4.1]. This proposition provides the formula for the optimal drift (12) via the Schrödinger potential. We write
$$
\begin{array}{l} 0 \leq \operatorname {K L} \left(\pi_ {1} ^ {*} \| \mathcal {N} (\cdot | 0, I)\right) = - H \left(\pi_ {1} ^ {*}\right) - \int_ {\mathcal {Y}} \log \mathcal {N} (y | 0, I) d \pi_ {1} ^ {*} (y) = \\ - H \left(\pi_ {1} ^ {*}\right) + \frac {D}{2} \log (2 \pi) + \frac {1}{2} \int_ {\mathcal {Y}} \| y \| ^ {2} d \pi_ {1} ^ {*} (y). \tag {27} \\ \end{array}
$$
From our Proposition 3.3 it follows that  $\mathbb{P}_1 = \pi_1^*$  has finite second moment. Hence, the latter constant in (27) is finite. Therefore,  $H(\pi_1^*)$  is upper bounded. To lower bound  $H(\pi_1^*)$ , recall that each  $\pi^{*}(\cdot |x)$  is a mixture of  $N$  Gaussians (Proposition 3.3) with  $(x$ -independent) covariances  $\Sigma_{n}$ . Thus, its density  $\frac{d\pi^{*}(y|x)}{dy}$  is upper bounded by  $\xi \stackrel{\mathrm{def}}{=} \max_n \left[(2\pi)^{-D/2}\right] (\det \Sigma_n)^{-1/2} > 0$  which also means that
$$
\frac {d \pi_ {1} ^ {*} (y)}{d y} = \int_ {\mathcal {X}} \frac {d \pi^ {*} (y | x)}{d y} d \pi_ {0} ^ {*} (x) \leq \int_ {\mathcal {X}} \xi d \pi_ {0} ^ {*} (x) \leq \xi .
$$
$$
H \left(\pi_ {1} ^ {*}\right) = - \int \log \frac {d \pi_ {1} ^ {*} (y)}{d y} d \pi_ {1} ^ {*} (y) \geq - \int \log \xi d \pi_ {1} ^ {*} (y) = - \log \xi , \tag {28}
$$
i.e.,  $H(\pi_1^*)$  is lower-bounded as well.
Having in mind our previous Corollary, we just substitute  $\exp \left(\frac{f^{*}(y)}{\epsilon}\right)$  of LSE (15) potential  $f^{*}$  as the Schrödinger potential  $\phi^{*}(y)$  to (12). We derive
$$
v ^ {*} (x, t) = \epsilon \nabla \log \int_ {\mathbb {R} ^ {D}} \mathcal {N} (y | x, (1 - t) \epsilon I) \varphi^ {*} (y) d y =
$$
$$
\epsilon \nabla \log \int_ {\mathbb {R} ^ {D}} \mathcal {N} (y | x, (1 - t) \epsilon I) \exp \left(\frac {f ^ {*} (y)}{\epsilon}\right) d y =
$$
$$
\epsilon \nabla \log \int_ {\mathbb {R} ^ {D}} \mathcal {N} (y | x, (1 - t) \epsilon I) \exp (\frac {\epsilon \log \sum_ {n = 1} ^ {N} w _ {n} \mathcal {Q} (y | b _ {n} , \epsilon^ {- 1} A _ {n})}{\epsilon}) d y =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \int_ {\mathbb {R} ^ {D}} \mathcal {N} (y | x, (1 - t) \epsilon I) \mathcal {Q} (y | b _ {n}, \epsilon^ {- 1} A _ {n})) d y =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \int_ {\mathbb {R} ^ {D}} (2 \pi \epsilon (1 - t)) ^ {- \frac {D}{2}} \exp (- (y - x) ^ {T} \frac {I}{2 \epsilon (1 - t)} (y - x)) \mathcal {Q} (y | b _ {n}, \epsilon^ {- 1} A _ {n}) d y =
$$
3The authors of [38] consider SB with the reversible Wiener prior  $R$ , i.e., the standard Brownian motion starting at the Lebesgue measure. They deal with  $\inf_{T\in \mathcal{F}(\mathbb{P}_0,\mathbb{P}_1)}\mathrm{KL}\left(T\| R\right)$  which matches (up to an additive constant) our formulation (9) for  $\epsilon = 1$ . Indeed, using the measure disintegration theorem, one can derive  $\mathrm{KL}\left(T\| R\right) = -H\left(\mathbb{P}_0\right) + \mathrm{KL}\left(T\| W^{\epsilon}\right)$ . For other  $\epsilon >0$ , the analogous equivalence holds true.
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \int_ {\mathbb {R} ^ {D}} \exp (- (y - x) ^ {T} \frac {I}{2 \epsilon (1 - t)} (y - x)) \mathcal {Q} (y | b _ {n}, \epsilon^ {- 1} A _ {n}) d y +
$$
$$
\underbrace{\epsilon\nabla\log\left(\left(2\pi\epsilon(1 - t)\right)^{-\frac{D}{2}}\right)}_{= 0} =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \int_ {\mathbb {R} ^ {D}} \exp (- (y - x) ^ {T} \frac {I}{2 \epsilon (1 - t)} (y - x)) \exp (- (y - b _ {n}) ^ {T} \frac {A _ {n}}{2 \epsilon} (y - b _ {n})) d y =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \int_ {\mathbb {R} ^ {D}} \exp \left(- \frac {1}{2 (1 - t)} \left\{\left(y - x\right) ^ {T} \frac {I}{\epsilon} (y - x) + \left(y - b _ {n}\right) ^ {T} \frac {\overbrace {(1 - t) A _ {n}} ^ {A _ {n} ^ {t}} (\mathbf {y} - b _ {n}) \right\}\right) d y
$$
Next, we use (22) but with  $A_{n}^{t}$  instead of  $A_{n}$  and  $\Sigma_{n}^{t}$  instead of  $\Sigma_{n}$ . Also, we denote  $\mu_{n}^{t} = (A_{n}^{t} + I)^{-1}(A_{n}^{t}b_{n} + x)$ :
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \int_ {\mathbb {R} ^ {D}} \exp \left(- \frac {1}{2 (1 - t)} \left\{\left(y - x\right) ^ {T} \frac {I}{\epsilon} (y - x) + \left(y - b _ {n}\right) ^ {T} \overbrace {\frac {(1 - t) A _ {n}}{\epsilon}} ^ {A _ {n} ^ {t}} (y - b _ {n}) \right\}\right) d y =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \int_ {\mathbb {R} ^ {D}} \exp \left(- \frac {1}{2 (1 - t)} \left\{\left(y - \mu_ {n} ^ {t}\right) ^ {T} \left(\Sigma_ {n} ^ {t}\right) ^ {- 1} \left(y - \mu_ {n} ^ {t}\right) + \right. \right.
$$
$$
\left. \left(x - b _ {n}\right) ^ {T} \left(\frac {I}{\epsilon} - \frac {\Sigma_ {n} ^ {t}}{\epsilon^ {2}}\right) \left(x - b _ {n}\right) \right\} \Bigg) d y =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} \left\{w _ {n} \exp \left(- \frac {1}{2} (x - b _ {n}) ^ {T} \frac {\epsilon I - \Sigma_ {n} ^ {t}}{\epsilon^ {2} (1 - t)} (x - b _ {n})\right) \right.
$$
$$
\int_ {\mathbb {R} ^ {D}} \exp \left(- \frac {1}{2} (y - \mu_ {n} ^ {t}) ^ {T} \frac {\left(\Sigma_ {n} ^ {t}\right) ^ {- 1}}{(1 - t)} (y - \mu_ {n} ^ {t})\right) d y \Bigg \} =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \mathcal {Q} \big (x \big | b _ {n}, \frac {\epsilon I - \Sigma_ {n} ^ {t}}{\epsilon^ {2} (1 - t)} \big) \int_ {\mathbb {R} ^ {D}} \exp \big (- \frac {1}{2} (y - \mu_ {n} ^ {t}) ^ {T} \frac {\left(\Sigma_ {n} ^ {t}\right) ^ {- 1}}{(1 - t)} (y - \mu_ {n} ^ {t}) \big) d y =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \mathcal {Q} (x | b _ {n}, \frac {\epsilon I - \Sigma_ {n} ^ {t}}{\epsilon^ {2} (1 - t)}) \int_ {\mathbb {R} ^ {D}} (2 \pi) ^ {\frac {D}{2}} \det  ((1 - t) \Sigma_ {n} ^ {t}) ^ {\frac {1}{2}} \mathcal {N} (y | \mu_ {n} ^ {t}, (1 - t) \Sigma_ {n} ^ {t}) d y =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \mathcal {Q} \big (x \big | b _ {n}, \frac {\epsilon I - \Sigma_ {n} ^ {t}}{\epsilon^ {2} (1 - t)} \big) (2 \pi (1 - t)) ^ {\frac {D}{2}} \det  (\Sigma_ {n} ^ {t}) ^ {\frac {1}{2}} \underbrace {\int_ {\mathbb {R} ^ {D}} \mathcal {N} (y | \mu_ {n} ^ {t} , (1 - t) \Sigma_ {n} ^ {t}) d y} _ {= 1} =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \mathcal {Q} \left(x \mid b _ {n}, \frac {\epsilon I - \Sigma_ {n} ^ {t}}{\epsilon^ {2} (1 - t)}\right) \left(2 \pi (1 - t)\right) ^ {\frac {D}{2}} \det  \left(\Sigma_ {n} ^ {t}\right) ^ {\frac {1}{2}} =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \mathcal {Q} \left(x \mid b _ {n}, \frac {\epsilon I - \Sigma_ {n} ^ {t}}{\epsilon^ {2} (1 - t)}\right) \det  \left(\Sigma_ {n} ^ {t}\right) ^ {\frac {1}{2}} + \underbrace {\epsilon \nabla \log \left((2 \pi (1 - t)) ^ {\frac {D}{2}}\right)} _ {= 0} =
$$
$$
\epsilon \nabla \log \sum_ {n = 1} ^ {N} w _ {n} \sqrt {\det  (\Sigma_ {n} ^ {t})} \mathcal {Q} \left(x \mid b _ {n}, \frac {\epsilon I - \Sigma_ {n} ^ {t}}{\epsilon^ {2} (1 - t)}\right),
$$
B Mixtures Benchmark Pairs: Details and Results
Parameters for constructing benchmark pairs. In our benchmark pairs, we choose all their hyperparameters manually to make sure the constructed distributions  $\mathbb{P}_0, \mathbb{P}_1$  are visually pleasant and
distinguishable. As  $\mathbb{P}_0$ , we always use the centered Gaussian whose covariance matrix is  $0.25I$ . We use LSE function (15) with  $N = 5$  for constructing the distribution  $\mathbb{P}_1$ . In each setup, all  $A_{n}$  are the same and given in Table 3. We pick  $w_{n}$  such that  $\gamma_{n} = \frac{1}{5}\mathcal{N}(x|b_{n},(\frac{1}{\epsilon} I - \frac{1}{\epsilon^{2}}\Sigma_{n})^{-1})$ . We sample  $b_{n}$  randomly from a uniform distribution on a sphere with the radius  $R = 5$ .
D=2	D=16	D=64	D=128
ε=0.1	1/16I	1/16I	1/16I	1/16I
ε=1	1/16I	1/16I	1/16I	1/16I
ε=10	9/40I	1/100I	1/100I	1/100I
Evaluation details. For computing  $\mathbb{BW}_2^2$ -UVP  $(\widehat{\pi}_1,\mathbb{P}_1)$ , we use  $10^{5}$  random samples from  $\mathbb{P}_1$  and  $10^{5}$  random samples from learned distribution  $\widehat{\pi}_1$ . For computing  $\mathrm{cBW}_2^2$ -UVP  $(\widehat{\pi},\pi^{*})$ , we use the hold-out test set containing 1000 samples  $x\sim \mathbb{P}_0$ . We compute the expectation and covariance matrices of  $\pi^{*}(\cdot |x)$  analytically (Proposition 3.3) and we estimate the expectation and covariance matrix of  $\widehat{\pi} (\cdot |x)$  by using  $10^{3}$  samples. We present results of evaluation in Table 4 and Table 5.
We present an additional trivial baseline for the conditional metric  $\mathrm{cB}\mathbb{W}_2^2$ -UVP  $(\widehat{\pi},\pi^{*})$ , which is given by the independent plan  $\mathbb{P}_0\times \mathbb{P}_1$ . We compare other methods with this baseline in Table 5.
Table 3: Matrices  $A_{n}$  that we use to construct our mixtures benchmark pairs.
ε=0.1	ε=1	ε=10
D=2	D=16	D=64	D=128	D=2	D=16	D=64	D=128	D=2	D=16	D=64	D=128
[LSOT]	-	-	-	-	-	-	-	-	-	-	-	-
[SCONES]	-	-	-	-	1.06	4.24	6.67	11.54	1.11	2.98	1.33	7.89
[NOT]	0.016	0.63	1.53	2.62	0.08	1.13	1.62	2.62	0.225	2.603	1.872	6.12
[ElgNOT]	0.09	0.31	0.88	0.22	0.46	0.3	0.85	0.12	0.077	0.02	0.15	0.23
[ENOT]	0.2	2.9	1.8	1.4	0.22	0.4	7.8	29	1.2	2	18.9	28
[MLE-SB]	0.01	0.14	0.97	2.08	0.005	0.09	0.56	1.46	0.01	1.02	6.65	23.4
[DiffSB]	2.88	2.81	153.22	232.67	0.87	0.99	1.12	1.56	-	-	-	-
[FB-SDE-A]	2.37	2.55	68.19	27.11	0.6	0.63	0.65	0.71	-	-	-	-
[FB-SDE-J]	0.03	0.05	0.25	2.96	0.07	0.13	1.52	0.48	-	-	-	-
Table 4: Comparisons of  $\mathbf{BW}_2^2$ -UVP  $\downarrow$  (\%) between the target  $\mathbb{P}_1$  and learned marginal  $\pi_1$ . Colors indicate the metric value:  $\mathbf{BW}_2^2$ -UVP  $\leq 0.5$ ,  $\mathbf{BW}_2^2$ -UVP  $\in (0.5, 1]$ ,  $\mathbf{BW}_2^2$ -UVP  $> 1.0$ .
ε=0.1	ε=1	ε=10
D=2	D=16	D=64	D=128	D=2	D=16	D=64	D=128	D=2	D=16	D=64	D=128
[LSOT]	-	-	-	-	-	-	-	-	-	-	-	-
[SCONES]	-	-	-	-	34.88	71.34	59.12	136.44	32.9	50.84	60.44	52.11
[NOT]	1.94	13.67	11.74	11.4	4.77	23.27	41.75	26.56	2.86	4.57	3.41	6.56
[EqNOT]	129.8	75.2	60.4	43.2	80.4	74.4	63.8	53.2	4.14	2.64	2.36	1.31
[ENOT]	3.64	22	13.6	12.6	1.04	9.4	21.6	48	1.4	2.4	19.6	30
[MLE-SB]	4.57	16.12	16.1	17.81	4.13	9.08	18.05	15.226	1.61	1.27	3.9	12.9
[DiffSB]	73.54	59.7	1386.4	1683.6	33.76	70.86	53.42	156.46	-	-	-	-
[FB-SDE-A]	86.4	53.2	1156.82	1566.44	30.62	63.48	34.84	131.72	-	-	-	-
[FB-SDE-J]	51.34	89.16	119.32	173.96	29.34	69.2	155.14	177.52	-	-	-	-
Independent	166.0	152.0	126.0	110.0	86.0	80.0	72.0	60.0	4.2	2.52	2.26	2.4
Table 5: Comparisons of  $\mathrm{{cB}}{\mathbb{W}}_{2}^{2}$  -UVP  $\downarrow  \left( \% \right)$  between the optimal plan  ${\pi }^{ * }$  and the learned plan  $\widehat{\pi }$  . Colors indicate the ratio of the metric to the independent baseline metric: ratio  $\leq  {0.2}$  ,ratio  $\in  \left( {{0.2},{0.5}}\right)$  ,ratio  $> {0.5}$  .
Colors for the Table 2. To assign a color for the metric  $\mathsf{BW}_2^2$ -UVP and  $\mathsf{cBW}_2^2$ -UVP for each  $\epsilon$  in the Table 2, we use the following rule: we assign the rank 1 if a method's metric for a given dimension  $D$  has the color green, the rank 2 if a method's metric  $\mathsf{BW}_2^2$ -UVP has the color orange and the rank 3 if a method's metric  $\mathsf{BW}_2^2$ -UVP has the color red. To get the average rank, we take the mean of 4 ranks obtained for each dimension  $D$  and round it (1.5 and 2.5 are rounded to 1 and 2 respectively).
Extra qualitative results of EOT/SB solvers. In Figure 5 and Figure 6, we present the additional qualitative comparison of solvers on our mixtures benchmark pairs in  $D = 16$  with  $\epsilon \in \{0.1, 10\}$ . The figures are designed similarly to Figure 3 for  $(D, \epsilon) = (16, 1)$  in the main text. Note that case  $\epsilon = 10$  (Figure 6) is extremely challenging; only [EgNOT] provides more-or-less reasonable results.
Figure 5: Qualitative results of EOT/SB solvers on our mixtures benchmark pair with  $(D,\epsilon) = (16,0.1)$ . The distributions are visualized using 2 PCA components of target distr.  $\mathbb{P}_1$ .
Figure 6: Qualitative results of EOT/SB solvers on our mixtures benchmark pair with  $(D, \epsilon) = (16, 10)$ . The distributions are visualized using 2 PCA components of target distr.  $\mathbb{P}_1$ .
Computational complexity. Sampling from  $\mathbb{P}_0$  is lightspeed as it is just sampling a Normal noise. Sampling from  $\mathbb{P}_1$  is also fast, as it is the Gaussian mixture (Proposition 3.3).
C Images Benchmark Pairs: Details and Results
Parameters for constructing image benchmark pairs. We fix  $N = 100$  random samples from  $\mathbb{P}_0$  for  $b_{n}$  and choose all  $A_{n}\equiv I$ . We use  $w_{n}$  such that  $\gamma_{n} = \frac{1}{100}\mathcal{N}(x|b_n,(\frac{1}{\epsilon} I - \frac{1}{\epsilon^2}\Sigma_n)^{-1})$ .
GLOW details. We use the code from the repository with the default parameters:
https://github.com/rosinality/glow-pytorch
After training, the latent variable  $z$  is sampled from  $N(0,\sigma^2 I)$  with  $\sigma^2 = 0.49$  for image generation. That is, the image distribution  $\mathbb{P}_0$  is produced by the mapping  $z\sim N(0,\sigma^2 I)$  to the image space with the learned normalizing flow  $G$ , i.e.,  $\mathbb{P}_0\stackrel {\mathrm{def}}{=}G\sharp \mathcal{N}(\cdot |\sigma^2 I)$  in our construction.
MCMC in the latent space of the normalizing flow. We test EOT/SB solvers in  $\mathbb{P}_1\to \mathbb{P}_0$  direction, i.e., recovering  $\pi^{*}(x|y)$  and generating clean samples  $x$  from noised  $y$ . Unfortunately, the reverse conditional OT plans  $\pi^{*}(x|y)$  are not as tractable as  $\pi^{*}(y|x)$ . However, we note that
$$
\frac {d \pi^ {*} (x | y)}{d y} \propto \frac {d \pi^ {*} (y | x)}{d y} \frac {d \mathbb {P} _ {0} (x)}{d x}, \tag {29}
$$
i.e., the density of  $\pi^{*}(\cdot |y)$  it known up to the normalizing constant. Recall that here  $\mathbb{P}_0$  is constructed using the normalizing flow and  $\pi^{*}(\cdot |x)$  is a Gaussian mixture (Proposition 3.3), i.e., we indeed know the values of both terms. Therefore, one may use the well-celebrated Langevin dynamics to sample from  $\pi^{*}(y|x)$ . Unfortunately, we found that such sampling in the image space is rather slow.
To overcome this issue, we employ the Langevin sampling in the latent space of the normalizing flow. It is possible since the normalizing flow is a bijection between the space of images and the latent space. We use the standard notation  $z$  for the latent variable and  $G: \mathbb{R}^D \to \mathbb{R}^D$  for the normalizing flow, i.e.,  $x = G(z) \sim \mathbb{P}_0$  for  $z \sim p(z) \stackrel{\mathrm{def}}{=} \mathcal{N}(z|0, \sigma^2 I)$ . In this case, we have
$$
\begin{array}{l} \frac {d \pi^ {*} (z | y)}{d z} = \frac {d \pi^ {*} (x | y)}{d x} | \det  J _ {G ^ {- 1}} (x) | \propto \frac {d \pi^ {*} (y | x)}{d y} \frac {d \mathbb {P} _ {0} (x)}{d x} | \det  J _ {G ^ {- 1}} (x) | = \\ \frac {d \pi^ {*} (y | G (z))}{d x} \underbrace {\frac {d \mathbb {P} _ {0} (x)}{d x} | \det  J _ {G ^ {- 1}} (x) |} _ {p (z)} = \frac {d \pi^ {*} (y | G (z))}{d y} p (z), \\ \end{array}
$$
and we can derive the score function  $\nabla_{z}\log \frac{d\pi^{*}(z|y)}{dz}$  which is needed for the Langevin dynamic as
$$
\nabla_ {z} \log \frac {d \pi^ {*} (z | y)}{d z} = \nabla_ {z} \frac {d \pi^ {*} (y | G (z))}{d y} + \nabla_ {z} \log p (z). \tag {30}
$$
Hence, instead of doing non-trivial Langevin in the data space with  $\nabla_{x}\frac{d\pi^{*}(x|y)}{dx}$ , one may equivalently do the sampling in the latent space by using the score (30) and then get  $x = G(z)$ . We empirically found this approach works much better, presumably due to the fact that (30) is just the score of the Normal distribution which is slightly adjusted with the information coming from  $\pi^{*}(y|G(z))$ .
For sampling, we employ the Metropolis-adjusted Langevin algorithm with the time steps  $10^{-3}, 10^{-4}$  and  $10^{-5}$  for  $\epsilon = 10$ ,  $\epsilon = 1$  and  $\epsilon = 0.1$ , respectively. It provides the theoretical guarantees that the constructed Markov chain  $z_{1}, z_{2}, \ldots, \ldots$  converges to the distribution  $\frac{d\pi^{*}(z|y)}{dz}$ . For initializing the Markov chain, we sample a pair  $(x,y) \sim \pi^{*}$  and use  $z = G^{-1}(x)$  as the initial state for the Langevin sampling to get new samples from  $\pi^{*}(\cdot | y)$ . This trick allows for improving the stability of sampling and the convergence speed since it provides a good starting point. We use  $N = 200$  steps for all the setups for the Metropolis-adjusted Langevin algorithm.
In Figures 7 and 8, we provide additional examples of the samples from the ground truth plan  $\pi^{*}$ .
Metric 1. For each  $\epsilon = 0.1, 1, 10$  we prepare a test set with  $10^{4}$  samples from  $\mathbb{P}_0$ . We use this set to calculate the FID [24] metric between the ground truth distribution  $\mathbb{P}_0$  and the model's marginal distribution  $\pi_1$  to estimate how well the model restores the target distribution. This allows us access to the generative performance of solvers, i.e., the quality of generated images and matching the target distribution. However, this metric does not assess the accuracy of the recovered EOT plan.
Metric 2. For each  $\epsilon = 0.1, 1, 10$ , we prepare a test set containing 100 "noised" samples  $y \sim \mathbb{P}_1$  and 5K samples  $x \sim \pi^*(\cdot | y)$  for each "noised" sample  $y$ , i.e.,  $5\mathrm{K} \times 100$  images for each  $\epsilon$  in consideration. We propose to compute conditional FID to evaluate the difference between the conditional plans  $\pi^*(\cdot | y)$  and  $\widehat{\pi}(\cdot | y)$ . That is, for each  $y$  we compute FID between  $\pi^*(\cdot | y)$  and  $\widehat{\pi}(\cdot | y)$ , and then average the result for all test  $y$ . Clearly, such an evaluation is approximately  $100 \times$  times more consuming than computing the base FID. However, it allows us to fairly assess the quality of the recovered EOT solution, and we recommend this metric as the main for future EOT/SB studies.
In Tables 6, 7, we present the evaluation results for [ENOT] [23]. We again emphasize that, to the best of our knowledge, there is no scalable data  $\rightarrow$  data EOT/SB solver to compare against. Hence, we report the results as-is for future methods to be able to compare with them as the baseline.
Computational complexity. Sampling  $x \sim \mathbb{P}_0$  is just applying the trained GLOW neural network to noise vectors  $z \sim \mathcal{N}(\cdot | 0, \sigma^2 I)$ . Sampling  $y \sim \mathbb{P}_1$  (or  $y|x$ ) takes comparable time, as it is just extra
sampling from the Gaussian mixture with  $x$ -dependent parameters (Proposition 3.3). In turn, as we noted above, sampling  $x|y$  requires using the Langevin dynamic and takes considerable time. To obtain 3 test sets of 5K samples  $y \sim \pi^{*}(\cdot|x)$  per each of 100 samples  $x \sim \mathbb{P}_0$ , we employed  $8 \times \mathrm{A100}$  GPUs. This generation of test datasets took approximately 1 week.
Figure 7: Ground truth samples  $x \sim \pi^{*}(\cdot | y)$  on images benchmark pairs.
Figure 8: Ground truth samples  $y \sim \pi^{*}(\cdot | x)$  on images benchmark pairs.
ε	0.1	1	10
FID	5.99	3.21	4.9
Table 6: Test FID of [ENOT] on our images benchmark pairs.
€	0.1	1	10
cFID	40.5	19.8	14.47
Table 7: Test conditional FID of  $\left\lfloor  \mathbf{{ENOT}}\right\rceil$  on our images benchmark pairs
D Details of EOT/SB Solvers
D.1 Mixtures Benchmark Pairs
[LSOT] [49]. We use the part of the code of [SCONES] solver from the authors' repository
https://github.com/mdnls/scones-synthetic/blob/main/cpat.py
corresponding to learning dual OT potentials blob/main/cpat.py and the barycentric projection blob/main/bproj.py in the Gaussian case with configuration blob/main/config.py.
[SCONES] [14]. We use the aforementioned official code for training of dual OT potentials. We employ sklearn.mixture.GaussianMixture with 20 components to approximate the score of the target distribution. For the rest, we employ their configuration blob/main/config.py with batch size=1024 and the learning rate for Langevin sampling is  $5 \cdot 10^{-4}$ .
$\lfloor \mathbf{NOT}\rfloor$  This algorithm [35, Algorithm 1] is a generic algorithm for weak OT. It works for transport costs  $C(x,\pi (\cdot |x))$  which are straightforward to estimate by using samples of  $\pi (\cdot |x)$ . Entropic cost  $C_{c,\epsilon}$  (6) does not fit this requirement, as it is not easy to estim know the density of  $\pi (\cdot |x)$ . Thus, the authors of  $\lfloor \mathbf{NO}\rfloor$  a minor modification to their algorithm. As the base
lr (potential and transport map)	Tsteps	σz
1e-4	99	1.0
for the mixture benchmark pairs experiment.
https://github.com/iamalexkorotin/NeuralOptimalTransport
Instead of the multi-layer perceptron generator, we take a conditional normalizing flow with RealNVP architecture with context-dependent latent normal distribution. This enables the access to the density of  $\pi (\cdot |x)$  and allows applying [NOT] algorithm to EOT. Our reimplementation is available at
https://github.com/Penchekrak/FlowNOT
Due to the decreased expressivity of RealNVP compared to MLP from  $\lfloor \mathbf{NOT}\rfloor$ , we do more optimization steps for the transport map before updating potential as well as larger parameter count compared to the original solver implementation for a similar task. We use the same set of hyperparameters across all experiments with different  $(\epsilon, D)$ . The hyperparameters are summarized in Table 8.
[ElgNOT] [42] We use the official code for [EgNOT] from
https://github.com/PetrMokrov/Energy-guided-Entropic-OT.