SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Nikita Gushchin
Skoltech*
n.gushchin@skoltech.ru
Alexander Kolesov
a.kolesov@skoltech.ru
Petr Mokrov
petr.mokrov@skoltech.ru
Polina Karpikova
polina.karpikova@skoltech.ru
Andrey Spiridonov
andrew.spiridonov@skoltech.ru
Evgeny Burnaev
AIRI†
e.burnaev@skoltech.ru
Alexander Korotin
a.korotin@skoltech.ru
Over the last several years, there has been significant progress in developing neural solvers for the Schrödinger Bridge (SB) problem and applying them to generative modelling. This new research field is justifiably fruitful as it is interconnected with the practically well-performing diffusion models and theoretically grounded entropic optimal transport (EOT). Still, the area lacks non-trivial tests allowing a researcher to understand how well the methods solve SB or its equivalent continuous EOT problem. We fill this gap and propose a novel way to create pairs of probability distributions for which the ground truth OT solution is known by the construction. Our methodology is generic and works for a wide range of OT formulations, in particular, it covers the EOT which is equivalent to SB (the main interest of our study). This development allows us to create continuous benchmark distributions with the known EOT and SB solutions on high-dimensional spaces such as spaces of images. As an illustration, we use these benchmark pairs to test how well existing neural EOT/SB solvers actually compute the EOT solution. Our code for constructing benchmark pairs under different setups is available at:
https://github.com/ngushchin/EntropicOTBenchmark
Diffusion models are a powerful tool to solve image synthesis [25, 45] and image-to-image translation [50, 47] tasks. Still, they suffer from the time-consuming inference which requires modeling thousands of diffusion steps. Recently, the Schrodinger Bridge (SB) has arisen as a promising framework to cope with this issue [15, 9, 54]. Informally, SB is a special diffusion which has rather straight trajectories and finite time horizon. Thus, it may require fewer discretization steps to infer the diffusion.
* Skolkovo Institute of Science and Technology
† Artificial Intelligence Research Institute
In addition to promising practical features, SB is known to have good and well-studied theoretical properties. Namely, it is equivalent [38] to the Entropic Optimal Transport problem (EOT, [13, 20]) about moving the mass of one probability distribution to the other in the most efficient way. This problem has gained a genuine interest in the machine learning community thanks to its nice sample complexity properties, convenient dual form and a wide range of applications [28, 6, 44].
Expectidely, recent neural EOT/SB solvers start showing promising performance in various tasks [15, 52, 9, 14, 23, 42]. However, it remains unclear to which extent this success is actually attributed to the fact that these methods properly solve EOT/SB problem rather than to a good choice of parameterization, regularization, tricks, etc. This ambiguity exists because of the lack of ways to evaluate the performance of solvers qualitatively in solving EOT/SB. Specifically, the class of continuous distributions with the analytically known EOT/SB solution is narrow (Gaussians [10, 41, 26, 7]) and these solutions have been obtained only recently. Hence, although papers in the field of neural EOT/SB frequently appear, we never know how well they actually solve EOT/SB.
Contributions. We develop a generic methodology for evaluating continuous EOT/SB solvers.
1. We propose a generic method to create continuous pairs of probability distributions with analytically known (by our construction) EOT solution between them (§3.1, §3.2).
2. We use log-sum-exp of quadratic functions (§3.3, §3.4) to construct pairs of distributions (§4) that we use as a benchmark with analytically-known EOT/SB solution for the quadratic cost.
3. We use these benchmark pairs to evaluate (§5) many popular neural EOT/SB solvers (§2) in high-dimensional spaces, including the space of  $64 \times 64$  celebrity faces.
In the field of neural OT, there already exist several benchmarks for the Wasserstein-2 [32], the Wasserstein-1 [31] and the Wasserstein-2 barycenter [31] OT tasks. Their benchmark construction methodologies work only for specific OT formulations and do not generalize to EOT which we study.
1 Background: Optimal Transport and Schrödinger Bridges Theory
We work in Euclidean space  $\mathcal{X} = \mathcal{Y} = \mathbb{R}^D$  equipped with the standard Euclidean norm  $\|\cdot\|$ . We use  $\mathcal{P}(\mathcal{X}) = \mathcal{P}(\mathcal{Y}) = \mathcal{P}(\mathbb{R}^D)$  to denote the sets of Borel probability distributions on  $\mathcal{X}, \mathcal{Y}$ , respectively.
Classic (Kantorovich) OT formulation [27, 53, 48]. For two distributions  $\mathbb{P}_0\in \mathcal{P}(\mathcal{X}),\mathbb{P}_1\in \mathcal{P}(\mathcal{Y})$  and a cost function  $c:\mathcal{X}\times \mathcal{Y}\to \mathbb{R}$ , consider the following problem (Fig. 1a):
$$
\mathrm {O T} _ {c} \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right) \stackrel {\text {d e f}} {=} \inf  _ {\pi \in \Pi \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right)} \int_ {\chi \times \mathcal {Y}} c (x, y) d \pi (x, y), \tag {1}
$$
where the optimization is performed over the set  $\Pi(\mathbb{P}_0, \mathbb{P}_1)$  of transport plans, i.e., joint distributions on  $\mathcal{X} \times \mathcal{Y}$  with marginals  $\mathbb{P}_0, \mathbb{P}_1$ , respectively. The set  $\Pi(\mathbb{P}_0, \mathbb{P}_1)$  is non-empty as it always contains the trivial plan  $\mathbb{P}_0 \times \mathbb{P}_1$ . With mild assumptions, a minimizer  $\pi^*$  of (1) exists and is called an  $OT$  plan. Typical examples of  $c$  are powers of Euclidean norms, i.e.,  $c(x, y) = \frac{1}{q} \|x - y\|^q$ ,  $q \geq 1$ .
(a) Classic OT formulation (1).
(b) Weak OT formulation (2).
Figure 1: Classic (Kantorovich's) and weak OT formulations.
Weak OT formulation [22, 2, 3]. Let  $C: \mathcal{X} \times \mathcal{P}(\mathcal{Y}) \to \mathbb{R} \cup \{+\infty\}$  be a weak cost which takes a point  $x \in \mathcal{X}$  and a distribution of  $y \in \mathcal{Y}$  as inputs. The weak OT cost between  $\mathbb{P}_0$ ,  $\mathbb{P}_1$  is (Fig. 1b)
$$
\operatorname {W O T} _ {C} \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right) \stackrel {\text {d e f}} {=} \inf  _ {\pi \in \Pi \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right)} \int_ {\mathcal {X}} C (x, \pi (\cdot | x)) d \pi_ {0} (x) = \inf  _ {\pi \in \Pi \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right)} \int_ {\mathcal {X}} C (x, \pi (\cdot | x)) d \mathbb {P} _ {0} (x), \tag {2}
$$
where  $\pi (\cdot |x)$  denotes the conditional distribution of  $y\in \mathcal{V}$  given  $x\in \mathcal{X}$  and  $\pi_0$  is the projection of  $\pi$  to  $\mathcal{X}$  which equals  $\mathbb{P}_0$  since  $\pi \in \Pi (\mathbb{P}_0,\mathbb{P}_1)$ . Weak OT formulation (2) generalizes classic OT (1): it suffices to pick  $C(x,\pi (\cdot |x)) = \int_{\mathcal{V}}c(x,y)d\pi (y|x)$  to obtain (1) from (2). A more general case of a
weak cost is  $C\big(x,\pi (\cdot |x)\big) = \int_{\mathcal{Y}}c(x,y)d\pi (y|x) + \epsilon \mathcal{R}\big(\pi (\cdot |x)\big)$ , where  $\epsilon >0$  and  $\mathcal{R}:\mathcal{P}(\mathcal{Y})\to \mathbb{R}$  is some functional (a.k.a. regularizer), e.g., variance [3, 35], kernel variance [34] or entropy [3]. With mild assumptions on the weak cost function  $C$ , an OT plan  $\pi^*$  in (2) exists. We say that the family of its conditional distributions  $\{\pi^{*}(\cdot |x)\}_{x\in \mathcal{X}}$  is the conditional OT plan.
Entropic OT formulation [13, 20]. It is common to consider entropy-based regularizers for (1):
$$
\left\{ \right.\begin{array}{l}\mathrm {E O T} _ {c, \epsilon} ^ {(1)} \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right)\\\mathrm {E O T} _ {c, \epsilon} ^ {(2)} \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right)\\\mathrm {E O T} _ {c, \epsilon} \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right)\end{array}\stackrel {\text {d e f}} {=} \min  _ {\pi \in \Pi \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right)} \int_ {\mathcal {X} \times \mathcal {Y}} c (x, y) \pi (x, y) + \left\{\begin{array}{l l}+ \epsilon \mathrm {K L} (\pi \| \mathbb {P} _ {0} \times \mathbb {P} _ {1}),&(3)\\- \epsilon H (\pi),&(4)\\- \epsilon \int_ {\mathcal {X}} H (\pi (\cdot | x)) d \mathbb {P} _ {0} (x).&(5)\end{array}\right.
$$
Here KL is the Kullback-Leibler divergence and  $H$  is the differential entropy, i.e., the minus KL divergence with the Lebesgue measure. Since  $\pi \in \Pi (\mathbb{P}_0,\mathbb{P}_1)$ , it holds that  $\mathrm{KL}(\pi \| \mathbb{P}_0\times \mathbb{P}_1) = H(\mathbb{P}_0) - \int_{\mathcal{X}}H\big(\pi (y|x)\big)d\mathbb{P}_0(x) = -H(\pi) + H(\mathbb{P}_0) + H(\mathbb{P}_1)$ , i.e., these formulations are equal up to an additive constant when  $\mathbb{P}_0\in \mathcal{P}_{ac}(\mathcal{X})$  and  $\mathbb{P}_1\in \mathcal{P}_{ac}(\mathcal{V})$  and have finite entropy. Here we introduce "ac" subscript to indicate the subset of absolutely continuous distributions. With mild assumptions on  $c,\mathbb{P}_0,\mathbb{P}_1$ , the minimizer  $\pi^*$  exists, it is unique and called the entropic OT plan. It is important to note that entropic OT (5) is a case of weak OT (2). Indeed, for the weak cost
$$
C _ {c, \epsilon} (x, \pi (\cdot | x)) \stackrel {\text {d e f}} {=} \int_ {\mathcal {Y}} c (x, y) d \pi (y | x) - \epsilon H \left(\pi (\cdot | x)\right), \tag {6}
$$
formulation (2) immediately turns to (5). This allows us to apply the theory of weak OT to EOT.
Dual OT formulation. There exists a wide range of dual formulations of OT [53, 48], WOT [2, 22] and EOT [20, 44]. We only recall the particular dual form for WOT from [3, 2] which serves as the main theoretical ingredient for our paper. For technical reasons, from now on we consider only  $\mathbb{P}_1\in \mathcal{P}_p(\mathcal{Y})\subset \mathcal{P}(\mathcal{Y})$  for some  $p\geq 1$ , where subscript "  $p$  " indicates distributions with a finite  $p$ -th moment. We also assume that the weak cost  $C:\mathcal{X}\times \mathcal{P}_p(\mathcal{Y})\to \mathbb{R}\cup \{+\infty \}$  is lower bounded, convex in the second argument and jointly lower-semicontinuous in  $\mathcal{X}\times \mathcal{P}_p(\mathcal{Y})$ . In this case, a minimizer  $\pi^{*}$  of WOT (2) exists [3, Theorem 3.2] and the following dual formulation holds [3, Eq. 3.3]:
$$
\mathrm {W O T} _ {C} \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right) = \sup  _ {f} \left\{\int_ {\mathcal {X}} f ^ {C} (x) d \mathbb {P} _ {0} (x) + \int_ {\mathcal {Y}} f (y) d \mathbb {P} _ {1} (y) \right\}, \tag {7}
$$
where  $f \in \mathcal{C}_p(\mathcal{Y}) \stackrel{\mathrm{def}}{=} \{f : \mathcal{Y} \to \mathbb{R} \text{ continuous s.t. } \exists \alpha, \beta \in \mathbb{R} : |f(\cdot)| \leq \alpha \| \cdot \|^p + \beta\}$  and  $f^C$  is the so-called weak  $C$ -transform of  $f$  which is defined by
$$
f ^ {C} (x) \stackrel {\text {d e f}} {=} \inf  _ {\nu \in \mathcal {P} _ {p} (\mathcal {Y})} \left\{C (x, \nu) - \int_ {\mathcal {Y}} f (y) d \nu (y) \right\}. \tag {8}
$$
Function  $f$  in (7) is typically called the dual variable or the Kantorovich potential.
SB problem with Wiener prior [38, 11]. Let  $\Omega$  be the space of  $\mathbb{R}^D$ -valued functions of time  $t \in [0,1]$  describing trajectories in  $\mathbb{R}^D$ , which start at time  $t = 0$  and end at time  $t = 1$ . We use  $\mathcal{P}(\Omega)$  to denote the set of probability distributions on  $\Omega$ , i.e., stochastic processes.
Consider two distributions  $\mathbb{P}_0\in \mathcal{P}_{2,ac}(\mathcal{X})$  and  $\mathbb{P}_1\in$ $\mathcal{P}_{2,ac}(\mathcal{Y})$  with finite entropy. Let  $\mathcal{F}(\mathbb{P}_0,\mathbb{P}_1)\subset \mathcal{P}(\Omega)$  be the subset of processes which have marginals  $\mathbb{P}_0$  and  $\mathbb{P}_1$  at times  $t = 0$  and  $t = 1$  , respectively. Let  $dW_{t}$  be the differential of the standard  $\mathbb{R}^D$  -valued Wiener process. Let  $W^{\epsilon}\in \mathcal{P}(\Omega)$  be the Wiener process with the variance  $\epsilon >0$  which starts at  $\mathbb{P}_0$  at time  $t = 0$  . It can be represented via the following stochastic differential equation (SDE):  $dX_{t} = \sqrt{\epsilon} dW_{t}$  with  $X_0\sim \mathbb{P}_0$
Figure 2: The bridge of Schrödinger.
The Schrödinger Bridge problem with the Wiener prior is the following:
$$
\inf  _ {T \in \mathcal {F} \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right)} \mathrm {K L} (T \| W ^ {\epsilon}). \tag {9}
$$
The inf is attained uniquely at some process  $T^{*}$  [38, Proposition 4.1]. This process turns out to be a diffusion process and can be (uniquely) represented as the following SDE:
$$
T ^ {*} \quad : \quad d X _ {t} = v ^ {*} \left(X _ {t}, t\right) d t + \sqrt {\epsilon} d W _ {t}, \tag {10}
$$
where  $v^{*}:\mathbb{R}^{D}\times [0,1]\to \mathbb{R}^{D}$  is its drift function which we call the optimal drift. Hence, in (9), one may consider only diffusion processes  $\subset \mathcal{F}(\mathbb{P}_0,\mathbb{P}_1)$  with the volatility  $\epsilon$  coinciding with the volatility of the Wiener prior  $W^{\epsilon}$ . In turn, solving SB can be viewed as finding the optimal drift  $v^{*}$ .
Link between SB and EOT problem. The process  $T^{*}$  solving SB (9) is related to the solution  $\pi^{*}$  of EOT problem (5) with the quadratic cost function  $c(x, y) = \frac{1}{2} \| x - y \|^2$ . We start with some notations. For a process  $T \in \mathcal{P}(\Omega)$ , denote the joint distribution at time moments  $t = 0, 1$  by  $\pi^T \in \mathcal{P}(\mathcal{X} \times \mathcal{Y})$ . Let  $T|_{x, y}$  be the distribution of  $T$  for  $t \in (0, 1)$  conditioned on  $T$ 's values  $x, y$  at  $t = 0, 1$ .
For the solution  $T^{*}$  of SB (9), it holds that  $\pi^{T^{*}} = \pi^{*}$ , where  $\pi^{*}$  is the EOT plan solving (5). Moreover,  $T_{|x,y}^{*} = W_{|x,y}^{\epsilon}$ , i.e., informally, the "inner" part of  $T^{*}$  matches that of the prior  $W^{\epsilon}$ .
Conditional process  $W_{|x,y}^{\epsilon}$  is well-known as the Brownian Bridge. Due to this, given  $x, y$ , simulating the trajectories of  $W_{|x,y}^{\epsilon}$  is rather straightforward. Thanks to this aspect, SB and EOT can be treated as nearly equivalent problems. Still EOT solution  $\pi^{*}$  does not directly yield the optimal drift  $v^{*}$ . However, it is known that the density  $\frac{d\pi^{*}(x,y)}{d(x,y)}$  of  $\pi^{*}$  has the specific form [38, Theorem 2.8], namely,  $\frac{d\pi^{*}(x,y)}{d(x,y)} = \widetilde{\varphi}^{*}(x)\mathcal{N}(y|x,\epsilon I)\varphi^{*}(y)$ , where functions  $\varphi^{*}, \widetilde{\varphi}^{*}: \mathbb{R}^{D} \to \mathbb{R}$  are called the Schrödinger potentials. From this equality one gets the expression for  $\varphi^{*}(\cdot)$  and the density of  $\pi^{*}(\cdot|x)$ :
$$
\frac {d \pi^ {*} (y | x)}{d y} \propto \mathcal {N} (y | x, \epsilon I) \varphi^ {*} (y) \quad \Longrightarrow \quad \varphi^ {*} (y) \propto \frac {d \pi^ {*} (y | x)}{d y} \cdot \left[ \mathcal {N} (y | x, \epsilon I) \right] ^ {- 1} \tag {11}
$$
up to multiplicative constants. One may recover the optimal drift  $v^{*}$  via [38, Proposition 4.1]
$$
v ^ {*} (x, t) = \epsilon \nabla \log \int_ {\mathbb {R} ^ {D}} \mathcal {N} (y | x, (1 - t) \epsilon I _ {D}) \varphi^ {*} (y) d y. \tag {12}
$$
Here the normalization constant vanishes when one computes  $\nabla \log (\cdot)$ . Thus, technically, knowing the (unnormized) density of  $\pi^{*}$ , one may recover the optimal drift  $v^{*}$  for SB (9).
2 Background: Solving Continuous OT and SB Problems
Although OT (2), EOT (5) and SB (9) problems are well-studied in theory, solving them in practice is challenging. Existing OT solvers are of two main types: discrete [44] and continuous [32]. Our benchmark is designed for continuous EOT solvers; discrete OT/EOT is out of the scope of the paper.
Continuous OT assumes that distributions  $\mathbb{P}_0$  and  $\mathbb{P}_1$  are continuous and accessible only via their random samples  $X = \{x_{1},\ldots ,x_{N}\} \sim \mathbb{P}_{0}$  and  $Y = \{y_{1},\dots ,y_{M}\} \sim \mathbb{P}_{1}$ . The goal is to recover an OT plan  $\pi^{*}$  between entire  $\mathbb{P}_0$  and  $\mathbb{P}_1$  but using only  $X$  and  $Y$ . Most continuous OT solvers do this via employing neural networks to implicitly learn the conditional distributions  $\widehat{\pi} (\cdot |x)\approx \pi^{*}(\cdot |x)$ . In turn, SB solvers learn the optimal drift  $\widehat{v}\approx v^{*}$  but it is anyway used to produce samples  $y\sim \widehat{\pi} (\cdot |x)$  via solving SDE  $dX_{t} = \widehat{v} (x,t)dt + \sqrt{\epsilon} dW_{t}$  starting from  $X_0 = x$  (sampled from  $\mathbb{P}_0$ ) at time  $t = 0$ .
After training on available samples  $X$  and  $Y$ , continuous solvers may produce  $y \sim \widehat{\pi}(\cdot | x_{\text{test}})$  for previously unseen samples  $x_{\text{test}} \sim \mathbb{P}_0$ . This is usually called the out-of-sample estimation. It allows applying continuous OT solver to generative modelling problems such as the image synthesis (noise-to-data) and translation (data-to-data). In both these cases,  $\mathbb{P}_1$  is a data distribution, and  $\mathbb{P}_0$  is either a noise (in synthesis) or some other data distribution (in translation). Many recent OT solvers achieve competitive performance in synthesis [12, 15, 46] and translation [35, 34] tasks.
Continuous OT/SB solvers are usually referred to as **neural OT/SB** because they employ neural networks. There exist a lot of neural OT solvers for classic OT (1) [46, 17, 56, 40, 30, 19], see also [32, 31] for surveys, weak OT (2) [34, 35, 1], entropic OT (5) [49, 14, 42] and SB (9) [52, 15, 9, 23]. Providing a concise but still explanatory overview of them is nearly impossible as the underlying principles of many of them are rather different and non-trivial. We list only EOT/SB solvers which are relevant to our benchmark and provide a brief summary of them in Table 1. In §5, we test all these solvers on our continuous benchmark distributions which we construct in subsequent §3.
Approaches to evaluate solvers. As seen from Table 1, each paper usually tests its solver on a restricted set of examples which rarely intersects with those from the other papers. In particular, some papers consider data→data tasks, while the others focus on noise→data. Due to this, there is
Solver	Underlying principle and parameterization	Evaluated as EOT/SB	Tested in generation (noise→data)	Tested in translation (data→data)
EOT solvers	LSOT [49, 21]	Solves classic dual EOT [20, §3.1] with 2 NNs. Learns 1 more NN for the barycentric projection.	X	MNIST (32x32)	MNIST→USPS(16x16), USPS→MNIST (16x16), SVHN→MNIST (3x32x32)
SCONES [14]	Combines LSOT's potentials with a score model for P1 to sample from π* (·|x) via Langevin dynamics.	Gaussians	X	CelebA Upscale (3x64x64)
NOT* [35]	Solves max-min reformulation of weak OT dual (7) with 2 NNs (transport map and potential).	* This is a generic neural solver for weak OT but it has not been tested with the entropic cost function.
EGNOT [42]	Employs energy-based modeling (EBM [37]) to solve weak EOT dual (7); non-minimax; 1NN.	Gaussians	X	Colored MNIST 2→3 (3x32x32)
SB solvers	ENOT [23]	Solves max-min reformulation of SB with 2 NNs (potential and SDE drift).	Gaussians	X	CelebA Upscale (3x64x64), Colored MNIST 2→3 (3x32x32)
MLE-SB [52]	Alternate solving of two Half Bridge (HB) problems. HB is solved via drift estimation with GP [55].	X	X	Single Cell data (D=5), Motion Capture (D=4)
DiffSB [15]	Iterative Mean-Matching Proportional Fitting 2 NNs for forward and backward SDE drifts	Gaussians	CelebA (32x32)	MNIST→EMNIST (32x32)
FB-SDE [9]	Likelihood training of SB 2 NNs for the ∇ log of Schrödinger potentials	X	MNIST (32x32), CelebA (3x32x32), CIFAR-10 (3x32x32)	X
Table 1: Table of existing continuous (neural) solvers for EOT/SB.
no clear understanding of the superiority of one solver over the other. Importantly, in many cases, the qualitative evaluation is done exclusively via the metrics of the downstream task. For example, [15, 9, 23, 14] consider image generation or translation tasks and test the quality of generated images via FID [24]. That is, they compare generated marginal distribution  $\widehat{\pi}_1$  with target  $\mathbb{P}_1$ . This allows to access the generative performance of solvers but gives no hint whether they actually learn the true EOT/SB solution. Works [14, 23, 42, 15] do a step toward opening the veil of secrecy and test their solvers in the Gaussian case. Unfortunately, it is rather trivial and may not be representative.
3 Constructing Benchmark Pairs for OT and SB: Theory
In this section, we present our theoretical results allowing us to construct pairs of distributions with the EOT/SB solution known by the construction. We provide proofs in Appendix A.
3.1 Generic Optimal Transport Benchmark Idea
For a given distribution  $\mathbb{P}_0\in \mathcal{P}(\mathcal{X})$ , we want to construct a distribution  $\mathbb{P}_1\in \mathcal{P}_p(\mathcal{Y})$  such that some OT plan  $\pi^{*}\in \Pi (\mathbb{P}_{0},\mathbb{P}_{1})$  for a given weak OT cost function  $C$  between them is known by the construction. That is,  $\pi_0^* = \mathbb{P}_0$ ,  $\pi_1^* = \mathbb{P}_1$  and  $\pi^{*}$  minimizes (2). In this case,  $(\mathbb{P}_0,\mathbb{P}_1)$  may be used as a benchmark pair with a known OT solution. Our following main theorem provides a way to do so.
Theorem 3.1 (Optimal transport benchmark constructor). Let  $\mathbb{P}_0\in \mathcal{P}(\mathcal{X})$  be a given distribution,  $f^{*}\in \mathcal{C}_{p}(\mathcal{Y})$  be a given function and  $C:\mathcal{X}\times \mathcal{P}_p(\mathcal{Y})\to \mathbb{R}$  be a given jointly lower semi-continuous, convex in the second argument and lower bounded weak cost. Let  $\pi^*\in \mathcal{P}(\mathcal{X}\times \mathcal{Y})$  be a distribution for which  $\pi_0^* = \mathbb{P}_0$  and for all  $x\in \mathcal{X}$  it holds that
$$
\pi^ {*} (\cdot | x) \in \operatorname * {a r g i n f} _ {\mu \in \mathcal {P} _ {p} (\mathcal {Y})} \left\{C (x, \mu) - \int_ {\mathcal {Y}} f ^ {*} (y) d \mu (y) \right\}. \tag {13}
$$
Let  $\mathbb{P}_1 \stackrel{\text{def}}{=} \pi_1^*$  be the second marginal of  $\pi^*$  and assume that  $\mathbb{P}_1 \in \mathcal{P}_p(\mathcal{Y})$ . Then  $\pi^*$  is an OT plan between  $\mathbb{P}_0$  and  $\mathbb{P}_1$  (it minimizes (2)) and  $f^*$  is an optimal dual potential (it maximizes (7)).
Thanks to our theorem, given a pair  $(\mathbb{P}_0,f^*)\in \mathcal{P}(\mathcal{X})\times \mathcal{C}_p(\mathcal{Y})$  of a distribution and a potential, one may produce a distribution  $\mathbb{P}_1$  for which an OT plan between them is known by the construction. This may be done by picking  $\pi^{*}\in \Pi (\mathbb{P})$  whose conditionals  $\pi^{*}(\cdot |x)$  minimize (13).
While our theorem works for rather general costs  $C$ , it may be non-trivial to compute a minimizer  $\pi^{*}(\cdot |x)$  in the weak  $C$ -transform (8), e.g., to sample from it or to estimate its density. Also, we note that our theorem states that  $\pi^{*}$  is optimal but does not claim that it is the unique OT plan. These aspects may complicate the usage of the theorem for constructing the benchmark pairs  $(\mathbb{P}_0, \mathbb{P}_1)$  for general costs  $C$ . Fortunately, both these issues vanish when we consider EOT, see below.
3.2 Entropic Optimal Transport Benchmark Idea
For  $C = C_{c,\epsilon}$  (6) with  $\epsilon > 0$ , the characterization of minimizers  $\pi^{*}(\cdot |x)$  in (13) is almost explicit.