SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In §B.1, we show that entropic OT can be reformulated as a maximin problem. This is a technical intermediate result needed to derive our main maximin reformulation of SB (Theorem 4.1). More precisely, in §B.2, we show that these maximin problems for entropic OT and SB are actually equivalent. By using this observation and related facts, in §B.3, we prove our Theorems 4.1 and 4.3.
B.1 Relaxation of entropic OT
To begin with, we recall some facts regarding EOT and SB. Recall the definition of EOT (2):
$$
\inf  _ {\pi \in \Pi \left(\mathbb {P} _ {0}, \mathbb {P} _ {1}\right)} \int_ {\mathcal {X} \times \mathcal {Y}} \frac {\| x - y \| ^ {2}}{2} d \pi (x, y) - \epsilon H (\pi). \tag {20}
$$
Henceforth, we assume that  $\mathbb{P}_0$  and  $\mathbb{P}_1$  are absolutely continuous. The situation when  $\mathbb{P}_0$  or  $\mathbb{P}_1$  is not absolutely continuous is not of any practical interest: there is no  $\pi \in \Pi(\mathbb{P}_0, \mathbb{P}_1)$  for which the differential entropy  $H(\pi)$  is finite which means that (20) equals to  $+\infty$  for every  $\pi \in \Pi(\mathbb{P}_0, \mathbb{P}_1)$ , i.e., every plan is optimal. In turn, when  $\mathbb{P}_0$  and  $\mathbb{P}_1$  are absolutely continuous, the OT plan is unique thanks to the strict convexity of entropy (on the set of absolutely continuous plans).
Recall equation (19) for  $\mathrm{KL}(\pi ||\pi^{W^{\epsilon}})$ :
$$
\operatorname {K L} \left(\pi \| \pi^ {W ^ {\epsilon}}\right) = \int_ {\mathcal {X} \times \mathcal {Y}} \frac {\| x - y \| ^ {2}}{2 \epsilon} d \pi (x, y) - H (\pi) + C. \tag {21}
$$
We again note that
$$
\inf _ {\pi \in \Pi (\mathbb {P} _ {0}, \mathbb {P} _ {1})} \mathrm {K L} (\pi | | \pi^ {W ^ {\epsilon}}) = \frac {1}{\epsilon} \inf _ {\pi \in \Pi (\mathbb {P} _ {0}, \mathbb {P} _ {1})} \left\{\int_ {\mathcal {X} \times \mathcal {Y}} \frac {| | x - y | | ^ {2}}{2} d \pi (x, y) - \epsilon H (\pi) \right\} + C,
$$
i.e., problems (20) and (21) can be viewed as equivalent as their minimizers are the same. For convenience, we proceed with  $\inf_{\pi \in \Pi (\mathbb{P}_0,\mathbb{P}_1)}\mathrm{KL}(\pi ||\pi^{W^\epsilon})$  and denote its optimal value by  $\mathcal{L}^*$ , i.e.,
$$
\mathcal {L} ^ {*} \stackrel {\text {d e f}} {=} \inf  _ {\pi \in \Pi (\mathbb {P} _ {0}, \mathbb {P} _ {1})} \mathrm {K L} (\pi | | \pi^ {W ^ {\epsilon}}).
$$
For a given  $\beta \in \mathcal{C}_{b,2}(\mathcal{V})$ , we define an auxiliary joint distribution  $d\pi^{\beta}(x,y) = d\pi^{\beta}(y|x)d\mathbb{P}_0(x)$  where  $d\pi^{\beta}(y|x)$  is given by
$$
d \pi^ {\beta} (y | x) = \frac {1}{C _ {\beta} ^ {\alpha}} \exp (\beta (y)) d \pi^ {W ^ {\epsilon}} (y | x),
$$
where  $C_{\beta}^{x}(x) \stackrel{\mathrm{def}}{=} \int_{\mathcal{Y}} \exp(\beta(y)) d\pi^{W^{\epsilon}}(y|x)$ . Note that  $C_{\beta}^{x} < \infty$  since  $\beta \in \mathcal{C}_{b,2}(\mathcal{Y})$  is upper bounded. Before going further, we need to introduce several technical auxiliary results.
Proposition B.1. For  $\nu \in \mathcal{P}_2(\mathcal{Y})$  and  $x\in \mathcal{X}$  it holds that
$$
\mathrm {K L} (\nu | | \pi^ {W ^ {\epsilon}} (\cdot | x)) - \int_ {\mathcal {Y}} \beta (y) d \nu (y) = \mathrm {K L} (\nu | | \pi^ {\beta} (\cdot | x)) - \log C _ {\beta} ^ {x}. \tag {22}
$$
Proof of Proposition B.1. We derive
$$
\begin{array}{l} \operatorname {K L} (\nu | | \pi^ {W ^ {\epsilon}} (\cdot | x)) - \int_ {\mathcal {Y}} \beta (y) d \nu (y) = \int_ {\mathcal {Y}} \log \frac {d \nu (y)}{d \pi^ {W ^ {\epsilon}} (y | x)} d \nu (y) - \int_ {\mathcal {Y}} \beta (y) d \nu (y) = \\ \int_ {\mathcal {Y}} \log \frac {d \nu (y)}{\exp (\beta (y)) d \pi^ {W ^ {\epsilon}} (y | x)} d \nu (y) = \int_ {\mathcal {Y}} \log \frac {C _ {\beta} ^ {x} d \nu (y)}{C _ {\beta} ^ {x} \exp (\beta (y)) d \pi^ {W ^ {\epsilon}} (y | x)} d \nu (y) = \\ \int_ {\mathcal {Y}} \log \frac {d \nu (y)}{d \pi^ {\beta} (y | x)} d \nu (y) - \log C _ {\beta} ^ {x} = \mathrm {K L} (\nu | | \pi^ {\beta} (\cdot | x)) - \log C _ {\beta} ^ {x}. \\ \end{array}
$$
Lemma B.2. For  $\pi \in \Pi (\mathbb{P}_0)$ , i.e., probability distributions  $\pi \in \mathcal{P}_2(\mathcal{X}\times \mathcal{Y})$  whose projection to  $\mathcal{X}$  equals  $\mathbb{P}_0$ , we have
$$
\mathrm {K L} \left(\pi | | \pi^ {W ^ {\epsilon}}\right) - \int_ {\mathcal {Y}} \beta (y) d \pi (y) = \mathrm {K L} \left(\pi | | \pi^ {\beta}\right) - \int_ {\mathcal {X}} \log C _ {\beta} ^ {x} d \mathbb {P} _ {0} (x). \tag {23}
$$
Proof of Lemma B.2. For each  $x \in \mathcal{X}$ , we substitute  $\nu = \pi(\cdot|x)$  to (22) and integrate over  $x \sim \mathbb{P}_0$ . For the left part, we obtain the following:
$$
\begin{array}{l} \int_ {\mathcal {X}} \left(\mathrm {K L} (\pi (\cdot | x) | | \pi^ {W ^ {\epsilon}} (\cdot | x)) - \int_ {\mathcal {Y}} \beta (y) d \pi (y | x)\right) d \mathbb {P} _ {0} (x) = \\ \int_ {\mathcal {X}} \mathrm {K L} (\pi (\cdot | x) | | \pi^ {W ^ {\epsilon}} (\cdot | x)) d \mathbb {P} _ {0} (x) - \int_ {\mathcal {X} \times \mathcal {Y}} \beta (y) d \pi (y | x) d \mathbb {P} _ {0} (x) = \\ \end{array}
$$
$$
\int_ {\mathcal {X}} \int_ {\mathcal {Y}} \log \frac {d \pi (y | x)}{d \pi^ {W ^ {\epsilon}} (y | x)} d \pi (y | x) d \mathbb {P} _ {0} (x) - \int_ {\mathcal {Y}} \beta (y) d \pi_ {1} (y) =
$$
$$
\int_ {\mathcal {X}} \int_ {\mathcal {Y}} \log \frac {d \pi (y | x) d \mathbb {P} _ {0} (x)}{d \pi^ {W ^ {\epsilon}} (y | x) d \mathbb {P} _ {0} (x)} d \pi (y | x) d \mathbb {P} _ {0} (x) - \int_ {\mathcal {Y}} \beta (y) d \pi_ {1} (y) =
$$
$$
\int_ {\mathcal {X} \times \mathcal {Y}} \log \frac {d \pi (x , y)}{d \pi^ {W ^ {\epsilon}} (x , y)} d \pi (x, y) - \int_ {\mathcal {Y}} \beta (y) d \pi_ {1} (y) =
$$
$$
\operatorname {K L} \left(\pi | | \pi^ {W ^ {\epsilon}}\right) - \int_ {\mathcal {Y}} \beta (y) d \pi_ {1} (y).
$$
For the right part, we obtain:
$$
\int_ {\mathcal {X}} \left\{\mathrm {K L} (\pi (\cdot | x) | | \pi^ {\beta} (\cdot | x)) - \log C _ {\beta} ^ {x} \right\} d \mathbb {P} _ {0} (x) =
$$
$$
\int_ {\mathcal {X}} \operatorname {K L} (\pi (\cdot | x) | | \pi^ {\beta} (\cdot | x)) d \mathbb {P} _ {0} (x) - \int_ {\mathcal {X}} \log C _ {\beta} ^ {x} d \mathbb {P} _ {0} (x) =
$$
$$
\int_ {\mathcal {X}} \int_ {\mathcal {Y}} \log \frac {d \pi (y | x)}{d \pi^ {\beta} (y | x)} d \pi (y | x) d \mathbb {P} _ {0} (x) - \int_ {\mathcal {X}} \log C _ {\beta} ^ {x} d \mathbb {P} _ {0} (x) =
$$
$$
\int_ {\mathcal {X}} \int_ {\mathcal {Y}} \log \frac {d \pi (y | x) d \mathbb {P} _ {0} (x)}{d \pi^ {\beta} (y | x) d \mathbb {P} _ {0} (x)} d \pi (y | x) d \mathbb {P} _ {0} (x) - \int_ {\mathcal {X}} \log C _ {\beta} ^ {x} d \mathbb {P} _ {0} (x) =
$$
$$
\int_ {\mathcal {X} \times \mathcal {Y}} \log \frac {d \pi (x , y)}{d \pi^ {\beta} (x , y)} d \pi (x, y) - \int_ {\mathcal {X}} \log C _ {\beta} ^ {x} d \mathbb {P} _ {0} (x) =
$$
$$
\operatorname {K L} \left(\pi | | \pi^ {\beta}\right) - \int_ {\mathcal {X}} \log C _ {\beta} ^ {x} d \mathbb {P} _ {0} (x).
$$
Hence, the equality (23) holds.
Now we introduce the following auxiliary functional  $\widetilde{\mathcal{L}}$ :
$$
\widetilde {\mathcal {L}} (\beta , \pi) \stackrel {{\text {d e f}}} {{=}} \mathrm {K L} (\pi | | \pi^ {W ^ {\epsilon}}) - \int_ {\mathcal {Y}} \beta (y) d \pi_ {1} (y) + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y).
$$
Recall that  $\pi_1$  denotes the second marginal distribution of  $\pi$ . We use this functional to derive the saddle point reformulation of EOT.
Lemma B.3 (Relaxation of entropic optimal transport). It holds that
$$
\mathcal {L} ^ {*} = \inf  _ {\pi \in \Pi (\mathbb {P} _ {0}, \mathbb {P} _ {1})} \mathrm {K L} \left(\pi \| \pi^ {W ^ {\epsilon}}\right) = \sup  _ {\beta} \inf  _ {\pi \in \Pi (\mathbb {P} _ {0})} \widetilde {\mathcal {L}} (\beta , \pi), \tag {24}
$$
where sup is taken over potentials  $\beta \in \mathcal{C}_{b,2}(\mathcal{Y})$  and inf over  $\pi \in \Pi (\mathbb{P}_0)$
Proof of Lemma B.3. We obtain
$$
\begin{array}{l} \inf  _ {\pi \in \Pi (\mathbb {P} _ {0}, \mathbb {P} _ {1})} \mathrm {K L} \left(\pi | | \pi^ {W ^ {\epsilon}}\right) = \inf  _ {\pi \in \Pi (\mathbb {P} _ {0}, \mathbb {P} _ {1})} \left\{\int_ {\mathcal {X}} \mathrm {K L} \left(\pi (y | x) | | \pi^ {W ^ {\epsilon}} (y | x)\right) d \mathbb {P} _ {0} (x) \right\} = \\ \inf  _ {\pi \in \Pi (\mathbb {P} _ {0}, \mathbb {P} _ {1})} \int_ {\mathcal {X}} C (x, \pi (y | x)) d \mathbb {P} _ {0} (x), \tag {25} \\ \end{array}
$$
where  $C(x,\nu)\stackrel {\mathrm{def}}{=}\mathrm{KL}\bigl (\nu ||\pi^{W^{\epsilon}}(y|x)\bigr)$ . The last problem in (25) is known as weak OT [7, 20] with a weak OT cost  $C$ . For a given  $\beta \in \mathcal{C}_{b,2}(\mathcal{Y})$ , consider its weak  $C$ -transform given by:
$$
\beta^ {C} (x) \stackrel {\text {d e f}} {=} \inf  _ {\nu \in \mathcal {P} _ {2} (\mathcal {Y})} \left\{C (x, \nu) - \int_ {\mathcal {Y}} \beta (y) d \nu (y) \right\}. \tag {26}
$$
Since  $C: \mathcal{X} \times \mathcal{P}_2(\mathcal{V}) \to \mathbb{R}$  is lower bounded (by zero), convex in the second argument and jointly lower semi-continuous, the following equality holds [7, Theorem 1.3]:
$$
\mathcal {L} ^ {*} = \inf  _ {\pi \in \Pi (\mathbb {P} _ {0}, \mathbb {P} _ {1})} \int_ {\mathcal {X}} C (x, \pi (y | x)) d \mathbb {P} _ {0} (x) = \sup  _ {\beta} \left\{\int_ {\mathcal {X}} \beta^ {C} (x) d \mathbb {P} _ {0} (x) + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y) \right\}, \tag {27}
$$
where sup is taken over  $\beta \in \mathcal{C}_{b,2}(\mathcal{Y})$ . We use our Proposition B.1 to note that
$$
\begin{array}{l} \beta^ {C} (x) = \inf  _ {\nu \in \mathcal {P} _ {2} (\mathcal {Y})} \left\{\mathrm {K L} \left(\nu | | \pi^ {W ^ {\epsilon}} (y | x)\right) - \int_ {\mathcal {Y}} \beta (y) d \nu (y) \right\} = \\ \inf_{\nu \in \mathcal{P}_{2}(\mathcal{Y})}\bigl\{\mathrm{KL}(\nu ||\pi^{\beta}(\cdot |x)) - \log C^{\alpha}_{\beta}\bigr \} = -\log C^{\alpha}_{\beta}. \\ \end{array}
$$
This allows us to derive
$$
\begin{array}{l} \int_ {\mathcal {X}} \beta^ {C} (x) d \mathbb {P} _ {0} (x) + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y) = (28) \\ - \int_ {\mathcal {X}} \log C _ {\beta} ^ {x} d \mathbb {P} _ {0} (x) + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y) = \\ \overbrace {\left\{\inf  _ {\pi \in \Pi (\mathbb {P} _ {0})} \mathrm {K L} \left(\pi \mid \mid \pi^ {\beta}\right) \right\}} ^ {= 0} - \int_ {\mathcal {X}} \log C _ {\beta} ^ {x} d \mathbb {P} _ {0} (x) + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y) = \\ \inf  _ {\pi \in \Pi (\mathbb {P} _ {0})} \left\{\mathrm {K L} \left(\pi | | \pi^ {\beta}\right) - \overbrace {\int_ {\mathcal {X}} \log C _ {\beta} ^ {x} d \mathbb {P} _ {0} (x) + \int_ {y} \beta (y) d \mathbb {P} _ {1} (y)} ^ {\text {D o n t d e p e n d o n} \pi} \right\} = \\ \inf  _ {\pi \in \Pi (\mathbb {P} _ {0})} \left\{\mathrm {K L} \left(\pi | | \pi^ {W ^ {\epsilon}}\right) - \int_ {\mathcal {Y}} \beta (y) d \pi (y) + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y) \right\} = \inf  _ {\pi \in \Pi (\mathbb {P} _ {0})} \widetilde {\mathcal {L}} (\beta , \pi). (29) \\ \end{array}
$$
Here in transition to line (29), we use our Lemma B.2. It remains to take  $\sup_{\beta}$  in equality between (28) and (29) and then recall (27) to finish the proof and obtain desired (24).
Thus, we can obtain the value  $\mathcal{L}^*$  (8) by solving maximin problem (24) with only one constraint  $\pi \in \Pi (\mathbb{P}_0)$ . Moreover, our following lemma shows that in all optimal pairs  $(\beta^{*},\pi^{*})$  which solve maximin problem (24),  $\pi^{*}$  is necessary the unique entropic OT plan between  $\mathbb{P}_0$  and  $\mathbb{P}_1$ .
Lemma B.4 (Entropic OT plan solves the relaxed entropic OT problem). Let  $\pi^{*}$  be the entropic OT plan between  $\mathbb{P}_0$  and  $\mathbb{P}_1$ . For every optimal  $\beta^{*} \in \operatorname{argsup}_{\beta} \inf_{\pi \in \Pi(\mathbb{P}_0)} \mathcal{L}(\beta, \pi)$ , we have
$$
\pi^ {*} = \underset {\pi \in \Pi \left(\mathbb {P} _ {0}\right)} {\operatorname {a r g i n f}} \widetilde {\mathcal {L}} \left(\beta^ {*}, \pi\right). \tag {30}
$$
Proof of Lemma B.4. Since  $\beta^{*}$  is optimal, we know from Lemma B.3 that  $\inf_{\pi \in \Pi (\mathbb{P}_0)}\mathcal{L}(\beta^*,\pi) = \mathcal{L}^*$ . Thanks to  $\pi^{*}\in \Pi (\mathbb{P}_{0},\mathbb{P}_{1})$ , we have  $\pi_1^* = \mathbb{P}_1$ . We substitute  $\pi^{*}$  to  $\mathcal{L}(\beta^{*},\pi)$  and obtain
$$
\mathcal {L} \left(\beta^ {*}, \pi^ {*}\right) = \mathrm {K L} \left(\pi^ {*} \mid \mid \pi^ {W ^ {\epsilon}}\right) + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y) - \int_ {\mathcal {Y}} \beta (y) \overbrace {d \pi_ {1} ^ {*} (y)} ^ {= d \mathbb {P} _ {1} (y)} = \mathrm {K L} \left(\pi^ {*} \mid \mid \pi^ {W ^ {\epsilon}}\right) = \mathcal {L} ^ {*}. \tag {31}
$$
The functional  $\pi \mapsto \mathcal{L}(\beta^{*},\pi)$  is strictly convex (in the convex subset of  $\Pi (\mathbb{P}_0)$  of distributions  $\pi$  for which  $\mathrm{KL}(\pi ||\pi^{W^{\epsilon}}) < \infty$ ). Thus, it has a unique minimizer, which is  $\pi^{*}$ .
From our Lemmas B.3 and B.4 it follows that to get the OT plan  $\pi^*$  one may solve the maximin problem (24) to obtain an optimal saddle point  $(\beta^{*},\pi^{*})$ . Unfortunately, it is challenging to estimate  $\mathrm{KL}(\pi ||\pi^{W^{\epsilon}})$  from samples, which limits the usage of this objective in practice.
B.2 Equivalence of EOT and DSB relaxed problems
Below we show how to relax SB problem (11) and link its solution to the relaxed entropic OT (24).
For a given  $\beta \in \mathcal{C}_{b,2}(\mathcal{V})$ , we define an auxiliary process  $T^{\beta}$  such that its conditional distributions are  $T_{|x,y}^{\beta} = W_{|x,y}^{\epsilon}$  and its joint distribution  $\pi^{T^{\beta}}$  at  $t = 0,1$  is given by  $\pi^{\beta}$ .
To simplify many of upcoming formulas, we introduce  $C_{\beta} \stackrel{\mathrm{def}}{=} \int_{\mathcal{X}} \log C_{\beta}^{x} d\mathbb{P}_{0}(x)$ . Also, we introduce  $\mathcal{F}(\mathbb{P}_0)$  to denote the set of processes starting at  $\mathbb{P}_0$  at time  $t = 0$ .
Lemma B.5 (Inner objectives of relaxed EOT and SB are KL with  $T^{\beta}$  and  $\pi^{T^{\beta}}$ ). For  $\pi \in \Pi (\mathbb{P}_0)$  and  $T\in \mathcal{F}(\mathbb{P}_0)$ , the following equations hold:
$$
\widetilde {\mathcal {L}} (\beta , \pi) = \mathrm {K L} \left(\pi \left| \left| \pi^ {T ^ {\beta}}\right) - C _ {\beta} + \int_ {y} \beta (y) d \mathbb {P} _ {1} (y), \right. \right. \tag {32}
$$
$$
\mathcal {L} (\beta , T) = \operatorname {K L} (T | | T ^ {\beta}) - C _ {\beta} + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y). \tag {33}
$$
Note that the last two terms in each line depend only on  $\beta$  but not on  $\pi$  or  $T$ .
Proof of Lemma B.5. The first equation (32) directly follows from Lemma B.2. Now we prove (33):
$$
\mathcal {L} (\beta , T) - \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y) = \mathrm {K L} (T | | W ^ {\epsilon}) - \int \beta (y) d \pi_ {1} ^ {T} (y) =
$$
$$
\mathrm {K L} \left(\pi^ {T} \mid \pi^ {W ^ {\epsilon}}\right) + \int_ {\mathcal {X} \times \mathcal {Y}} \mathrm {K L} \left(T _ {| x, y} | | W _ {| x, y} ^ {\epsilon}\right) d \pi^ {T} (x, y) - \int \beta (y) d \pi_ {1} ^ {T} (y) = \tag {34}
$$
$$
\mathrm {K L} \left(\pi^ {T} | | \pi^ {T ^ {\beta}}\right) - C _ {\beta} + \int_ {\mathcal {X} \times \mathcal {Y}} \mathrm {K L} \left(T _ {| x, y} | | W _ {| x, y} ^ {\epsilon}\right) d \pi^ {T} (x, y) =
$$
$$
\mathrm {K L} \left(\pi^ {T} \mid \mid \pi^ {T ^ {\beta}}\right) - C _ {\beta} + \int_ {\mathcal {X} \times \mathcal {Y}} \mathrm {K L} \left(T _ {| x, y} | | T _ {| x, y} ^ {\beta}\right) d \pi^ {T} (x, y) = \mathrm {K L} (T | | T ^ {\beta}) - C _ {\beta}. \tag {35}
$$
In the transition to line (34), we use the disintegration formula (6). In line (35), we use the definition of  $T^{\beta}$ , i.e., we exploit the fact that  $T_{|x,y}^{\beta} = W_{|x,y}^{\epsilon}$  and again use (6).
As a result of Lemma B.5, we obtain the following important corollary.
Corollary B.6 (The solution to the inner problem of relaxed SB is a diffusion). Consider the problem
$$
\inf  _ {T \in \mathcal {F} \left(\mathbb {P} _ {0}\right)} \mathcal {L} (\beta , T). \tag {36}
$$
Then  $T^{\beta}$  is the unique optimizer of (36) and it holds that  $T^{\beta} \in \mathcal{D}(\mathbb{P}_0)$ , i.e., it is a diffusion process:
$$
T ^ {\beta} = \underset {T \in \mathcal {F} \left(\mathbb {P} _ {0}\right)} {\operatorname {a r g i n f}} \mathcal {L} (\beta , T) = \underset {T _ {f} \in \mathcal {D} \left(\mathbb {P} _ {0}\right)} {\operatorname {a r g i n f}} \mathcal {L} (\beta , T _ {f}). \tag {37}
$$
Proof. Thanks to (33), we see that  $T^{\beta}$  is the unique minimizer of (36). Now let  $\mathbb{Q} \stackrel{\mathrm{def}}{=} \pi_1^{T^{\beta}}$ . Then
$$
T ^ {\beta} = \operatorname * {a r g m i n} _ {T \in \mathcal {F} (\mathbb {P} _ {0})} \mathcal {L} (\beta , T) = \operatorname * {a r g m i n} _ {T \in \mathcal {F} (\mathbb {P} _ {0})} \left[ \mathrm {K L} (T | | W ^ {\epsilon}) - \int_ {\mathcal {Y}} \beta (y) d \pi_ {1} ^ {T} (y) \right] =
$$
$$
\operatorname *{arginf}_{T\in \mathcal{F}(\mathbb{P}_{0},\mathbb{Q})}\left[\mathrm{KL}(T||W^{\epsilon}) - \underbrace{\int_{\mathcal{Y}}\beta(y)d\pi_{1}^{T}(y)}_{= \mathrm{Const, since}\pi_{1}^{T} = \pi_{1}^{T\beta} = \mathbb{Q}}\right] = \operatorname *{arginf}_{T\in \mathcal{F}(\mathbb{P}_{0},\mathbb{Q})}\mathrm{KL}(T||W^{\epsilon}) =
$$
$$
\operatorname * {a r g i n f} _ {T _ {f} \in \mathcal {D} (\mathbb {P} _ {0}, \mathbb {Q})} \mathrm {K L} \left(T _ {f} | | W ^ {\epsilon}\right) = \operatorname * {a r g i n f} _ {T _ {f} \in \mathcal {D} (\mathbb {P} _ {0}, \mathbb {Q})} \frac {1}{2 \epsilon} \mathbb {E} _ {T _ {f}} \left[ \int_ {0} ^ {1} | | f (X _ {t}, t) | | ^ {2} d t \right]. \tag {38}
$$
In transition to (38), we use the fact that the process solving the Schrödinger Bridge (this time between  $\mathbb{P}_0$  and  $\mathbb{Q}$ ) with the Wiener Prior is a diffusion process (see Dynamic SB problem in §2.2 for details). As a result, we obtain  $T^{\beta} \in \mathcal{D}(\mathbb{P}_0, \mathbb{Q}) \subset \mathcal{D}(\mathbb{P}_0)$  and finish the proof.
Below we show that for a given  $\beta$ , minimization of the SB relaxed functional  $\mathcal{L}(\beta, T_f)$  over  $T_f$  is equivalent to the minimization of relaxed EOT functional  $\widetilde{\mathcal{L}}(\beta, \pi)$  (24) with the same  $\beta$ .
Lemma B.7 (Equivalence of the inf values of the relaxed functionals). It holds that
$$
\inf  _ {T _ {f} \in \mathcal {D} (\mathbb {P} _ {0})} \mathcal {L} (\beta , T _ {f}) = \inf  _ {\pi \in \Pi (\mathbb {P} _ {0})} \widetilde {\mathcal {L}} (\beta , \pi) = - C _ {\beta} + \int \beta (y) d \mathbb {P} _ {1} (y). \tag {39}
$$
Moreover, the unique minimizers are given by  $T^{\beta} \in \mathcal{D}(\mathbb{P}_0)$  and  $\pi^{T\beta} \in \Pi(\mathbb{P}_0)$ , respectively.
Proof of Lemma B.7. Follows from Lemma B.5 and Corollary B.6.  $\square$
Finally, we see that both the maximin problems are equivalent.
Corollary B.8 (Equivalence of EOT and DSB maximin problems). It holds that
$$
\mathcal {L} ^ {*} = \sup  _ {\beta} \inf  _ {T _ {f} \in \mathcal {D} \left(\mathbb {P} _ {0}\right)} \mathcal {L} (\beta , T _ {f}) = \sup  _ {\beta} \inf  _ {\pi \in \Pi \left(\mathbb {P} _ {0}\right)} \widetilde {\mathcal {L}} (\beta , \pi) \tag {40}
$$
Proof of Corollary B.8. We take  $\sup_{\beta}$  of both parts in equation (39).
Also, it follows that the maximization of  $\inf_{T_f\in \mathcal{D}(\mathbb{P}_0)}\mathcal{L}(\beta ,T_f)$  over  $\beta$  allows to solve entropic OT.
B.3 Proofs of main results
Finally, after long preparations, we prove our main Theorem 4.1.
Proof of Theorem 4.1 and Corollary 4.2. From our Lemma B.7 and Corollary B.8 it follows that
$$
\beta^{*}\in \operatorname *{argsup}_{\beta}\inf_{T_{f}\in \mathcal{D}(\mathbb{P}_{0})}\mathcal{L}(\beta ,T_{f})\Leftrightarrow \beta^{*}\in \operatorname *{argsup}_{\beta}\inf_{\pi \in \Pi (\mathbb{P}_{0})}\widetilde{\mathcal{L}} (\beta ,\pi),
$$
i.e., both maximin problems share the same optimal  $\beta^{*}$ . Thanks to our Lemma B.7, we already know that the process  $T^{\beta^{*}}\in \mathcal{D}(\mathbb{P}_{0})$  and the plan  $\pi^{T^{\beta^{*}}}\in \Pi (\mathbb{P}_{0})$  are the unique minimizers of problems
$$
\inf  _ {T _ {f} \in \mathcal {D} (\mathbb {P} _ {0})} \mathcal {L} (\beta^ {*}, T _ {f}) = \inf  _ {\pi \in \Pi (\mathbb {P} _ {0})} \widetilde {\mathcal {L}} (\beta^ {*}, \pi),
$$
respectively. Therefore,  $T_{f^*} = T^{\beta^*}$  and, in particular,  $\pi^{T_f*} = \pi^{T^{\beta^*}}$ . Moreover, since  $(\beta^*,\pi^{T_f*})$  is an optimal saddle point for  $\widetilde{\mathcal{L}}$ , from Lemma B.4 we conclude that  $\pi^{T_f*} = \pi^*$ , i.e.,  $\pi^{T_f*}$  is the EOT plan between  $\mathbb{P}_0$  and  $\mathbb{P}_1$ . In particular,  $\pi^{T_f*}\in \Pi (\mathbb{P}_0,\mathbb{P}_1)$  which also implies that  $T_{f^*}\in \mathcal{D}(\mathbb{P}_0,\mathbb{P}_1)$ . The last step is to derive
$$
\mathcal{L}^{*} = \mathcal{L}(\beta^{*},T_{f^{*}}) = \mathrm{KL}(T_{f^{*}}||W^{\epsilon}) + \underbrace{\int_{\mathcal{Y}}\beta^{*}(y)d\mathbb{P}_{1}(y) - \int_{\mathcal{Y}}\beta^{*}(y)\overbrace{d\pi_{1}^{T_{f^{*}}}(y)}^{= d\mathbb{P}_{1}(y)}}_{= 0\text{since} T_{f^{*}}\in \mathcal{D}(\mathbb{P}_{0},\mathbb{P}_{1})} = \mathrm{KL}(T_{f^{*}}||W^{\epsilon}).
$$
which concludes that  $T_{f^*}$  is the solution to SB (5).
Proof of Theorem 4.3. Part 1. From Lemma B.5 and Corollary B.6 it follows that  $\inf_{T_f}\mathcal{L}(\widehat{\beta},T_f)$  has the unique minimizer  $T^{\widehat{\beta}}$  whose conditional distributions are  $T_{|x,y}^{\widehat{\beta}} = W_{|x,y}^{\epsilon}$ . Therefore,
$$
\epsilon_ {1} = \mathcal {L} (\hat {\beta}, T _ {\hat {f}}) - \inf _ {T _ {f}} \mathcal {L} (\hat {\beta}, T _ {f}) =
$$
$$
\left[ \mathrm {K L} \left(T _ {\hat {f}} \mid \mid T ^ {\hat {\beta}}\right) - C _ {\hat {\beta}} + \int_ {\mathcal {Y}} \widehat {\beta} (y) d \mathbb {P} _ {1} (y) \right] - \left[ - C _ {\widehat {\beta}} + \int_ {\mathcal {Y}} \widehat {\beta} (y) d \mathbb {P} _ {1} (y) \right] = \mathrm {K L} \left(T _ {\hat {f}} \mid \mid T ^ {\hat {\beta}}\right). \tag {41}
$$
Part 2. Now we consider  $\epsilon_{2}$ . We know that
$$
\mathcal {L} ^ {*} = \mathrm {K L} \left(T _ {f ^ {*}} | | W ^ {\epsilon}\right) =
$$
$$
\mathrm {K L} (\pi^ {T _ {f ^ {*}}} | | \pi^ {W ^ {\epsilon}}) + \int_ {\mathcal {X} \times \mathcal {Y}} \mathrm {K L} (T _ {f ^ {*} | x, y} | | W _ {| x, y} ^ {\epsilon}) d \pi^ {T _ {f ^ {*}}} (x, y) = \mathrm {K L} (\pi^ {T _ {f ^ {*}}} | | \pi^ {W ^ {\epsilon}}).
$$
From Lemma B.5 and Corollary B.6, we also know that
$$
\inf  _ {T _ {f}} \mathcal {L} (\hat {\beta}, T _ {f}) = - C _ {\hat {\beta}} + \int \hat {\beta} (y) d \mathbb {P} _ {1} (y).
$$
$$
\epsilon_ {2} = \mathcal {L} ^ {*} - \inf _ {T _ {f}} \mathcal {L} (\hat {\beta}, T _ {f ^ {*}}) = \mathrm {K L} (\pi^ {T _ {f ^ {*}}} | | \pi^ {W ^ {\epsilon}}) + C _ {\hat {\beta}} - \int \hat {\beta} (y) d \mathbb {P} _ {1} (y) =
$$