SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We conduct a toy experiment to support this claim and consider  $Q_{v}$  with  $\epsilon = 0.01$  and  $v(x,t) = \nabla \log p(x)$ , where  $\log p(x)$  is a 2D distribution with a wave shape, see Figure 8. Intuitively, it means that trajectories should be concentrated in the regions with a high density of  $p$ . In Figure e 8, there the grey-scale color map represents the density of  $p$ , start points  $(\mathbb{P}_0)$  are green, target points  $(\mathbb{P}_1)$  are red, obtained trajectories are pink and mapped points are blue.
J ENOT for the unregularized OT  $(\epsilon = 0)$
Our proposed algorithm is designed to solve entropic OT and the equivalent SB problem. This implies that  $\epsilon > 0$ . Nevertheless, our algorithm technically allows using even  $\epsilon = 0$ , in which case it presumably computes the unregularized OT map for the quadratic cost. Here we present some empirical evidence supporting this claim as well some theoretical insights.
EMPIRICAL EVIDENCE. We consider the experimental setup with images from the continuous Wasserstein-2 benchmark [28, §4.4]. The images benchmark provides 3 pairs of distributions (Early, Mid, Late) for which the ground truth unregularized OT map for the quadratic cost is known by the construction. Hence, we may compare the map learned with our method  $(\epsilon = 0)$  with the true one.
We train our method with  $\epsilon = 0$  on each of 3 benchmark pairs and present the quantitative results in Table 6. We use the same  $\mathcal{L}^2$ -UVP metric [28, §4.2] as the authors of the benchmark. As the baselines, we include the results of MM:R method from [28] and the method from [3]. Both methods are minimax and have some similarities with our approach. As we can see, ENOT with  $\epsilon = 0$
Figure 8: Toy example with ENOT (ours) for the complex prior  $Q_{v}: dX_{t} = v(X_{t}, t)dt + \sqrt{\epsilon} dW_{t}$ .
Benchmark	Early	Mid	Late
[28]*	1.4	0.4	0.22
[3]*	0.61	0.20	0.09
ENOT (ours)	0.77	0.21	0.09
Table 6: Comparison on W2 benchmark. *Results are taken from [3, Table 2].
works better than the MM:R solver but slightly underperforms compared to [3]. This evaluation demonstrates that our method recovers the unregularized OT map for the quadratic cost with the comparable quality to the existing saddle point OT methods.
THEORETICAL INSIGHTS. We see that empirically our method with  $\epsilon = 0$  recovers the unregularized OT map. At the same time, this is not supported by our theoretical results as they work exclusively for  $\epsilon > 0$  and rely on the properties of the KL divergence.
Overall, it seems like for  $\epsilon = 0$  our method yields a saddle point reformulation of the Benamou-Brenier (BB) [8] problem which is also known as the dynamic version of the unregularized OT  $(\epsilon = 0)$  with the quadratic cost. This problem can be formulated as follows:
$$
\inf  _ {T _ {f}} \left\{\frac {1}{2} \mathbb {E} _ {T _ {f}} \left[ \int_ {0} ^ {1} | | f (X _ {t}, t) | | ^ {2} d t \right] \right\} \quad \text {s . t .} \quad T _ {f}: d X _ {t} = f (X _ {t}, t) d t, \quad X _ {0} \sim \mathbb {P} _ {0}, X _ {1} \sim \mathbb {P} _ {1}, \tag {45}
$$
i.e., the goal is to find the process  $T_{f}$  of the minimal energy which moves the probability mass of  $\mathbb{P}_0$  to  $\mathbb{P}_1$ . BB (45) is very similar to DSB (11) but there is no multiplier  $\frac{1}{\epsilon}$ , and the stochastic process  $T_{f}$  is restricted to be deterministic  $(\epsilon = 0)$ . It is governed by a vector field  $f$ . Just like the DSB (11) is equivalent to EOT (2), it is known that BB (45) is equivalent to unregularized OT with the quadratic cost  $(\epsilon = 0)$ . Namely, the distribution  $\pi^{T_{f^*}}$  is the unregularized OT plan between  $\mathbb{P}_0$  and  $\mathbb{P}_1$ .
In turn, our Algorithm 1 for  $\epsilon = 0$  optimizes the following saddle point objective:
$$
\sup  _ {\beta} \inf  _ {T _ {f}} \mathcal {L} (\beta , T _ {f}) \stackrel {\text {d e f}} {=} \sup  _ {\beta} \inf  _ {T _ {f}} \left\{\frac {1}{2} \mathbb {E} _ {T _ {f}} \left[ \int_ {0} ^ {1} | | f (X _ {t}, t) | | ^ {2} d t \right] + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y) - \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} ^ {T _ {f}} (y) \right\}, \tag {46}
$$
where  $T_{f}:dX_{t} = f(X_{t},t)dt$  with  $X_0\sim \mathbb{P}_0$  (the constraint  $X_{1}\sim \mathbb{P}_{1}$  here is lifted) and  $\beta \in \mathcal{C}_{2,b}(\mathcal{V})$ . Just like in the Entropic case, functional  $\mathcal{L}$  can be viewed as the Lagrangian for BB (45) with  $\beta$  playing the role of the Lagrange multiplier for the constraint  $d\pi_1^{T_f}(y) = d\mathbb{P}_1(y)$ . Naturally, it is expected that the value (45) coincides with (46), and we provide a sketch of the proof of this fact.
Overall, the proof logic is analogous to the Entropic case but the actual proof is much more technical as we can not use the  $KL$ -divergence machinery which helps to avoid non-uniqueness, etc.
Step 1 (Auxiliary functional, analog of Lemma B.3). We introduce an auxiliary functional
$$
\widetilde {\mathcal {L}} (\beta , H) \stackrel {\mathrm {d e f}} {=} \int_ {\mathcal {X}} \frac {1}{2} \| x - H (x) \| ^ {2} d \mathbb {P} _ {0} (x) - \int_ {\mathcal {X}} \beta (H (x)) d \mathbb {P} _ {0} (x) + \int_ {\mathcal {Y}} \beta (y) d \mathbb {P} _ {1} (y),
$$
where  $\beta$  is a potential and  $H: \mathbb{R}^D \to \mathbb{R}^D$  is a measurable map. This functional is nothing but the well-known max-min reformulation of static OT problem (in Monge's form) with the quadratic cost [3, Eq. 4], [28, Eq.9]. Hence,
$$
\sup_{\beta}\inf_{H}\widetilde{\mathcal{L}} (\beta ,H) = \underbrace{\inf_{H\sharp\mathbb{P}_{0} = \mathbb{P}_{1}}\int_{\mathcal{X}}\frac{1}{2}||x - H(x)||^{2}d\mathbb{P}_{0}(x)}_{\stackrel {\mathrm{def}}{=}\mathcal{L}^{*}}.
$$
Step 2 (Solution of the inner problem is always an OT map). An existence of some minimizer  $H = H^{\beta}$  in  $\inf_{H} \widetilde{L}(\beta, H)$  can be deduced from the measurable argmin selection theorem, e.g., [21, Theorem 18.19]. For this  $H^{\beta}$  we consider  $\mathbb{P}' \stackrel{\text{def}}{=} H^{\beta} \sharp \mathbb{P}_0$ . Recall that
$$
H ^ {\beta} \in \operatorname * {a r g i n f} _ {H} \widetilde {L} (\beta , H) = \operatorname * {a r g i n f} _ {H} \int_ {\mathcal {X}} \left\{\frac {\| x - H (x) \| ^ {2}}{2} - \beta (H (x)) \right\} d \mathbb {P} _ {0} (x).
$$
Here we may add the fictive constraint  $H\sharp \mathbb{P}_0 = \mathbb{P}'$  which is anyway satisfied by  $H^{\beta}$  and get
$$
H ^ {\beta} \in \operatorname * {a r g m i n} _ {H \sharp \mathbb {P} _ {0} = \mathbb {P} ^ {\prime}} \int_ {\mathcal {X}} \left\{\frac {\| x - H (x) \| ^ {2}}{2} - \beta (H (x)) \right\} d \mathbb {P} _ {0} (x) = \operatorname * {a r g m i n} _ {H \sharp \mathbb {P} _ {0} = \mathbb {P} ^ {\prime}} \int_ {\mathcal {X}} \frac {\| x - H (x) \| ^ {2}}{2} d \mathbb {P} _ {0} (x).
$$
The last equality holds since  $\int \beta (H(x))d\mathbb{P}_0(x) = \int \beta (y)d\mathbb{P}'(y)$  does not depend on the choice of  $H$  due to the constraint  $H\sharp \mathbb{P}_0 = \mathbb{P}'$ . The latter is the OT problem between  $\mathbb{P}$  and  $\mathbb{P}'$  and we see that  $H^{\beta}$  is its solution.
Step 3 (Equivalence for inner objective values). Since  $H^{\beta}$  is the OT map between  $\mathbb{P}_0, \mathbb{P}'$  (it is unique as  $\mathbb{P}_0$  is absolutely continuous [43]), it can be represented as an ODE solution  $T_{f^\beta}$  to the Benamour Brenier problem between  $\mathbb{P}_0, \mathbb{P}'$ , i.e.,  $T_{f^\beta}: dX_t = f^\beta(X_t, t) dt$  and  $H^\beta(X_0) = X_0 + \int_0^1 f^\beta(X_t, t) dt$ . Furthermore, in this case,  $\|X_0 - H^\beta(X_0)\|^2 = \int_0^1 ||f^\beta(X_t, t)||^2 dt$ . Hence, it can be derived that
$$
\inf _ {H} \widetilde {\mathcal {L}} (\beta , H) = \inf _ {T _ {f}} \mathcal {L} (\beta , T _ {f}).
$$
Step 4 (Equivalence of the saddle point objective). Take sup over  $\beta \in \mathcal{C}_{b,2}(\mathcal{Y})$  and get the final equivalence:
$$
\sup  _ {\beta} \inf  _ {H} \widetilde {\mathcal {L}} (\beta , H) = \sup  _ {\beta} \inf  _ {H} \mathcal {L} (\beta , T _ {f}) = \mathcal {L} ^ {*}.
$$
Step 5 (Dynamic OT solutions are contained in optimal saddle points). Pick any optimal  $\beta^{*} \in \operatorname{argsup}_{\beta} \inf_{H} \mathcal{L}(\beta, T_{f^{*}})$  and let  $T_{f^{*}}$  be any solution to the Benamou-Brenier problem. Checking that  $T^{*} \in \inf_{H} \mathcal{L}(\beta^{*}, T_{f})$  can be done analogously to [31, Lemma 4], [42, Lemma 4.1].
The derivation above shows the equivalence of objective values of dynamic unregularized OT (45) and our saddle point reformulation of BB (45). Additionally, it shows that solutions  $T_{f^*}$  can be recovered from some optimal saddle points  $(\beta^*, T_{f^*})$  of our problem. At the same time, unlike the EOT case  $(\epsilon > 0)$ , it is not guaranteed that for all the optimal saddle points  $(\beta^*, T_{f^*})$  it holds that  $T_{f^*}$  is the solution to the BB problem. This aspect seems to be closely related to the fake solutions issue in the saddle point methods of OT [30] and may require further studies.