SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Here we use the interchange between the integral and inf [59, Theorem 3A]; in (14) the inf is taken over measurable maps. Since  $(x,y)\mapsto c(x,y) - f(y)$  is a continuous function on a compact set, it admits a measurable selection  $T(x)\in \arg \min_{y\in \mathcal{Y}}\left\{c(x,y) - f(y)\right\}$  minimizing (14), see [2, Theorem 18.19]. Thus, inf can be replaced by min. We combine (14) and (13) and obtain an equivalent saddle point problem:
$$
\operatorname {C o s t} _ {w} (\mathbb {P}, \mathbb {Q}) = \max  _ {f \leq 0} \min  _ {T: \mathcal {X} \rightarrow \mathcal {Y}} \mathcal {L} (f, T), \tag {15}
$$
where the functional  $\mathcal{L}(f,T)$  is defined by
$$
\mathcal {L} (f, T) \stackrel {\text {d e f}} {=} \int_ {\mathcal {X}} c (x, T (x)) d \mathbb {P} (x) - \int_ {\mathcal {X}} f (T (x)) d \mathbb {P} (x) + w \int_ {\mathcal {Y}} f (y) d \mathbb {Q} (y). \tag {16}
$$
Functional  $\mathcal{L}(f,T)$  can be viewed as a Lagrangian with  $f\leq 0$  being a multiplier for the constraint  $T\sharp \mathbb{P} - w\mathbb{Q}\leq 0$ . By solving (15), one may obtain IT maps.
Theorem 5 (IT maps are contained in optimal saddle points). Let  $f^{*}$  be any maximizer in (13). If  $\pi^{*} \in \Pi^{w}(\mathbb{P},\mathbb{Q})$  is a deterministic IT plan, i.e., it solves (12) and has the form  $\pi^{*} = [\mathrm{id}_{\mathcal{X}},T^{*}]\sharp \mathbb{P}$  for some measurable  $T^{*}: \mathcal{X} \to \mathcal{Y}$ , then
$$
T^{*}\in \operatorname *{arg  min}_{T:\mathcal{X}\to \mathcal{Y}}\mathcal{L}(f^{*},T).
$$
Algorithm 1: Procedure to compute the IT map between  $\mathbb{P}$  and  $\mathbb{Q}$  for transport cost  $c(x,y)$  and weight  $w$ .
Input :distributions  $\mathbb{P},\mathbb{Q}$  accessible by samples; mapper  $T_{\theta}:\mathcal{X}\rightarrow \mathcal{V}$  ; potential  $f_{\psi}:\mathcal{X}\to \mathbb{R}_{-}$  transport cost  $c:\mathcal{X}\times \mathcal{Y}\to \mathbb{R}$  ; weight  $w\geq 1$  ; number  $K_{T}$  of inner iterations;   
Output:approximate IT map  $(T_{\theta})_{\#}\mathbb{P}\leq w\mathbb{Q};$    
repeat Sample batches  $X\sim \mathbb{P},Y\sim \mathbb{Q};$ $\mathcal{L}_f\gets w\cdot \frac{1}{|Y|}\sum_{y\in Y}f_\psi (y) - \frac{1}{|X|}\sum_{x\in X}f_\psi (T_\theta (x));$  Update  $\psi$  by using  $\frac{\partial\mathcal{L}_f}{\partial\psi}$  to maximize  $\mathcal{L}_f$  for  $k_{T} = 1,2,\ldots ,K_{T}$  do Sample batch  $X\sim \mathbb{P};$ $\mathcal{L}_T\gets \frac{1}{|X|}\sum_{x\in X}\left[c(x,T_\theta (x)) - f_\psi (T_\theta (x))\right];$  Update  $\theta$  by using  $\frac{\partial\mathcal{L}_T}{\partial\theta}$  to minimize  $\mathcal{L}_T$    
until not converged;
Our Theorem 5 states that in some optimal saddle points  $(f^{*},T^{*})$  of (15) it holds that  $T^{*}$  is the IT map between  $\mathbb{P},\mathbb{Q}$ . In general, the  $\arg \inf_{T}$  set for an optimal  $f^{*}$  might contain not only IT maps  $T^{*}$ , but other functions as well (fake solutions), see limitations in Appendix A.
We solve the optimization problem (15) by approximating the map  $T$  and potential  $f$  with neural networks  $T_{\theta}$  and  $f_{\psi}$ , respectively. To make  $f_{\psi}$  non-positive, we use  $x \mapsto -|x|$  as the last layer. The nets are trained using random batches from  $\mathbb{P}, \mathbb{Q}$  and stochastic gradient ascent-descent. We detail the optimization procedure in Algorithm 1.
OT in generative models. A popular way to apply OT in generative models is to use the OT cost as the loss function to update the generator [4, 28, 26], see [41] for a survey. These methods are not relevant to our study as they do not learn an OT map but only compute the OT cost.
Recent works [43, 42, 61, 20, 5, 24, 29] are the most related to our study. These papers show the possibility to learn the OT maps (or plans) via solving saddle point optimization problems derived from the standard  $c$ -transform-based duality formulas for OT. The underlying principle of our objective (15) is analogous to theirs. The key difference is that they consider OT problems (1), (2) and enforce the equality constraints, e.g.,  $T \sharp \mathbb{P} = \mathbb{Q}$ , while our approach enforces the inequality constraint  $T \sharp \mathbb{P} \leq w\mathbb{Q}$  allowing to partially align the measures. We provide a detailed discussion of relation with these works as well as with the fundamental OT (2) and partial OT (4) literature [22, 10, 62] in Appendix F, see bibliographic remarks after the proofs of Theorems 2, 4 and 5.
For completeness, we also mention other existing neural OT methods [25, 63, 49, 17, 39]. These works are less related to our work because they either underperform compared to the above-mentioned saddle point methods, see [40] for evaluation, or they solve specific OT formulations, e.g., entropic OT [56, §4], which are not relevant to our study.
The papers [66, 47, 19] are slightly more related to our work. They propose neural methods for unbalanced OT [14] which can also be used to partially align measures. As we will see in Appendix B, UOT is hardly suitable for ET (8) as it is not easy to control how it spreads the probability mass. Besides, these methods consider OT between small-dimensional datasets or in latent spaces. It is not clear whether they scale to high-dimensions, e.g., images.
Discrete OT methods, including partial OT [11], are not relevant to us, see Appendix D for details.
Unpaired domain translation [3] is a generic problem which includes image super-resolution [12], inpainting [71], style translation [34] tasks, etc. Hence, we do not mention all of the existing approaches but focus on their common main features instead. In many applications, it is important to preserve semantic information during the translation, e.g., the image content. In most cases, to do this it is sufficient to use convolutional neural networks. They preserve the image content thanks to their design which is targeted to only locally change the image [18]. However, in some of the tasks
(a) Input, target measures.
(b) IT map,  $w = 3$ .
(c) IT map,  $w = \frac{3}{2}$ .
(d) IT map,  $w = 1$ .
Figure 7: Incomplete transport (IT) maps learned by our Algorithm 1 in 'Wi-Fi' experiment.
(a) Input measure.
(b) IT map,  $w = 8$ .
(c) IT map,  $w = 4$ .
(d) IT map,  $w = 2$ .
(e) IT map,  $w = 1$
(f) Target measure.
Figure 8: PCA projections of input  $x \sim \mathbb{P}$ , mapped  $T(x) \sim T \sharp \mathbb{P}$  and target  $y \sim \mathbb{Q}$  test samples (handbag→shoes experiment).
additional image properties must be kept, e.g., image colors in super-resolution. Typical approaches to such problems are based on GANs and use additional similarity losses, e.g., the basic CycleGAN [72] enforces  $\ell^1$  similarity. Such methods are mostly related to our work. However, without additional modifications most of these approaches only partially maintain the image properties, see [44, Figure 5]. As we show in experiments (Appendix C), IT achieves better similarity than popular CycleGAN, StarGAN-v2 [15] methods, both default and modified to better preserve the image content.
In §5.1, we provide illustrative toy 2D examples. In §5.2, we evaluate our method on the unpaired image-to-image translation task. Technical training details (architectures, learning rates, etc.) are given in Appendix E. The code is written using PyTorch framework and is publicly available at
https://github.com/milenagazdieva/ExtremalNeuralOptimalTransport.
Transport costs. We experiment with the quadratic cost  $c(x,y) = \ell^2 (x,y)$  as this cost already provides reasonable performance. We slightly abuse the notation and use  $\ell^2$  to denote the squared error normalized by the dimension. Experiments with the perceptual cost are given in Appendix G.3.
In this section, we provide 'Wi-Fi' and 'Accept' examples in 2D to show how the choice of  $w$  affects the fraction of the target measure  $\mathbb{Q}$  to which the probability mass of the input  $\mathbb{P}$  is mapped. In both cases, measure  $\mathbb{P}$  is Gaussian. In Appendix B, we demonstrate how other methods perform on these 'Wi-Fi' and 'Accept' toy examples.
In 'Wi-Fi' experiment (Fig. 7), target  $\mathbb{Q}$  contains 3 arcs. We provide the learned IT maps for  $w\in [1,\frac{3}{2},3]$ . The results show that by varying  $w$  it is possible to control the fraction of  $\mathbb{Q}$  to which the mass of  $\mathbb{P}$  will be mapped. In Fig. 7, we see that for  $w = 1$  our IT method learns all 3 arcs. For  $w = \frac{3}{2}$ , it captures 2 arcs, i.e.,  $\approx \frac{2}{3}$  of  $\mathbb{Q}$ . For  $w = 3$ , it learns 1 arc which corresponds to  $\approx \frac{1}{3}$  of  $\mathbb{Q}$ .
In 'Accept' experiment (Fig. 2), target  $\mathbb{Q}$  is a two-line text. Here we put  $w = 2$  and, as expected, our method captures only one line of the text which is the closest to  $\mathbb{P}$  in  $\ell^2$ .
5.2 Unpaired Image-to-image Translation
Here we learn IT maps between various pairs of datasets. We test  $w \in \{1, 2, 4, 8\}$  in all experiments. For completeness, we consider bigger weights  $w \in \{16, 32\}$  in Appendix G.4.
(a) IT results for textures  $\rightarrow$  chairs  $(64\times 64)$
Figure 9: Unpaired Translation with our Incomplete Transport.
(b) IT results for  $ffhq \to$  comics (128×128).
Experiment	w = 1	w = 2	w = 4	w = 8
celeba → anime	0.297	0.154	0.133	0.094
handbag → shoes	0.368	0.320	0.259	0.252
textures → chairs	0.603	0.516	0.474	0.408
ffhq → comics	0.224	0.220	0.200	0.196
(a) Test  $\ell^2$  transport cost of our learned IT maps.
Experiment	w = 1	w = 2	w = 4	w = 8
celeba → anime	14.65	20.79	22.18	22.84
handbag → shoes	27.10	29.70	42.90	53.80
texts → chairs	N/A	N/A	N/A	N/A
ffhq → comics	20.95	22.38	22.77	23.67
(b) Test FID of our learned IT maps.
Table 1: Test  ${\ell }^{2}$  cost and FID of our learned IT maps.
Image datasets. We utilize the following publicly available datasets as  $\mathbb{P}$ ,  $\mathbb{Q}$ : celebrity faces [46], aligned anime faces $^3$ , flickr-faces-HQ [36], comic faces $^4$ , Amazon handbags from LSUN dataset [68], shoes [67], textures [16] and chairs from Bonn furniture styles dataset [1]. The sizes of datasets are from 5K to 500K samples. We work with  $64 \times 64$  and  $128 \times 128$  images.
Train-test split. We use  $90\%$  of each dataset for training. The remaining  $10\%$  are held for test. All the presented qualitative and quantitative results are obtained for test images.
Experimental results. Our evaluation shows that with the increase of  $w$  the images  $\hat{T}(x)$  translated by our IT method become more similar to the respective inputs  $x$  w.r.t.  $\ell^2$ . In Table 1a, we quantify this effect. Namely, we show that the test transport cost  $\frac{1}{N_{\mathrm{test}}} \sum_{n=1}^{N_{\mathrm{test}}} c(x, \hat{T}(x))$  decreases with the increase of  $w$  which empirically verifies our Theorem 2.
We qualitatively demonstrate this effect in Fig. 1b, 1a, 9b, 9a and 8. In celeba (female)  $\rightarrow$  anime (Fig. 1b), the hair and background colors of the learned anime images become closer to celebrity faces' colors with the increase of  $w$ . For example, in the 4th column of Fig. 1b, the anime hair color changes from green to brown, which is close to that of the respective celebrity. In the 6th column, the background is getting darker. In handbag  $\rightarrow$  shoes (Fig. 1a), the color, texture and size of the shoes become closer to that of handbag. Additionally, for this experiment we plot the projections of the learned IT maps to the first 2 principal components of  $\mathbb{Q}$  (Fig. 8). We see that projections are close to  $\mathbb{Q}$  for  $w = 1$  and become closer to  $\mathbb{P}$  with the increase of  $w$ . In  $fhhq \rightarrow$  comics, the changes mostly affect facial expressions and individual characteristics. In textures  $\rightarrow$  chairs, the changes are mostly related to chairs' size which is expected since we use pixel-wise  $\ell^2$  as the cost function. Additional qualitative results are given in Appendix G.5 (Fig. 24, 25, 9a).
For completeness, we measure test FID [32] of the translated samples, see Table 1b. We do not calculate FID in the handbag  $\rightarrow$  chairs experiment because of too small sizes of the test parts of the datasets (500 textures, 2K chairs). However, we emphasize that FID is not representative when  $w > 1$ . In this case, IT maps learn by construction only a part of the target measure  $\mathbb{Q}$ . At the same time, FID tests how well the transported samples represent the entire target distribution and is very sensitive to mode dropping [48, Fig. 1b]. Therefore, while the cost decreases with the growth of  $w$ , FID, on the contrary, increases. This is expected since IT maps to smaller part of  $\mathbb{Q}$ . Importantly, the visual quality of the translated images  $\hat{T}(x)$  is not decreasing.
$^{3}$ kaggle.com/reitanaka/alignedanimefaces
kaggle.com/datasets/defileroff/comic-faces-paired-synthetic-v2
In Appendix C, we compare our IT method with other image-to-image translation methods and show that IT better preserves the input-output similarity.
6 Potential Impact
Inequality constraints for generative models. The majority of optimization objectives in generative models (GANs, diffusion models, normalizing flows, etc.) enforce the equality constraints, e.g.,  $T \sharp \mathbb{P} = \mathbb{Q}$ , where  $T \sharp \mathbb{P}$  is the generated measure and  $\mathbb{Q}$  is the data measure. Our work demonstrates that it is possible to enforce inequality constraints, e.g.,  $T \sharp \mathbb{P} \leq w\mathbb{Q}$ , and apply them to a large-scale problem. While in this work we primarily focus on the image-to-image translation task, we believe that the ideas presented in our paper have several visible prospects for further improvement and applications. We list them below.
(1) Partial OT. Learning alignments between measures of unequal mass is a pervasive topic which has already perspective applications in biology to single-cell data [66, 47, 19]. The mentioned works use unbalanced OT [14]. This is an unconstrained problem where the mass spread is softly controlled by the regularization. Therefore, may be hard to control how the mass is actually distributed. Using partial OT which enforces hard inequality constraints might potentially soften this issue. Our IT problem (12) is a particular case of partial OT (4). We believe that our study is useful for future development of partial OT methods.
(2) Generative nearest neighbors. NNs play an important role in machine learning applications such as, e.g., image retrieval [6]. These methods typically rely on fast discrete approximate NN [50, 35] and perform matching with the latent codes of the train samples. In contrast, nowadays, with the rapid growth of large generative models such as DALL-E [58], CLIP [57], GPT-3 [9], it becomes relevant to perform out-of-sample estimation, e.g., map the latent vectors to new vectors which are not present in the train set to generate new data. Our IT approach (for  $w \to \infty$ ) is a theoretically justified way to learn approximate NN maps exclusively from samples. We think our approach might acquire applications here as well, especially since there already exist ways to apply OT in latent spaces of such models [20].
(3) Robustness and outlier detection. Our IT aligns the input measure  $\mathbb{P}$  only with a part of the target measure  $\mathbb{Q}$ . This property might be potentially used to make the learning robust, e.g., ignore outliers in the target dataset. Importantly, the choice of outliers and contamination level are tunable via  $c(x,y)$  and  $w$ , but their selection may be not obvious. At the same time, the potential  $f^{*}$  vanishes on the outliers, i.e., samples in  $\operatorname{Supp}(\mathbb{Q})$  to which the mass is not transported.
Proposition 4 (The potential vanishes at outliers). Under the assumptions of Theorem 5, the equality  $f^{*}(y) = 0$  holds for all  $y \in \operatorname{Supp}(\mathbb{Q}) \setminus \operatorname{Supp}(T^{*}\sharp \mathbb{P})$ .
We empirically illustrate this statement in Appendix A. As a result of this proposition, a possible byproduct of our method is an outlier-score  $f^{*}(y)$  for the target data. Such applications of OT are promising and there already exist works [52, 7, 54] developing approaches to make OT more robust.
Limitations, societal impact. We discuss limitations, societal impact of our study in Appendix A.
ACKNOWLEDGEMENTS. This work was partially supported by Skoltech NGP Program (Skoltech-MIT joint project).
[1] D. Aggarwal, E. Valiyev, F. Sener, and A. Yao. Learning style compatibility for furniture. In German Conference on Pattern Recognition, pages 552-566. Springer, 2018.
[2] C. D. Aliprantis and K. C. Border. Infinite dimensional analysis. Technical report, Springer, 2006.
[3] A. Alotaibi. Deep generative adversarial networks for image-to-image translation: A review. Symmetry, 12(10):1705, 2020.
[4] M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214-223. PMLR, 2017.
[60] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234-241. Springer, 2015.
[61] L. Rout, A. Korotin, and E. Burnaev. Generative modeling with optimal transport maps. In International Conference on Learning Representations, 2022.
[62] F. Santambrogio. Optimal transport for applied mathematicians. *Birkhäuser*, NY, 55(58-63):94, 2015.
[63] V. Seguy, B. B. Damodaran, R. Flamary, N. Courty, A. Rolet, and M. Blondel. Large scale optimal transport and mapping estimation. In International Conference on Learning Representations, 2018.
[64] F. Terkelsen. Some minimax theorems. Mathematica Scandinavica, 31(2):405-413, 1972.
[65] C. Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.
[66] K. D. Yang and C. Uhler. Scalable unbalanced optimal transport using generative adversarial networks. In International Conference on Learning Representations, 2018.
[67] A. Yu and K. Grauman. Fine-grained visual comparisons with local learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 192-199, 2014.
[68] F. Yu, A. Seff, Y. Zhang, S. Song, T. Funkhouser, and J. Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015.
[69] Y. Yuan, S. Liu, J. Zhang, Y. bing Zhang, C. Dong, and L. Lin. Unsupervised image superresolution using cycle-in-cycle generative adversarial networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 814-81409, 2018.
[70] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.
[71] X. Zhang, D. Zhai, T. Li, Y. Zhou, and Y. Lin. Image inpainting based on deep learning: A review. Information Fusion, 2022.
[72] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223-2232, 2017.
Transport costs. Our theoretical results hold true for any continuous cost function  $c(x,y)$ , but our experimental study uses  $\ell^2$  as it already yields a reasonable performance in many cases. Considering more semantically meaningful costs for image translation, e.g., perceptual [70], is a promising future research direction.
Intersecting supports. ET is the nearest neighbor assignment (§3.1). Using ET may be unreasonable when  $\mathcal{X} = \mathcal{Y}$  and  $\operatorname{Supp}(\mathbb{P})$  intersects with  $\operatorname{Supp}(\mathbb{Q})$ . For example, if  $c(x,y)$  attains minimum over  $y \in \mathcal{Y}$  for a given  $x \in \mathcal{X}$  at  $x = y$ , e.g.,  $c = \ell^2$ , then there exists an ET plan satisfying  $\pi^*(y|x) = \delta_x$  for all  $x \in \operatorname{Supp}(\mathbb{P}) \cap \operatorname{Supp}(\mathbb{Q})$ . It does not move the mass of points  $x$  in this intersection. We provide an illustrative toy 2D example in Fig. 10b.
Limited diversity. It is theoretically impossible to preserve input-output similarity better than ET maps. Still one should understand that in some cases these maps may yield degenerate solutions. In Fig. 10a, we provide a toy 2D example of an IT map  $(w = 8)$  which maps all inputs to nearly the same point. In Fig. 9a (texture  $\rightarrow$  chair translation), we see that with the increase of  $w$  the IT map produces less small chairs but more large armchairs. In particular, when  $w = 8$ , only armchairs appear, see Fig. 9a. This is because they are closer (in  $\ell^2$ ) to textures due to having smaller white background area.
(a) Limited diversity. The true ET map is degenerate: it maps the entire  $\mathbb{P}$  to a single vertex of the triangle  $\mathbb{Q}$ . The example shows the learned IT map with high  $w = 20$  approximating the ET map.
(b) Intersecting supports. The true ET map is the identity map: it does not move the probability mass of  $\mathbb{P}$ . The example shows the learned IT map with high  $w = 8$  approximating the ET map.
Figure 10: Toy 2D examples showing two (potential) limitations of IT maps.
Unused samples. Doing experiments, we noticed that the model training slows down with the increase of  $w$ . A possible cause of this is that some samples from  $\mathbb{Q}$  become non-informative for training (this follows from our Proposition 4). Intuitively, the part of  $\operatorname{Supp}(\mathbb{Q})$  to which the samples of  $\mathbb{P}$  will not be mapped to is not informative for training. We illustrate this effect on toy 'Wi-Fi' example and plot the histogram of values of  $f^{*}$  in Fig. 11. One possible negative of this observation is that the training of IT maps or, more generally, partial OT maps, may naturally require larger training datasets.
(a) Surface of  $f^{*}$  (trained with  $w = 1$ ).
(b) Surface of  $f^{*}$  (trained with  $w = \frac{3}{2}$ ).
(c) Surface of  $f^*$  (trained with  $w = 3$ ).
(d) Histograms of values of  $f^{*}(y)$  for  $y\sim \mathbb{Q}$
Figure 11: Illustration the unused samples. In Figures 11a, 11b, 11c, we visualize the surface of the learned potential  $f^{*}$  on the 'Wi-Fi' example and  $w \in \{1, \frac{3}{2}, 3\}$ . In Figures 11b, 11c, the potential vanishes on the arcs of  $\mathbb{Q}$  to which the mass of  $\mathbb{P}$  is not mapped, i.e.,  $f^{*}(y) = 0$ . In Figure 11d, we plot the distribution of values of  $f^{*}(y)$  for  $y \sim \mathbb{Q}$ . For  $w \in \{\frac{3}{2}, 3\}$ , we see large pikes around  $f^{*}(y) = 0$  further demonstrating that the potential equals to zero on a certain part of  $\mathbb{Q}$ .
Limited quantitative metrics. In experiments (§5), we use a limited amount of quantitative metrics. This is because existing unpaired metrics, e.g., FID [32], KID [8], etc., are not suitable for our setup. They aim to test equalities, such as  $T \sharp \mathbb{P} = \mathbb{Q}$ , while our learned maps disobey it by the construction (they capture only a part of  $\mathbb{Q}$ ). Developing quality metrics for partial generative models is an important future research direction. Meanwhile, for our method, we have provided a toy 2D analysis (§5.1) and explanatory metrics (§5.2), such as the transport cost (Table 1a).
Inexistent IT maps. The actual IT plans between  $\mathbb{P},\mathbb{Q}$  may be non-deterministic, while our approach only learns a deterministic map  $T$ . Nevertheless, thanks to our Proposition 3, for every  $\epsilon >0$  there always exists a 1-to-1 map  $T_{\epsilon}\sharp \mathbb{P}\leq w\mathbb{Q}$  which provides  $\epsilon$ -sub-optimal cost  $\int_{\mathcal{X}}c\big(x,T_{\epsilon}(x)\big)d\mathbb{P}(x)\leq$ $\mathrm{Cost}_w(\mathbb{P},\mathbb{Q}) + \epsilon$ . Thus, IT cost (12) can be approached arbitrary well with deterministic transport maps. A potential way to modify our algorithm is to learn stochastic plans is to add random noise  $z$  to generator  $T(x,z)$  as input, although this approach may suffer from ignoring  $z$ , see [43, §5.1].