SlowGuess/NeurIPS_2023_OCR huggingface dataset:
1. Does UOTM offer more robustness to outliers than OT-based model (OTM)? (§5.1)
2. Does the UOT map accurately match the source distribution to the target distribution? (§5.2)
3. Does UOTM provide stable and fast convergence? (§5.3)
4. Does UOTM provide decent performance across various choices of  $\Psi_1^*$  and  $\Psi_2^*$ ? (§5.4)
5. Does UOTM show reasonable performance even without a regularization term  $\mathcal{L}_{reg}$ ? (§5.4)
6. Does UOTM provide decent performance across various choices of  $\tau$  in  $c(x,y)$ ? (§5.4)
Unless otherwise stated, we set  $\Psi \coloneqq \Psi_{1} = \Psi_{2}$  and  $D_{\Psi}$  as a KL divergence in our UOTM model (Eq 9). The source distribution  $\mu$  is a standard Gaussian distribution  $\mathcal{N}(\mathbf{0},\mathbf{I})$  with the same dimension as the target distribution  $\nu$ . In this case, the entropy function  $\Psi$  is given as follows:
$$
\Psi (x) = \left\{ \begin{array}{l l} x \log x - x + 1, & \text {i f} x > 0 \\ \infty , & \text {i f} x \leq 0 \end{array} , \right. \quad \Psi^ {*} (x) = e ^ {x} - 1. \tag {13}
$$
In addition, when the generator  $T_{\theta}$  and potential  $v_{\phi}$  are parameterized by the same neural networks as OTM* [61], we call them a small model. When we use the same network architecture in RGM
Table 1: Target Distribution Matching Test. UOTM achieves a better approximation of target distribution  $\nu$ , i.e.,  $T_{\#}\mu \approx \nu$ .  $\dagger$  indicates the results conducted by ourselves.
Model	Toy (τ = 0.1)	Toy (τ = 0.02)	CIFAR-10
Metric	DKL(T#μ|ν) (↓)	FID (↓)
OTM†	0.05	0.05	7.68
Fixed-μ†	0.02	0.004	7.53
UOTM†	0.02	0.005	2.97
Table 3: Image Generation on CelebA-HQ.
Class	Model	FID (↓)
Diffusion	Score SDE (VP) [69]	7.23
Probability Flow [69]	128.13
LSGM [74]	7.22
UDM [37]	7.16
DDGAN [80]	7.64
RGM [13]	7.15
GAN	PGGAN [34]	8.03
Adv. LAE [59]	19.2
VQ-GAN [16]	10.2
DC-AE [55]	15.8
StyleSwin [83]	3.25
VAE	NVAE [73]	29.7
NCP-VAE [4]	24.8
VAEBM [79]	20.4
OT-based	UOTM†	6.36
[13], we refer to them as large model. Unless otherwise stated, we consider the large model. For implementation details of experiments, please refer to Appendix B.2.
Table 2: Image Generation on CIFAR-10.
Class	Model	FID (↓)	IS (↑)
GAN	SNGAN+DGflow [5]	9.62	9.35
AutoGAN [24]	12.4	8.60
TransGAN [31]	9.26	9.02
StyleGAN2 w/o ADA [35]	8.32	9.18
StyleGAN2 w/ ADA [35]	2.92	9.83
DDGAN (T=1)[80]	16.68	-
DDGAN [80]	3.75	9.63
RGM [13]	2.47	9.68
Diffusion	NCSN [68]	25.3	8.87
DDPM [30]	3.21	9.46
Score SDE (VE) [69]	2.20	9.89
Score SDE (VP) [69]	2.41	9.68
DDIM (50 steps) [67]	4.67	8.78
CLD [15]	2.25	-
Subspace Diffusion [32]	2.17	9.94
LSGM [74]	2.10	9.87
VAE&EBM	NVAE [73]	23.5	7.18
Glow [38]	48.9	3.92
PixelCNN [75]	65.9	4.60
VAEBM [79]	12.2	8.43
Recovery EBM [22]	9.58	8.30
OT-based	WGAN [6]	55.20	-
WGAN-GP[27]	39.40	6.49
Robust-OT [8]	21.57	-
AE-OT-GAN [3]	17.10	7.35
OTM* (Small) [61]	21.78	-
OTM (Large)†	7.68	8.50
UOTM (Small)†	12.86	7.21
UOTM (Large)†	2.97±0.07	9.68
5.1 Outlier Robustness of UOTM
One of the main features of UOT is its robustness against outliers. In this subsection, we investigate how this robustness is reflected in generative modeling tasks by comparing our UOTM and OT-based model (OTM). For evaluation, we generated two datasets that have  $1 - 2\%$  outliers: Toy and Image datasets. The toy dataset is a mixture of samples from  $\mathcal{N}(1,0.5^2)$  (in-distribution) and  $\mathcal{N}(-1,0.5^2)$  (outlier). Similarly, the image dataset is a mixture of CIFAR-10 [43] (in-distribution) and MNIST [14] (outlier) as in Balaji et al. [8].
Figure 2 illustrates the learned transport map  $T$  and the probability density  $T_{\#} \mu$  of generated distribution for OTM and UOTM models trained on the toy dataset. The density in Fig 2a demonstrates that OTM attempts to fit both the in-distribution and outlier samples simultaneously. However, this attempt causes undesirable behavior of transporting density outside the target support. In other words, in order to address  $1 - 2\%$  of outlier samples, OTM generates additional failure samples that do not belong to the in-distribution or outlier distribution. On the other hand, UOTM focuses on matching the in-distribution samples (Fig 2b). Moreover, the transport maps show that OTM faces challenging optimization. The right-hand side of Figure 2a and 2b visualize the correspondence between the source domain samples  $x$  and the corresponding target domain samples  $T(x)$ . Note that the optimal  $T^*$  should be a monotone-increasing function. However, OTM failed to learn such a transport map  $T$ , while UOTM succeeded.
Figure 3 shows the generated samples from OTM and UOTM models on the image dataset. A similar phenomenon is observed in the image data. Some of the generative images from OTM show some unintentional artifacts. Also, MNIST-like generated images display a red or blue-tinted background. In contrast, UOTM primarily generates in-distribution samples. In practice, UTOM generates MNIST-like data at a very low probability of  $0.2\%$ . Notably, UOTM does not exhibit artifacts as in OTM. Furthermore, for quantitative comparison, we measured Fréchet Inception Distance (FID) score [29] by collecting CIFAR-10-like samples. Then, we compared this score with the model trained on a clean
(b) Fixed-  $\mu$
Figure 4: Visualization of OT Map  $(x, T(x))$  trained on clean Toy data. The comparison suggests that Fixed- $\mu$  and UOTM models are closer to the optimal transport map compared to OTM.
(c) UOTM
dataset. The presence of outliers affected OTM by increasing FID score over 6 (7.68 → 13.82) while the increase was less than 2 in UOTM (2.97 → 4.56). In summary, the experiments demonstrate that UOTM is more robust to outliers.
5.2 UOTM as Generative Model
Target Distribution Matching As aforementioned in Sec 3.3, UOT allows some flexibility to the marginal constraints of OT. This means that the optimal generator  $T_{\theta}^{\star}$  does not necessarily transport the source distribution  $\mu$  precisely to the target distribution  $\nu$ , i.e.,  $T_{\#} \mu \approx \nu$ . However, the goal of the generative modeling task is to learn the target distribution  $\nu$ . In this regard, we assessed whether our UOTM could accurately match the target (data) distribution. Specifically, we measured KL divergence [78] between the generated distribution  $T_{\#} \mu$  and data distribution  $\nu$  on the toy dataset (See the Appendix B.1 for toy dataset details). Also, we employed the FID score for CIFAR-10 dataset, because FID assesses the Wasserstein distance between the generated samples and training data in the feature space of the Inception network [70].
In this experiment, our UOTM model is compared with two other methods: OTM [18] (Constraints in both margins) and UOTM with fixed  $\mu$  (Constraints only in the source distribution). We introduced Fixed-  $\mu$  variant because generative models usually sample directly from the input noise distribution  $\mu$ . This direct sampling implies the hard constraint on the source distribution. Note that this hard constraint corresponds to setting  $\Psi_1^* (x) = x$  in Eq 9 (Remark 3.1). (See Appendix B.2 for implementation details of UOTM with fixed  $\mu$ .)
Table 1 shows the data distribution matching results. Interestingly, despite the soft constraint, our UOTM matches data distribution better than OTM in both datasets. In the toy dataset, UOTM achieves similar KL divergence with and without fixed  $\mu$  for each  $\tau$ , which is much smaller than OTM. This result can be interpreted through the theoretical properties in Sec 3.3. Following Thm 3.3, both UOTM models exhibit smaller  $D_{KL}(T_{\#}\mu|\nu)$  for the smaller  $\tau$  in the toy dataset. It is worth noting that, unlike UOTM,  $D_{KL}(T_{\#}\mu|\nu)$  does not change for OTM. This is because, in the standard OT problem, the optimal transport map  $\pi^{\star}$  does not depend on  $\tau$  in Eq 1. Moreover, in CIFAR-10, while Fixed- $\mu$  shows a similar FID score to OTM, UOTM significantly outperforms both models. As discussed in Thm 3.4, we interpret that relaxing both marginals improved the stability of the potential network  $v$ , which led to better convergence of the model on the more complex data. This convergence benefit can be observed in the learned transport map in Fig 4. Even on the toy dataset, the UTOM transport map presents a better-organized correspondence between source and target samples than OTM and Fixed- $\mu$ . In summary, our model is competitive in matching the target distribution for generative modeling tasks.
Image Generation We evaluated our UOTM model on the two generative model benchmarks: CIFAR-10 [43]  $(32 \times 32)$  and CelebA-HQ [46]  $(256 \times 256)$ . For quantitative comparison, we adopted FID [29] and Inception Score (IS) [62]. The qualitative performance of UOTM is depicted in Fig 1. As shown in Table 2, UOTM model demonstrates state-of-the-art results among existing OT-based methods, with an FID of 2.97 and IS of 9.68. Our UOTM outperforms the second-best performing OT-based model OTM(Large), which achieves an FID of 7.68, by a large margin. Moreover, UOTM(Small) model surpasses another UOT-based model with the same backbone network (Robust-OT), which achieves an FID of 21.57. Furthermore, our model achieves the state-of-the-art FID score of 6.36 on CelebA-HQ  $(256 \times 256)$ . To the best of our knowledge, UOTM is the first OT-based generative model that has shown comparable results with state-of-the-art models in various datasets.
Figure 5: FID Scores during Training on CIFAR-10.
Figure 6: Ablation Study on Regularizer Intensity  $\lambda$ .
Figure 7: Ablation Study on  $\tau$  in  $c(x,y) = \tau \| x - y\| _2^2$
5.3 Fast Convergence
The discussion in Thm 3.4 suggests that UOTM offers some optimization benefits over OTM, such as faster and more stable convergence. In addition to the transport map visualization in the toy dataset (Fig 2 and 4), we investigate the faster convergence of UOTM on the image dataset. In CIFAR-10, UOTM converges in 600 epochs, whereas OTM takes about 1000 epochs (Fig 5). To achieve the same performance as OTM, UOTM needs only 200 epochs of training, which is about five times faster. In addition, compared to several approaches that train CIFAR-10 with NCSN++ (large model) [69], our model has the advantage in training speed; Score SDE [69] takes more than 70 hours for training CIFAR-10, 48 hours for DDGAN [80], and 35-40 hours for RGM on four Tesla V100 GPUs. OTM takes approximately 30-35 hours to converge, while our model only takes about 25 hours.
Generalized  $\Psi_{1}, \Psi_{2}$  We investigate the various choices of Csiszar divergence in UOT problem, i.e.,  $\Psi_{1}^{*}$  and  $\Psi_{2}^{*}$  in Eq 9. As discussed in Remark 3.2, the necessary condition of feasible  $\Psi^{*}$  is non-decreasing, convex, and differentiable functions. For instance, setting the Csiszar divergence  $D_{\Psi}$  as  $f$ -divergence families such as KL divergence,  $\chi^2$  divergence, and Jensen-Shannon divergence satisfies the aforementioned conditions. For practical optimization, we chose  $\Psi^{*}(x)$  that gives finite values for all  $x$ , such as KL divergence (Eq 13) and  $\chi^2$  divergence (Eq 14):
Table 4: Ablation Study on Csiszar Divergence  ${D}_{{\Psi }_{i}}\left( {\cdot  \mid   \cdot  }\right)$  .
(Ψ1, Ψ2)	FID
(KL,KL)	2.97
(χ2, χ2)	3.02
(KL, χ2)	3.21
(χ2, KL)	2.78
(Softplus, Softplus)	3.17
$$
\Psi (x) = \left\{ \begin{array}{l l} (x - 1) ^ {2}, & \text {i f} x \geq 0 \\ \infty , & \text {i f} x <   0 \end{array} , \right. \quad \Psi^ {*} (x) = \left\{ \begin{array}{l l} \frac {1}{4} x ^ {2} + x, & \text {i f} x \geq - 2 \\ - 1, & \text {i f} x <   - 2 \end{array} . \right. \tag {14}
$$
We assessed the performance of our UOTM models for the cases where  $D_{\Psi}$  is KL divergence or  $\chi^2$  divergence. Additionally, we tested our model when  $\Psi^{*} = \mathrm{Softplus}$  (Eq 37), which is a direct parametrization of  $\Psi^{*}$ . Table 4 shows that our UOTM model achieves competitive performance across all five combinations of  $(\Psi_1^*, \Psi_2^*)$ .
$\lambda$  in Regularizer  $\mathcal{L}_{reg}$  We conduct an ablation study to investigate the effect of the regularization term  $\mathcal{L}_{reg}$  on our model's performance. The WGAN family is known to exhibit unstable training dynamics and to highly depend on the regularization term [6, 27, 51, 56]. Similarly, Figure 6 shows that OTM is also sensitive to the regularization term and fails to converge without it (OTM shows an FID score of 152 without regularization. Hence, we excluded this result in Fig 6 for better visualization). In contrast, our model is robust to changes in the regularization hyperparameter and produces reasonable performance even without the regularization.
$\tau$  in the Cost Function We performed an ablation study on  $\tau$  in the cost function  $c(x,y) = \tau \| x - y\| _2^2$ . In our model, the hyperparameter  $\tau$  controls the relative weight between the cost  $c(x,y)$  and the marginal matching  $D_{\Psi_1},D_{\Psi_2}$  in Eq 4. In Fig 7, our model maintains a decent performance of  $\mathrm{FID}(\leq 5)$  for  $\tau = \{0.5,1,2\} \times 10^{-3}$ . However, the FID score sharply degrades at  $\tau = \{0.1,5\} \times 10^{-3}$ . Thm 3.3 suggests that a smaller  $\tau$  leads to a tighter distribution matching. However, this cannot explain the degradation at  $\tau = 0.1\times 10^{-3}$ . We interpret this is because the cost function provides some regularization effect that prevents a mode collapse of the model (See the Appendix C.2 for a detailed discussion).
In this paper, we proposed a generative model based on the semi-dual form of Unbalanced Optimal Transport, called UOTM. Our experiments demonstrated that UOTM achieves better target distribution matching and faster convergence than the OT-based method. Moreover, UOTM outperformed existing OT-based generative model benchmarks, such as CIFAR-10 and CelebA-HQ-256. The potential negative societal impact of our work is that the Generative Model often learns the dependence in the semantics of data, including any existing biases. Hence, deploying a Generative Model in real-world applications requires careful monitoring to prevent the amplification of existing societal biases present in the data. It is important to carefully control the training data and modeling process of Generative Models to mitigate potential negative societal impacts.
This work was supported by KIAS Individual Grant [AP087501] via the Center for AI and Natural Sciences at Korea Institute for Advanced Study, the NRF grant [2012R1A2C3010887] and the MSIT/IITP ([1711117093], [2021-0-00077], [No. 2021-0-01343, Artificial Intelligence Graduate School Program(SNU)]).
[1] Brandon Amos, Lei Xu, and J Zico Kolter. Input convex neural networks. In International Conference on Machine Learning, pages 146-155. PMLR, 2017.
[2] Dongsheng An, Yang Guo, Na Lei, Zhongxuan Luo, Shing-Tung Yau, and Xianfeng Gu. Ae-ot: A new generative model based on extended semi-discrete optimal transport. *ICLR* 2020, 2019.
[3] Dongsheng An, Yang Guo, Min Zhang, Xin Qi, Na Lei, and Xianfang Gu. Ae-ot-gan: Training gans from data specific latent distribution. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVI 16, pages 548–564. Springer, 2020.
[4] Jyoti Aneja, Alex Schwing, Jan Kautz, and Arash Vahdat. A contrastive learning approach for training variational autoencoder priors. Advances in neural information processing systems, 34: 480-493, 2021.
[5] Abdul Fatir Ansari, Ming Liang Ang, and Harold Soh. Refining deep generative models via discriminator gradient flow. arXiv preprint arXiv:2012.00780, 2020.
[6] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pages 214-223. PMLR, 2017.
[7] Shiri Artstein-Avidan and Vitali Milman. The concept of duality in convex analysis, and the characterization of the legendre transform. Annals of mathematics, pages 661-674, 2009.
[8] Yogesh Balaji, Rama Chellappa, and Soheil Feizi. Robust optimal transport with applications in generative modeling and domain adaptation. Advances in Neural Information Processing Systems, 33:12934-12944, 2020.
[9] Dimitri Bertsekas. Convex optimization theory, volume 1. Athena Scientific, 2009.
[10] Dimitri Bertsekas and Steven E Shreve. Stochastic optimal control: the discrete-time case, volume 5. Athena Scientific, 1996.
[11] Laetitia Chapel, Rémi Flamary, Haoran Wu, Cédric Févotte, and Gilles Gasso. Unbalanced optimal transport through non-negative penalized linear regression. Advances in Neural Information Processing Systems, 34:23270-23282, 2021.
[12] Lenaic Chizat, Gabriel Peyré, Bernhard Schmitzer, and François-Xavier Vialard. Unbalanced optimal transport: Dynamic and kantorovich formulations. Journal of Functional Analysis, 274 (11):3090-3123, 2018.
[13] Jaemoo Choi, Yesom Park, and Myungjoo Kang. Restoration based generative models. In Proceedings of the 40th International Conference on Machine Learning, volume 202. PMLR, 2023.
[74] Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. Advances in Neural Information Processing Systems, 34:11287-11302, 2021.
[75] Aaron Van Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In International conference on machine learning, pages 1747-1756. PMLR, 2016.
[76] Cédric Villani. Topics in optimal transportation, volume 58. American Mathematical Soc., 2021.
[77] Cédric Villani et al. Optimal transport: old and new, volume 338. Springer, 2009.
[78] Qing Wang, Sanjeev R Kulkarni, and Sergio Verdu. Divergence estimation for multidimensional densities via  $k$ -nearest-neighbor distances. IEEE Transactions on Information Theory, 55(5): 2392-2405, 2009.
[79] Zhisheng Xiao, Karsten Kreis, Jan Kautz, and Arash Vahdat. Vaebm: A symbiosis between variational autoencoders and energy-based models. arXiv preprint arXiv:2010.00654, 2020.
[80] Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemma with denoising diffusion gans. arXiv preprint arXiv:2112.07804, 2021.
[81] Yujia Xie, Minshuo Chen, Haoming Jiang, Tuo Zhao, and Hongyuan Zha. On scalable and efficient computation of large scale optimal transport. In International Conference on Machine Learning, pages 6882-6892. PMLR, 2019.
[82] KD Yang and C Uhler. Scalable unbalanced optimal transport using generative adversarial networks. In International Conference on Learning Representations, 2019.
[83] Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang, and Baining Guo. Styleswin: Transformer-based gan for high-resolution image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11304-11314, 2022.
Notations and Assumptions Let  $\mathcal{X}$  and  $\mathcal{Y}$  be compact complete metric spaces which are subsets of  $\mathbb{R}^d$ , and  $\mu, \nu$  be positive Radon measures of the mass 1. Let  $\Psi_1$  and  $\Psi_2$  be a convex, differentiable, nonnegative function defined on  $\mathbb{R}$ . We assume that  $\Psi_1(0) = \Psi_2(0) = 1$ , and  $\Psi_1(x) = \Psi_2(x) = \infty$  for  $x < 0$ . The convex conjugate of  $\Psi$  is denoted by  $\Psi^*$ . We assume that  $\Psi_1^*$  and  $\Psi_2^*$  are differentiable and non-decreasing on its domain. Moreover, let  $c$  be a quadratic cost,  $c(x,y) = \tau \| x - y\|_2^2$ , where  $\tau$  is positive constant.
First, for completeness, we present the derivation of the dual and semi-dual formulations of the UOT problem.
Theorem A.1 ([12, 21, 72]). The dual formulation of Unbalanced OT (4) is given by
$$
C _ {u b} (\mu , \nu) = \sup  _ {u (x) + v (y) \leq c (x, y)} \left[ \int_ {\mathcal {X}} - \Psi_ {1} ^ {*} (- u (x)) d \mu (x) + \int_ {\mathcal {Y}} - \Psi_ {2} ^ {*} (- v (y)) d \nu (y) \right], \tag {15}
$$
where  $u\in \mathcal{C}(\mathcal{X}),v\in \mathcal{C}(\mathcal{Y})$  with  $u(x) + v(y)\leq c(x,y)$ . Note that strong duality holds.
Also, the semi-dual formulation of Unbalanced OT (4) is given by
$$
C _ {u b} (\mu , \nu) = \sup  _ {v \in \mathcal {C}} \left[ \int_ {\mathcal {X}} - \Psi_ {1} ^ {*} (- v ^ {c} (x))) d \mu (x) + \int_ {\mathcal {Y}} - \Psi_ {2} ^ {*} (- v (y)) d \nu (y) \right], \tag {16}
$$
where  $v^{c}(x)$  denotes the  $c$ -transform of  $v$ , i.e.,  $v^{c}(x) = \inf_{y\in \mathcal{V}}(c(x,y) - v(y))$ .
Proof. The primal problem Eq 4 can be rewritten as
$$
\inf  _ {\pi \in \mathcal {M} _ {+} (\mathcal {X} \times \mathcal {Y})} \int_ {\mathcal {X} \times \mathcal {Y}} c (x, y) d \pi (x, y) + \int_ {\mathcal {X}} \Psi_ {1} \left(\frac {\pi_ {0} (x)}{d \mu (x)}\right) d \mu (x) + \int_ {\mathcal {Y}} \Psi_ {2} \left(\frac {\pi_ {1} (y)}{d \nu (y)}\right) d \nu (y). \tag {17}
$$
We introduce slack variables  $P$  and  $Q$  such that  $P = \pi_0$  and  $Q = \pi_1$ . By Lagrange multipliers where  $u$  and  $v$  associated to the constraints, the Lagrangian form of Eq 17 is
$$
\begin{array}{l} \mathcal {L} (\pi , P, Q, u, v) := \int_ {\mathcal {X} \times \mathcal {Y}} c (x, y) d \pi (x, y) (18) \\ + \int_ {\mathcal {X}} \Psi_ {1} \left(\frac {d P (x)}{d \mu (x)}\right) d \mu (x) + \int_ {\mathcal {Y}} \Psi_ {2} \left(\frac {d Q (y)}{d \nu (y)}\right) d \nu (y) (19) \\ + \int_ {\mathcal {X}} u (x) \left(d P (x) - \int_ {\mathcal {Y}} d \pi (x, y)\right) + \int_ {\mathcal {Y}} v (y) \left(d Q (y) - \int_ {\mathcal {X}} d \pi (x, y)\right). (20) \\ \end{array}
$$
The dual Lagrangian function is  $g(u,v) = \inf_{\pi ,P,Q}\mathcal{L}(\pi ,P,Q,u,v)$ . Then,  $g(u,v)$  reduces into three distinct minimization problems for each variables;
$$
\begin{array}{l} g (u, v) = \inf  _ {P} \left[ \int_ {X} u (x) d P (x) + \Psi_ {1} \left(\frac {d P (x)}{d \mu (x)}\right) d \mu (x) \right] \\ + \inf  _ {Q} \left[ \int_ {Y} v (y) d Q (y) + \Psi_ {2} \left(\frac {d Q (y)}{d \nu (y)}\right) d \nu (y) \right] \tag {21} \\ + \inf _ {\pi} \left[ \int_ {\mathcal {X} \times \mathcal {Y}} \left(c (x, y) - u (x) - v (y)\right) d \pi (x, y) \right]. \\ \end{array}
$$
Note that the first term of Eq 21 is
$$
- \sup  _ {P} \left[ \int_ {\mathcal {X}} \left[ - u (x) \frac {d P (x)}{d \mu (x)} - \Psi_ {1} \left(\frac {d P (x)}{d \mu (x)}\right) \right] d \mu (x) \right] = - \int_ {\mathcal {X}} \Psi_ {1} ^ {*} (- u (x)) d \mu (x). \tag {22}
$$
Thus, Eq 21 can be rewritten as follows:
$$
\begin{array}{l} g (u, v) = \inf  _ {\pi} \left[ \int_ {\mathcal {X} \times \mathcal {Y}} \left(c (x, y) - u (x) - v (y)\right) d \pi (x, y) \right] \tag {23} \\ - \int_ {\mathcal {X}} \Psi_ {1} ^ {*} (- u (x)) d \mu (x) - \int_ {\mathcal {Y}} \Psi_ {2} ^ {*} (- v (x)) d \nu (x). \\ \end{array}
$$
Suppose that  $c(x,y) - u(x) - v(y) < 0$  for some point  $(x,y)$ . Then, by concentrating all mass of  $\pi$  into the point, the value of the minimization problem of Eq 23 becomes  $-\infty$ . Thus, to avoid the trivial solution, we should set  $c(x,y) - u(x) - v(y) \geq 0$  almost everywhere. Within the constraint, the solution of the optimization problem of Eq 23 is  $c(x,y) = u(x) + v(y)$ $\pi$ -almost everywhere. Finally, we obtain the dual formulation of UOT:
$$
\sup  _ {u (x) + v (y) \leq c (x, y)} \left[ \int_ {\mathcal {X}} - \Psi_ {1} ^ {*} (- u (x)) d \mu (x) + \int_ {\mathcal {Y}} - \Psi_ {2} ^ {*} (- v (y)) d \nu (y) \right], \tag {24}
$$
where  $c(x,y) = u(x) + v(y)$ $\pi$  -almost everywhere (since  $\Psi_1^*$  and  $\Psi_{2}^{*}$  is non-decreasing). In other words,
$$
\sup  _ {(u, v) \in \mathcal {C} (\mathcal {X}) \times \mathcal {C} (\mathcal {Y})} \left[ \int_ {\mathcal {X}} - \Psi_ {1} ^ {*} (- u (x)) d \mu (x) + \int_ {\mathcal {Y}} - \Psi_ {2} ^ {*} (- v (y)) d \nu (y) - \iota (u + v \leq c) \right], \tag {25}
$$