Paper: Analyzing and Improving Optimal-Transport-based Adversarial Networks (ICLR-2024)

Reviewer comment (Novelty):
The proposed scheduling scheme is a very simple idea which proved to be useful for a basic experiment in an extremely low dimensional setting.

Author response:
We evaluated our UOTM-SD model on the high-resolution dataset, **CelebA-HQ ($256 \times 256$)**. Due to limitations in time and resources, we conducted one experiment for each model with NCSN++ backbone: UOTM-SD with Linear scheduling ($g_1=g_2=SP$), UOTM ($g_1=g_2=SP$), and OTM. We selected these three models as they represent competitive options among OT-based adversarial networks on CIFAR-10 (Table 3). The FID scores below demonstrate our UOTM-SD model outperforms the other two OT map models.

|Model|FID ($\downarrow$)|
|:---|:---|
|OTM|13.56|
|UOTM (SP)|9.78|
|UOTM-SD (Linear)|**8.19**|

The training hyperparameters are as follows: the cost intensity $\tau=0.00001$, R1 regularization intensity $\lambda=5$, Scheduling intensity $(\alpha_{min}, \alpha_{max})=(1/5, 5)$.

Reviewer comment (Experiments):
There is uncertainty about whether this method can be scaled to high dimensional settings easily.

Author response:
We evaluated our UOTM-SD model on the high-resolution dataset, **CelebA-HQ ($256 \times 256$)**. Due to limitations in time and resources, we conducted one experiment for each model with NCSN++ backbone: UOTM-SD with Linear scheduling ($g_1=g_2=SP$), UOTM ($g_1=g_2=SP$), and OTM. We selected these three models as they represent competitive options among OT-based adversarial networks on CIFAR-10 (Table 3). The FID scores below demonstrate our UOTM-SD model outperforms the other two OT map models.

|Model|FID ($\downarrow$)|
|:---|:---|
|OTM|13.56|
|UOTM (SP)|9.78|
|UOTM-SD (Linear)|**8.19**|

The training hyperparameters are as follows: the cost intensity $\tau=0.00001$, R1 regularization intensity $\lambda=5$, Scheduling intensity $(\alpha_{min}, \alpha_{max})=(1/5, 5)$.

Reviewer comment (Experiments):
The reviewer is surprised that WGAN didn't work at all for the first toy problem and asks how many hyperparameter settings were tried.

Author response:
We conducted experiments on WGAN with various hyperparameters, including learning rates (1e-4, 2e-4), weight-clipping parameters (0.01, 0.1, 1), and $R_1$ penalization term ($\lambda=0,1,5$). Unfortunately, the model did not exhibit satisfactory convergence. In response to the comment from the reviewer 12Gz, **we conducted experiments on other Toy datasets: Uneven Gaussian Mixture and Moon-to-Spiral. In these datasets, WGAN showed good performance in Fig 10.** The Toy dataset in Fig 5 requires challenging mode coverage for the generative model. We believe this is why WGAN is not successful in Fig 5.

Reviewer comment (Theory):
The reviewer asks what the reason might be for thinking that WGAN just won't work.

Author response:
We conducted experiments on WGAN with various hyperparameters, including learning rates (1e-4, 2e-4), weight-clipping parameters (0.01, 0.1, 1), and $R_1$ penalization term ($\lambda=0,1,5$). Unfortunately, the model did not exhibit satisfactory convergence. In response to the comment from the reviewer 12Gz, **we conducted experiments on other Toy datasets: Uneven Gaussian Mixture and Moon-to-Spiral. In these datasets, WGAN showed good performance in Fig 10.** The Toy dataset in Fig 5 requires challenging mode coverage for the generative model. We believe this is why WGAN is not successful in Fig 5.

Reviewer comment (Writing):
I think it is slightly misleading that using $\Psi$s in eq.(8) that would correspond to $g$s in Algorithm 1. > After discussion to the authors, I concluded that this issue will appear to be less of a problem, and I raised Soundness a little.

Author response:
**A1.**
$g_{1,2}$ in Algorithm 1 correspond to the $\Psi_{1,2}^{\*}$, which represents the **convex conjugate** of $\Psi\_{1,2}$, i.e., $\Psi\_{i}^{\*}(y) = \sup\_{x \in \mathbb{R}}\{\langle x, y \rangle - f(x)\}$ for $\Psi\_{i}:\mathbb{R}\rightarrow [-\infty, \infty]$. Hence, for brevity, we changed the notation from $\Psi_{1,2}^{*}$ to $g_{1,2}$.

Reviewer comment (Experiments):
On the experiments for Lipschitz continuity, Fig 6, the experiments seem to be conducted only with 2d data, and I prefer counterparts of them in training for image generations also.

Author response:
**A2.**
We appreciate the reviewer for the thoughtful comment. **We conducted additional experiments for Lipschitz continuity of OTM and UOTM on CIFAR-10, and added results in Appendix (Fig 11).**
We measured the norm of the potential gradient $\\| \nabla_{y} v_{\phi}(y)\\|, \\| \nabla_{\hat{y}} v_{\phi}(\hat{y})\\|$ at a random real data $y$ and a randomly generated data $\hat{y}$. **The result shows a similar trend as in Fig 6.** The gradient norm of OTM potential is much larger than the gradient norm of UOTM potential throughout training on CIFAR-10.

Reviewer comment (Experiments):
UOTM-SD look working very good in CIFAR-10, but it is not evident that is also good for other domains.

Author response:
**A3.**
We appreciate the reviewer for the thoughtful comment. We evaluated our UOTM-SD model on the high-resolution dataset, **CelebA-HQ ($256 \times 256$)**. Due to limitations in time and resources, we conducted one experiment for each model with NCSN++ backbone: UOTM-SD with Linear scheduling ($g_1=g_2=SP$), UOTM ($g_1=g_2=SP$), and OTM. We selected these three models as they represent competitive options among OT-based adversarial networks on CIFAR-10 (Table 3). The FID scores below demonstrate our UOTM-SD model outperforms the other two OT map models.

|Model|FID ($\downarrow$)|
|:---|:---|
|OTM|13.56|
|UOTM (SP)|9.78|
|UOTM-SD (Linear)|**8.19**|

The training hyperparameters are as follows: the cost intensity $\tau=0.00001$, R1 regularization intensity $\lambda=5$, Scheduling intensity $(\alpha_{min}, \alpha_{max})=(1/5, 5)$.

Reviewer comment (Experiments):
The experiment results look promising as well. It would be great if the proposed method is applied on higher solution image dataset to showcase the image generation quality.

Author response:
We thank the reviewer for the thoughtful comment. We evaluated our UOTM-SD model on the high-resolution dataset, **CelebA-HQ ($256 \times 256$)**. Due to limitations in time and resources, we conducted one experiment for each model with NCSN++ backbone: UOTM-SD with Linear scheduling ($g_1=g_2=SP$), UOTM ($g_1=g_2=SP$), and OTM. We selected these three models as they represent competitive options among OT-based adversarial networks on CIFAR-10 (Table 3). The FID scores below demonstrate our UOTM-SD model outperforms the other two OT map models.


|Model|FID ($\downarrow$)|
|:---|:---|
|OTM|13.56|
|UOTM (SP)|9.78|
|UOTM-SD (Linear)|**8.19**|

The training hyperparameters are as follows: the cost intensity $\tau=0.00001$, R1 regularization intensity $\lambda=5$, Scheduling intensity $(\alpha_{min}, \alpha_{max})=(1/5, 5)$.

Reviewer comment (Theory):
For WGAN explanation in section 3.1, the authors categorize this case as c = 0. I don't think this is the case because the in the original paper, this cost is the L1 Euclidean distance, i.e. c(x, y) = |x-y|. Also, it seems the Lipschitz constraint is missing in this case. It would be great if this part is further clarified.

Author response:
We thank the reviewer for the careful comment. **We would like to clarify that the WGAN explanation in Sec 3.1 is presented in terms of the unified training algorithm (Alg. 1).** If we set $c=0$ in Line 5 and 10 in Alg. 1, the learning objective $\mathcal{L}$ becomes Eq. 4. Note that the L1 Euclidean distance $c(x, y) = \\|x-y\\|$ in WGAN is different from our cost function in Alg. 1. This L1 Euclidean distance is utilized in the definition of the Wasserstein-1 distance. Eq. 4 is derived from the Kantorovich-Rubinstein duality for the Wasserstein-1 distance.

Moreover, the **Lipschitz constraint** is implemented by weight clipping in the original WGAN paper and by gradient penalty in WGAN-GP. We incorporated the implementation details regarding the Lipschitz constraint of WGAN on Page 4 as follows:

> For the vanilla WGAN, we employed a weight clipping strategy for the potential network to impose the Lipschitz constraint, folllowing (Arjovsky et al., 2017). 

Furthermore, we rephrased the italic sentence after Eq. 5 to enhance clarity as follows:

> Note that, if we set $c=0$ and introduce a 1-Lipschitz constraint on $v_{\phi}$, this objective has the same form as WGAN (Eq. 4).

Reviewer comment (Writing):
Same comment for the italic sentence after Eq. 5.

Author response:
We thank the reviewer for the careful comment. **We would like to clarify that the WGAN explanation in Sec 3.1 is presented in terms of the unified training algorithm (Alg. 1).** If we set $c=0$ in Line 5 and 10 in Alg. 1, the learning objective $\mathcal{L}$ becomes Eq. 4. Note that the L1 Euclidean distance $c(x, y) = \\|x-y\\|$ in WGAN is different from our cost function in Alg. 1. This L1 Euclidean distance is utilized in the definition of the Wasserstein-1 distance. Eq. 4 is derived from the Kantorovich-Rubinstein duality for the Wasserstein-1 distance.

Moreover, the **Lipschitz constraint** is implemented by weight clipping in the original WGAN paper and by gradient penalty in WGAN-GP. We incorporated the implementation details regarding the Lipschitz constraint of WGAN on Page 4 as follows:

> For the vanilla WGAN, we employed a weight clipping strategy for the potential network to impose the Lipschitz constraint, folllowing (Arjovsky et al., 2017). 

Furthermore, we rephrased the italic sentence after Eq. 5 to enhance clarity as follows:

> Note that, if we set $c=0$ and introduce a 1-Lipschitz constraint on $v_{\phi}$, this objective has the same form as WGAN (Eq. 4).

Reviewer comment (Experiments):
More of a suggestion: in the experiment part (E.g. Fig. 1, Fig. 5 etc.), it would be great if the authors also include non-DNN based OT maps like the one proposed in An et. al 2019, as their solution is unique and can be found by a convex optimization.

Author response:
Thank you for the valuable comment. We agree with the reviewer that comparing the GT transport map with the OT map models is a meaningful approach to evaluate the quality of transport map from the OT maps models. Due to page constraints, **we included this result in Appendix D.1.** In the Toy datasets, **both UOTM and UOTM-SD succeeded in learning the optimal transport map**, with minor differences to the GT transport map. This GT transport map is discovered through convex optimization [1]. 


[1] Flamary, Rémi, et al. "Pot: Python optimal transport." The Journal of Machine Learning Research 22.1 (2021): 3571-3578.

Reviewer comment (Novelty):
As it is described in Algorithm 1, the proposed framework is a useful framework for exposition and experimental design, but the significance of this contribution is limited as it is a straightforward generalization of the already established lookalike adversarial objectives of Equations 4 and 5. This kind of framework is common in the GAN literature, e.g. in Nagarajan et al. (2017).

Author response:
**Q1.** It is a straightforward generalization of the already established lookalike adversarial objectives of Equations 4 and 5. This kind of framework is common in the GAN literature, e.g. in Nagarajan et al. (2017).

**A1.** 
**We would like to emphasize that the OT map models are derived from a fundamentally different approach when compared to previous GAN models.**

- In GAN models (OT Loss), we consider the OT problem between the generated distribution ($\mu$) and the data distribution ($\nu$). The OT problem is employed to measure the distance between $\mu$ and $\nu$. **The previous works suggesting the generalization of GAN models, such as Nagarajan et al. (2017), only cover these OT Loss GANs.**
- In contrast, OT map models consider the OT problem between the Gaussian distribution ($\mu$) and the data distribution ($\nu$). The OT map itself serves as a generative model.

Our key observation is that, although these models are derived from different approaches, these models can be integrated into a unified adversarial framework. **To the best of our knowledge, our work is the first attempt to propose a unified framework for OT Loss GANs and OT map models.** This framework provides a better understanding of **why recent OT map models exhibit significantly different dynamics compared to OT Loss GANs**.

Reviewer comment (Novelty):
The unified algorithm is not by itself the source of novel insights, or novel models. The proposed UOTM-SD does not necessitate this unified algorithm as it only interpolates between two OT map models using Equation (12).

Author response:
**Q2.** The unified algorithm is not by itself the source of novel insights, or novel models. The proposed UOTM-SD does not necessitate this unified algorithm as it only interpolates between two OT map models using Equation (12).

**A2.** 
Because we focused on the contribution of the unified framework in A1, **we would like to emphasize the contribution of analysis stemming from this framework and its connection to UOTM-SD.**

This unified framework allows us to conduct ablation studies on two building blocks of this framework: (1) Strictly convex $g_1$, $g_2$ and (2) Cost function $c(\cdot, \cdot)$. This comparative analysis uncovers the role of each component in generative modeling.

- The strictly convex $g_1$, $g_2$ helps stabilize the training process over setting $g_1 = g_2 = Id$.
- The cost function helps mitigate the mode collapse problem in the adversarial training framework.

However, simultaneously exploiting these two advantages (UOTM) reveals some limitations. 
- The generative performance is sensitive to the hyperparameter $\tau$.
- The UOT problem (Eq. 6) inherently incurs distribution errors.

In this respect, as discussed in the Motivation paragraph of Sec 4, we introduced UOTM-SD to address these limitations. **Our UOTM-SD minimizes distribution errors by designing a converging sequence of the UOT map towards the OT map through $\alpha$-scheduling (Theorem 4.1.). Moreover, for each $\alpha$, UOTM-SD can leverage the advantages of two building blocks, as UOTM.**

Reviewer comment (Experiments):
The experiments only weakly support the claims of the paper, as the convergence and mode collapse properties are mainly studied in Section 3.2 on a toy dataset, whereas the difference between low and high dimensions can be large when dealing with neural networks. Considering a higher-dimensional structured dataset could strengthen the experimental conclusions. Additionally, another low-dimensional dataset should be included to avoid any bias linked to having a data distribution evenly distributed around the prior distribution.

Author response:
**Q3.** The convergence and mode collapse properties are mainly studied in Section 3.2 on a toy dataset. Yet, the difference between low and high dimensions can be large when dealing with neural networks. Considering a higher-dimensional structured dataset could strengthen the experimental conclusions. 

**A3.** 
We appreciate the reviewer for the thoughtful advice. **In Table 2 and Fig 4, we presented the convergence and mode collapse analyses for the image dataset (CIFAR-10).** In Table 2, we measured FID scores for the convergence analysis. As discussed in Page 5, UOTM w/o cost and UOTM outperform their algorithmic counterparts concerning $g_1$, $g_2$, i.e., WGAN and OTM, respectively. Moreover, we presented the generated samples for each $\tau$ in Fig 4 for mode collapse analysis. As dicussed in Page 6, when $\tau$ is too small, the model exhibits a mode collapse problem, generating similar samples repeatedly.

Reviewer comment (Experiments):
The experiments in Section 3.2 lack a vanilla GAN baseline, especially to conclude on the advantage of the SP function. Similarly, experiments of Figures 5 and 6 miss the WGAN-GP baseline.

Author response:
**Q5** Still on Section 3.2, the experiments lack a vanilla GAN baseline, especially to conclude on advantage of the SP function.

**A5.** 
We would like to emphasize that **the goal of this work is to investigate OT-based adversarial networks, specifically derived from OT Loss GANs and OT map models.** Consequently, while the vanilla GAN could be accommodated within our unified framework (Sec 3.1), we considered the vanilla GAN to be beyond the scope of our current analysis. However, as we discussed in Sec 5, we agree with the reviewer that a comparative analysis with the vanilla GAN could be an interesting future research. This additional analysis could be helpful in investigating the effect of $g_3$. (Note that, except for the vanilla GAN, all OT-based adversarial networks in our analysis set $g_3=Id$).

Moreover, **concerning  $g_1$ and $g_2$, we discussed both advantageous aspects and drawbacks of setting $g_1, g_2$ as the SP function compared to the Identity function, throughout Sec 3.2.** Specifically, employing SP for $g_1, g_2$ contributes to stabilizing the training process through the adaptive gradient updates (Sec 3.2.1) and the Lipschitzness of potential (Sec. 3.2.3). However, the drawback is that the UOTM model (using $g_1 = g_2 = SP$ and the cost function) introduces inherent distribution errors (Eq 11). In this regard, we proposed UOTM-SD to address the drawback while leveraging the advantages.

Reviewer comment (Evaluation):
As a standalone model and with the available information, the comparative advantage of UOTM-SD is not significant enough. The gain in FID is minor and would thus require confidence intervals to be validated. As FID is computed on the training set, there is also a risk of overfitting to be taken into account. Furthermore, other datasets might be considered to test the robustness of the methods to other modalities and data dimensions.

Author response:
**Q7.** The gain in FID is minor and would thus require confidence intervals to be validated. The FID being computed on the training set, there is also a risk of overfitting to be taken into account. Furthermore, other datasets might be considered to test the robustness of the methods to other modalities and data dimensions.

**A7.** We thank the reviewer for the comment. We performed two additional experiments for each UOTM-SD model in Table 3 on CIFAR-10. Our UOTM-SD model demonstrates noticeably improved results comapred to UOTM model, considering the standard deviation of FID scores. The mean and standard deviation of FID scores (mean $\pm$ standard deviation) are as follows:

|Model|FID ($\downarrow$)|
|:---|:---|
|UOTM|$2.99 \pm 0.07$|
|UOTM-SD (Cosine)|$2.57 \pm 0.09$|
|UOTM-SD (Linear)|$2.50 \pm 0.11$|
|UOTM-SD (Step)|$2.77 \pm 0.4$|

Moreover, we evaluated our UOTM-SD model on the high-resolution dataset, **CelebA-HQ ($256 \times 256$)**. Due to limitations in time and resources, we conducted one experiment for each model with NCSN++ backbone: UOTM-SD with Linear scheduling ($g_1=g_2=SP$), UOTM ($g_1=g_2=SP$), and OTM. We selected these three models as they represent competitive options among OT-based adversarial networks on CIFAR-10 (Table 3). The FID scores below demonstrate our UOTM-SD model outperforms the other two OT map models.

|Model|FID ($\downarrow$)|
|:---|:---|
|OTM|13.56|
|UOTM (SP)|9.78|
|UOTM-SD (Linear)|**8.19**|

The training hyperparameters are as follows: the cost intensity $\tau=0.00001$, R1 regularization intensity $\lambda=5$, Scheduling intensity $(\alpha_{min}, \alpha_{max})=(1/5, 5)$.

Reviewer comment (Experiments):
Generators in GANs do not require their latent space to be of the same dimension as their output. Yet, it seems to be the case for OT map models, given that they learn a transport map between two distributions living in the same space. The authors should explicitly explain how this affects their experiments and their results, and whether the provided comparisons are fair between the two types of models in terms of dimensionality and neural network architectures. Both operation modes seem hard to articulate with each other, as Algorithm 1 features the sampling of both a random variable from the same space as the data and another latent variable to accommodate the two types of models.

Author response:
**Q8.** Generators in GANs do not require their latent space to be of the same dimension as their output. Yet, it seems to be the case for OT map models, given that they learn a transport map between two distributions living in the same space. I would suggest the authors to explicitly explain how this affects their experiments and their results. Are the provided comparisons fair between the two types of models, in terms of dimensionality and neural network architectures? Both operation modes seem hard to articulate with each other, as Algorithm 1 features the sampling of both a random variable from the same space as the data and another latent variable to accommodate the two types of models.

**A8.** We sincerely appreciate the reviewer for the thoughtful comment regarding the missing clarifications and details in our manuscript. We thoroughly revised the main part and appendix of our manuscript to incorporate these previously missing descriptions.

- **Description of auxiliary variable $z\sim \mathcal{N}(0,I)$:**
This random variable $z$ in Algorithm 1 does not represent the input Gaussian Noise in GAN models. Instead, this auxiliary noise is provided to the generator $T_{\theta}$ in the OT map models, such as UOTM [2], to enhance generative performance from the practical perspective. We included a description of this auxiliary variable $z$ in Sec 3.1 as follows: 

> Also, the Gaussian noise $z$ represents the auxiliary variable and is different from the input prior noise $x \sim \mu$. This auxiliary variable $z$ is introduced to represent the stochastic transport map $T_{\theta}$ in the OT map models, such as UOTM.

- **Description of latent space $\mathcal{X}$:**
As the reviewer commented, the original formulation of OT map models (Eq 5 and 8) requires their latent space $\mathcal{X}$ to be of the same dimension as their output $\mathcal{Y}$. However, **these OT map models can also accommodate a smaller dimension for $\mathcal{X}$ by employing a simple practical scheme suggested in OTM [3].** We adopted this scheme for the DCGAN backbone for CIFAR-10 in Table 2, following the implementation of OTM. Therefore, **we believe that there are no constraints on the latent dimension and, consequently, no restrictions on the neural network architecture.** We clarified these implementation details in Appendix B and added descriptions on Page 7 as follows:

> [Page 7] Note that the DCGAN backbone employs the deterministic upsampling strategy from OTM [3] to accommodate an input latent space with a smaller dimension than the data space. (See Appendix B for details).

> [Appendix B] In the DCGAN backbone, we adopt a simple practical scheme suggested in OTM [3] for accommodating a smaller dimension for the input latent space $X$. This practical scheme involves introducing a deterministic bicubic upsampling $Q$ from $\mathcal{X}$ to $\mathcal{Y}$. Then, we consider the OT map between $Q_{\sharp}\mu$ and $\nu$. In practice, we sample $x$ in Algorithm 1 from a 192-dimensional standard Gaussian distribution. Then, $x$ is directly used as an input for the DCGAN generator $T_\theta$. The random variable $z$ is not employed in the DCGAN implementation. Meanwhile, $Q(x)$ is obtained by reshaping $x$ into a $3\times 8\times 8$ dimensional tensor, and then bicubically upsampling it to match the shape of the image. The generator loss is defined as $c\left(Q(x), T_\theta (x) \right) - v_\phi \left( T_\theta (x) \right)$.

> [Appendix B] Specifically, we set $\mathcal{X}=\mathcal{Y}$ and use $c(x,y) = \tau \lVert x-y \rVert^2$ without introducing upsampling $Q$.  Here, the auxiliary variable $z$ is employed. We sample $z$ from a 256-dimensional Gaussian distribution and put it as an additional stochastic input to the generator. The input prior sample $x$ is fed into the NCSN++ network like UNet input. The auxiliary $z$ passes through embedding layers and is incorporated into the intermediate feature maps of the NCSN++ through an attention module.

Reviewer comment (Theory):
The authors should further comment why the Lipschitzness result of Theorem 3.1 is an advantage over GANs, as WGAN(-GP) also requires Lipschitz solutions, and the Lipschitzness constraint is even applied to other GAN models nowadays.

Author response:
**Q9** Moreover, the authors should further the comment why the Lipschitzness result of Theorem 3.1 is an advantage over GANs. WGAN(-GP) also requires Lipschitz solutions, and the Lipschitzness constraint is even applied to other GAN models nowadays.

**A9.** 
**We would like to distinguish between constrained optimization (WGAN(-GP)) and unconstrained optimization (Theorem 3.1).** For example, consider two optimization problems: for some $u \in \mathbb{R}^{d}$,
$$
    \text{maximize}\_{v \in \mathbb{R}^{d}, \\| v \\|\_2 = 1}   \langle u, v \rangle  \qquad
    \text{minimize}\_{v \in \mathbb{R}^{d}} \\| u - v \\|\_{2}^{2}.
$$
The constrained optimization problem on the left is meaningful only when we impose the constraint $\\| v \\|\_2 = 1$. Under the constraint, the maximum value is $\\| u \\|\_2$. Without the constraint, $\langle u, v \rangle$ simply diverges to $\infty$ as we increase $\\| v \\|\_2$. 
On the contrary, the optimization problem on the right does not impose any constraint on $v$. Nevertheless, minimizing the objective function $\\| u-v \\|\_{2}^{2}$ naturally provides some regularity on $v$: if $\\| u-v\_{i} \\|\_{2}^{2} < r^{2}$ for some iterate $v\_{i}$, $v\_{i}$ situated within a ball near $u$. A similar phenomenon occurs between WGAN(-GP) and UOTM.

**The learning objective of WGAN(-GP) conducts optimization over the 1-Lipschitz potential.** The Gradient Penalty regularizer is introduced to constrain the potential network to satisfy the 1-Lipschitz constraint. As discussed in Appendix C, several studies showed that such Lipschitz constraints may lead to mode collapse/mixture problems [4, 5].

On the contrary, **Theorem 3.1 states that, under unconstrained optimization, the potential networks $v_{\phi}$ with only minor conditions satisfy the equi-Lipschitzness**. Therefore, Theorem 3.1 suggests that the potential networks $v_{\phi}$ would consistently represent properly regular behavior throughout training, avoiding issues such as exploding gradients. This regularity of potential network in the UOTM model is verified through Fig 6. Following the reviewer's advice, we revised our manuscript on Page 7 as follows:


> Note that Theorem 3.1 is fundamentally different from the 1-Lipschitz constraint of WGAN (Eq. 4). WGAN involves constrained optimization over a 1-Lipschitz potential. In contrast, Theorem 3.1 states that, under unconstrained optimization, the potential networks $v_{\phi}$ with only minor conditions satisfy equi-Lipschitzness.

Reviewer comment (Writing):
It is not easy to understand without expertise in OT. It would be much easier to read if one or two lines of descriptions comparing each term and notation with those of the regular GAN setup were provided.

Author response:
We appreciate the reviewer for the comment. We included additional explanations comparing the regular GAN and OT map models below Eq. 5 as follows:


> Intuitively, $T_\theta$ and $v_\phi$ serve as the generator and the discriminator of a GAN. For convenience, we denote the optimization problem of Eq. 5 as an OT-based generative model (OTM). *Note that, if we set $c=0$ and introduce a 1-Lipschitz constraint on $v_{\phi}$, this objective has the same form as WGAN (Eq. 4).* In OT map models, the quadratic cost is usually employed, i.e., $c(x,y) = \tau \\|x-y\\|_{2}^{2}$.

Reviewer comment (Evaluation):
The analysis is good, but the benefit of the proposed method (UOTM-SD) is not clearly shown.

Author response:
We appreciate the reviewer for acknowledging the novelty of our analysis. **We would like to emphasize that our UOTM-SD provides additional benefits in terms of generative performance (FID score in Table 3) and robustness to hyperparameter (Fig 7 and Table 5) over UOTM.**

Our analysis demonstrated UOTM outperforms other OT-based adversarial networks by exploiting two key factors: (1) Stable training from the strictly convex $g_1, g_2$ and (2) Alleviation of mode collapse from the cost function $c(x, y)=\tau \\|x-y\\|_{2}^{2}$. **However, UOTM has some limitations.** The UOT problem incurs inherent distribution errors, which limit the generative performance of UOTM. Also, UOTM is sensitive to the cost intensity hyperparameter $\tau$.


**In this respect, UOTM-SD addresses this inherent distribution error of UOTM stemming from the UOT problem (Theorem 4.1). Moreover, UOTM-SD is significantly more robust to $\tau$ than UOTM (Fig 7 and Table 5).** For example, when $\tau=2e-4$, UOTM yields an FID score of 15.19, whereas UOTM-SD achieves a substantially lower FID score of 3.60. Similarly, when $\tau=5e-3$, UOTM yields an FID score of 218.02, while UOTM-SD achieves a markedly improved FID score of 5.42.

Reviewer comment (Experiments):
Experiments are limited to low-dimensional datasets which is not practical enough.

Author response:
We thank the reviewer for the thoughtful advice. We evaluated our UOTM-SD model on the high-resolution dataset, **CelebA-HQ ($256 \times 256$)**. Due to limitations in time and resources, we conducted one experiment for each model with NCSN++ backbone: UOTM-SD with Linear scheduling ($g_1=g_2=SP$), UOTM ($g_1=g_2=SP$), and OTM. We selected these three models as they represent competitive options among OT-based adversarial networks on CIFAR-10 (Table 3). The FID scores below demonstrate our UOTM-SD model outperforms the other two OT map models.


|Model|FID ($\downarrow$)|
|:---|:---|
|OTM|13.56|
|UOTM (SP)|9.78|
|UOTM-SD (Linear)|**8.19**|

The training hyperparameters are as follows: the cost intensity $\tau=0.00001$, R1 regularization intensity $\lambda=5$, Scheduling intensity $(\alpha_{min}, \alpha_{max})=(1/5, 5)$.

Reviewer comment (Experiments):
Is the analysis valid in relatively higher-dimensional data (e.g., 128^2 or 256^2) such as CelebA or FFHQ, CUB, or ImageNet?

Author response:
We thank the reviewer for the thoughtful advice. We evaluated our UOTM-SD model on the high-resolution dataset, **CelebA-HQ ($256 \times 256$)**. Due to limitations in time and resources, we conducted one experiment for each model with NCSN++ backbone: UOTM-SD with Linear scheduling ($g_1=g_2=SP$), UOTM ($g_1=g_2=SP$), and OTM. We selected these three models as they represent competitive options among OT-based adversarial networks on CIFAR-10 (Table 3). The FID scores below demonstrate our UOTM-SD model outperforms the other two OT map models.


|Model|FID ($\downarrow$)|
|:---|:---|
|OTM|13.56|
|UOTM (SP)|9.78|
|UOTM-SD (Linear)|**8.19**|

The training hyperparameters are as follows: the cost intensity $\tau=0.00001$, R1 regularization intensity $\lambda=5$, Scheduling intensity $(\alpha_{min}, \alpha_{max})=(1/5, 5)$.

Reviewer comment (Writing):
(Alg. 1) What is $X$ in line 3? I would assume $Y$ is from real data and $z$ is from the prior known distribution, e.g., Gaussian. Similarly, $T_\theta$ is consistently taking $x$ as input throughout Equations in the paper while $x$ and $z$ are taken as input in the algorithm (line 4). For example, in Eq. 5, I can see that $x$ is a prior distribution, $y$ is a real distribution, and the OT term $c$ is applied in between the Gaussian and the generated samples, which I believe is different from the Algorithm.

Author response:
We sincerely appreciate the reviewer for the thoughtful comment regarding the missing clarifications and details in our manuscript. We thoroughly revised the main part and appendix of our manuscript to incorporate these previously missing descriptions.

In our notation in Alg. 1, **$X$ represents the input latent noise, sampled from the source (prior) distribution $\mu$. $Y$ denotes the real data, sampled from the target (data) distribution $\nu$.** The additional Gaussian noise $z\sim \mathcal{N}(0,I)$ in Alg. 1 is the auxiliary variable. We introduced this auxiliary variable $z$ to represent the stochastic transport map $T_{\theta}$ in the OT map models, such as UOTM. This auxiliary noise is provided to the generator $T_{\theta}$ to enhance generative performance from the practical perspective. We included a description of this auxiliary variable $z$ in Sec 3.1 as follows: 

> Also, the Gaussian noise $z$ represents the auxiliary variable and is different from the input prior noise $x \sim \mu$. This auxiliary variable $z$ is introduced to represent the stochastic transport map $T_{\theta}$ in the OT map models, such as UOTM.

Reviewer comment (Theory):
(page 6, “Effect of Cost in Mode Collapse”) It is not straightforward how the cost function actually helps the mode coverage.

Author response:
In Sec 3, we observed that the cost function helps the mode coverage of OT-based adversarial networks in both the Toy dataset (Fig 1) and CIFAR-10 (Fig 4). **We hypothesize that this observation stems from the indirect supervision provided by the cost minimization in the OT problem.** More precisely, this cost minimization induces the generator $T$ to transport the generated image $T(x)$ close to the input noise $x$. **Therefore, when dealing with a multimodel data distribution $\nu$, this cost minimization property specifies which part of the source distribution $\mu$ should correspond to each mode of $\nu$, as visualized in Fig 5.** We believe this specification is helpful in addressing mode coverage. On the contrary, when there is no cost function (WGAN and UOTM w/o cost in Fig 5), each input noise is randomly transported to the support of data distribution.

Reviewer comment (Experiments):
Performance of UOTM-SD in the toy dataset? and Qualitative comparison results in Cifar10 (UOTM-SD v.s. UOTM)?

Author response:
In the initial manuscript, we provided the generated samples of UOTM-SD and UOTM on CIFAR-10 in Appendix D.5. We added additional clarifications on Page 9, indicating these qualitative samples as follows: 

> See Appendix D.5 for the qualitative comparison of generated samples.

Furthermore, we included UOTM-SD results on the toy dataset in Appendix D.1. In the Toy dataset, UOTM and UOTM-SD demonstrate almost similar results.