An Optimal Transport Perspective on Unpaired Image Super-Resolution
Authors: Milena Gazdieva
Litu Rout
Alexander Korotin
Andrey Kravchenko
Alexander Filippov
Evgeny Burnaev
Venue: Submitted to ICLR 2023 (2023)
Keywords: optimal transport
unpaired image super-resolution

Abstract: Real-world image super-resolution (SR) tasks often do not have paired
datasets, which limits the application of supervised techniques. As a result, the tasks are usually approached by unpaired techniques based on Generative Adversarial Networks (GANs), which yield complex training losses with several regularization terms, e.g., content or identity losses. We theoretically investigate optimization problems which arise in such models and find two surprizing observations. First, the learned SR map is always an optimal transport (OT) map. Second, we theoretically prove and empirically show that the learned map is biased, i.e., it does not actually transform the distribution of low-resolution images to high-resolution ones. Inspired by these findings, we propose an algorithm for unpaired SR which learns an unbiased OT map for the perceptual transport cost. Unlike the existing GAN-based alternatives, our algorithm has a simple optimization objective reducing the need for complex  hyperparameter selection and an application of additional regularizations. At the same time, it provides a nearly state-of-the-art performance on the large-scale unpaired AIM19 dataset.

Under review as a conference paper at ICLR 2023
AN OPTIMAL TRANSPORT PERSPECTIVE ON UNPAIRED IMAGE SUPER-RESOLUTION
Anonymous authors Paper under double-blind review
ABSTRACT
Real-world image super-resolution (SR) tasks often do not have paired datasets, which limits the application of supervised techniques. As result, the tasks are usually approached by umpaired techniques based on Generative Adversarial Networks (GANs), which yield complex training losses with several regularization terms, e.g., content or identity losses. We theoretically investigate optimization problems which arise in such models and find two surprizing observations. First, the learned SR map is always an optimal transport (OT) map. Second, we theoretically prove and empirically show that the learned map biased, i.e., does not actually transform the distribution of low-resolution images to righ-tesotution ones. Inspired by these findings, we propose an algorithm for unpaired SR which learns an unbiased OT map for the perceptual transport cost. Unlike the existing GAN-based alternatives, our algorithm has a simple optimization objective reducing the need for complex hyperparameter selection and an application of additional regularizations. At the same time, it provides a nearly statc-of-the-art performance on the large-scale unpaired AIM19 dataset.
INTRODUCTION
The problem of image super-resolution (SR) is to reconstruct a high-resolution (HR) image from its low-resolution (LR) counterpart. In many modern deep learning approaches, SR networks are trained in a supervised manner by using synthetic datasets con- taining LR- HR pairs (Lim et al. 2017 $4.1); (Zhang et al. 2018b 14.1). For example, is common to create LR images from HR with simple downscal- ing, e.g.. bicubic (Ledig et al. [017] 33.2). However, such artificial setup barely represents the practical setting, in which the degradation is more sophisti- cated and unknown (Maeda 2020). This obstacle suggests the necessity of developing methods capa- ble of learning SR maps from unpaired data without considering prescribed degradations.
Bicubic
OTS (ours)
DASR
Figure 1: Super-resolution of squirrel using Bicubic upsample, OTS (ours) and DASR (Wei et al. (02T]) methods (4x4 4 upsample, 370x 800 crops).
Contributions. We study the unpaired image SR task and its solutions based on Generative Adver- sarial Networks (Goodfellow al. 2014 GANs) and analyse them from the Optimal Transport (OT, see (Villani 2008)) perspective.
1. We investigate the GAN optimization objectives regularized with content losses, which are common in unpaired image SR methods ($5) 34.. We prove that the solution to such objectives is always an optimal transport map. We theoretically and empirically show that such maps are biased ($7.1) i.c., they do not transform the LR image distribution to the true HR image distribution.
2. We provide algorithm to fit an unbiased OT map for perceptual transport cost ($6.1) and apply it to the unpaired image SR problem 7.22. We establish connections between our algorithm and regularized GANs using integral probability metries (IPMs) as loss (56.2).
Our algorithm solves minimax optimization objective and does not require extensive hyperparameter search, which makes different from the existing methods for unpaired image SR. At the same time, the algorithm provides nearly state-of-art performance in the unpaired image SR problem ( (77.)).

---

Under review as a conference paper at ICLR 2023
Notation. We use X, to denote Polish spaces and (()), P()") to denote the respective sets of probability distributions on them. We denote by [I, Q) the set of probability distributions on X' x y with marginals P and Q. For measurable map T X' + V, denote the associated push-forward operator by T- The expression || denotes the usual Euclidean norm not stated otherwise. We denote the space of Q-integrable functions on y by L L'(@).
2 UNPAIRED IMAGE SUPER- RESOUUTION TASK
In this section, we formalize the unpaired image super-resolution task that we consider (Figure
Let and Q two distributions of LR and HR images, respectively, on spaces X' and V, respec- tively. We assume that obtained from Q via some unknown degradation. The learner has ac- cess to unpaired random samples from IP and Q The task is to fit map T X + y satisfying TæIP- = Q which inverts the degradation.
map
P
Highwres images
Degradation (unknown)
Figure 2: The task super-resolution we consider.
We highlight that the image SR task is theoretically ill-posed for two reasons.
1. Non-existence. The degradation filter may be non-injective and, consequently, non-invertible. This is theoretical obstacle to learn one-to-one SR maps T'
2. Ambiguity. There might exist multiple maps satisfying T.P Q but only one inverting the degradation. With no prior knowledge about the correspondence between IP and Q, it is unclear how to pick this particular map.
The first issue is usually not taken into account in practice. Most existing paired and unpaired SR methods learn one-to-one SR maps T, (Ledig et al. 2017 Lai et al., 2017 Wei et al. 2021).
The second issue typically softened by regularizing the model with the content loss. In the real-world, it is reasonable assume that HR and the corresponding LR images close. Thus, the fitted SR map 7' is expected to only slightly change the input image. Formally, one may require the learned map T have the small value of
RT(T) def J (())))
(I)
where X' x y R. is a function estimating how different the inputs are. The most popular example is the Ed identity loss, ie, formulation for X' = y = R and c(z, y) = || yl11.
More broadly, losses (T') typically called content losses and incorporated into training objec-
tives of methods for SR (Lugmayr et al. 2019a $3.4), (Kim et al. 2020 53) and other unpaired tasks beside SR (Taigman et al.) 2016 $4), (Zhu al. [2017] $5.2) regularizers. They stimulate
the learned map T to minimally change the image content.
3 BACKGROUND ON OPTIMAL TRANSPORT
In this section, we give the key concepts of the OT theory (Villani 2008) that we use in our paper.
Primal form. For two distributions P E P(X) and Q P())) and transport cost c X' y + R. Monge's primal formulation of the optimal transport cost is as follows:
/ c( ((s,)(())dP(L)) T+r=
Cos
inf
(2)
where the minimum is taken the measurable functions (transport maps) T X' - that map P to Q see Figure 3al The optimal T is called the optimal transport map.
Note that is not symmetric, and this formulation does not allow mass splitting, i.c., for some P. Q there may be no map T that satisfies Ty P = Q. Thus, (Kntorovitch 1958) proposed the relaxation:
Cost(IP
def inf =EII(P.Q) J c(x,
where the minimum taken over the transport plans π,, i.c., the measures on X' x y whose marginals P and @ (Figure 3b) The optimal x= E [TP,,, Q) is called the optimal transport plan.
2

---

Under review as a conference paper at ICLR 2023
c(x).((x:)
c(Xi.y:)
X1
T(Xi)
X1
y1
PP X: c(x:TT(x))
T(x.) Q
P
Y2 Q
T(x.)
Yn
Xn
c(x..T(x.)
Xn c(X-ym)
(a) Monge's formulation of OT.
(b) Kantorovich's formulation of OT.
Figure Monge's and Kantorovich's formulations of Optimal Transport.
With mild assumptions on the transport cost c(x, y) and distributions P. Q. the minimizer '² of 3 always exists (Villani 2008 Theorem 4.1) but might not be unique. If x² is of the form id.
T E (I,,, ( for some 7", then T is an optimal transport map that minimizes (
Dual form. The dual form Viillani 2003 of OT cost 33 is follows:
(x)))P(a)) + / + f()) yEy
Cost(P, Q = sup
(4)
here sup is taken over all (EC((Q)) and (+)) - inf [c(x, v) [vv] is the c-transform of
RELATED WORK
Existing approaches to unpaired image SR mainly solve the problem in two steps. One group of approaches learn the degradation operation the first step and then train a super-resolution model in a supervised manner using generated pseudo-pairs, (Bulat et al., 20118 Fritsche et al. 2019). Another group of approaches (Yuan et al. 2018 Maeda 2020) firstly learn mapping from weal-world LR images "clean" LR images, L.e., HR images, downscaled using predetermined (e.g., bicubic) operation, and then a mapping from "clean" LR to HR images. Most methods based on CycleGAN (Zhu et al. (017). initially designed for the domain transfer task, and utilize cycle-consistency loss. Methods are also usually endowed with several other losses, e.g. content (Kim al. 2020 53), identity (Wang al. 2021 53.2) or perceptual (Lugmayr et al. 2019a $3.4).
UNPAIRED IMAGE
OPTIMAL TRANSPORT GENERATIVE MODELS. The majority of existing OT-based generative models employ OT cost as the loss function to update the generative network, e.g., see (Arjovsky
al. 2017). These methods are out of scope the present paper, since they do not compute OT maps.
Existing methods to compute the OT map approach the primal (2) (3)
(3) or dual form (4). Primal-form
methods (Lu et al. 2020 Xie et al. 2019 [Bouqquet et al. 2017 Balaji et al.) 2020) optimize complex
GAN objectives such as 55) and provide biased solutions 5 7.1 For a comprehensive overview of dual-form methods, we refer to (Korotin et al. 2021). The authors conduct an evaluation of OT methods for the quadratic cost c(2,)) y) = 2 yll According to them, the best performing method is [.M:R]. It is based on the variational reformulation of (4), which is a particular case of our
formulation 12 Extensions of [MM:R] appear in (Rout al. 2022 Fan et al. 2021
5 BIASED OPTIMAL TRANSPORT IN GANS
In this section, we establish connections between GAN methods regularized by content losses [ and OT. Such GANs are popular in variety of tasks beside SR, e.g., style transfer (Huang et . 2018) 8). The the- oretical analysis in this section holds for these tasks as well. However, since we empirically demonstrate the findings we keep the corresponding notation throughout $5 A common approach to solve the unpaired SR via GANS to define a loss function D P())) x P(Y) + R, minimizing
X1
T'(x)) T(x))
P X2
X
T'(x))
Figure 4: Illustration of Lemmal | The
practically important SR problem,
solution T (5) is an OT map from P to TAP.. In general, TAP P * Q (Thm.
and train a generative neural network T via
inf T-X+y
P. + (T))
The term D P. Q) ensures that the generated distribution Tu of SR images is close to the true HR distribution Q: the second term R. (T') the content loss (.). For convenience, we assume that D(Q, Q) = 0 for all Q E P(J)). Two most popular examples of D are the Jensen-Shannon divergence (Goodfellow et al. 2014), i.e., the vanilla GAN loss, and the Wasserstein-1 loss (Arjovsky & Bottou
2017). In unpaired SR methods, the optimization objectives are typically more complex than (5). In
addition to the content or identity loss ( several other regularizations are usually introduced, see
In Appendix F we show that the learning objectives of popular SR methods can be represented 5)
3

---

Under review as a conference paper at ICLR 2023
For theoretical analysis, stick the basic formulation regularized with generic content loss (5). It represents the simplest and straightforward SR setup. We prove the following lemma, which connects the solution T of ( and optimal maps for transport cost c(x, y).
Lemma (The solution of the regularized GAN is an OT map). Assume that À > 0 and the minimizer
T of exists. Then T is an OT map between and C def T for cost c(x, y), ie., minimizes inf / P= xx
inf
(T)
(cx,T()))dl)
Tw P
Proof. Assume that T is not an optimal map between TP and T.P. Then there exists a more optimal Tt satisfying T T and (T*) < (Txx). We substitute this Tt to (5) and derive
D(T # P. Q) + XR = DI Q) + (T)) < D(T Q) XRe(T*).
which is contradiction, since T is minimizer of 5 but T provides the smaller value.
Our Lemma I states that the minimizer T of a regularized GAN problem is always an OT map between IP and the distribution QA generated by the same T from IP. However, below we prove that @A + QQ i.e., T does not actually produce the distribution of HR images (F..ure[4). To begin with, we state and prove the following auxiliary result.
Lemma 2 (Reformulation of the regularized GAN via distributions). Under the assumptions of Lemma 7 let X' = y be a compact subset of R with negligible boundary. Let P E P(X) be absolutely continuous, Q QEP()) and c(x, y) 2 y|l with p 1. Then (5) is equivalent
inf
F(O) def inf
[D(Q' Q) + Cost(P, Q')],
(6)
VEP())
('E
and the solutions of and are related as Q = T where Q is the minimizer of
6
Proof. We derive
[DTT Q +1 / c(x, (<))(()) TW--y inf [D(T Q (T)) + A Cost(PP,
inf
(7)
TW-Y
inf [D(T
inf (EP(())
[D(Q'
Cost(P, Q').
(8)
T--+Y
In transition from 7 to 8 we use the definition of OT cost 2 and our Lemma I which states that
the minimizer T (5) is an OT map, i.c., Sxx c(x (x,,T''(aa (x)) dI (x) Cost(IP, T P). The equality in
(8) follows from the fact that P is abs. cont. and c(x. y) | P: for all Q' E P()) there exists
(unique) solution 7 to the Monge OT problem for P. Q' (Santambrogio
2015 Thm. 1.17). O
In the following Theorem, we prove that, in general, Q^ + Q for the minimizer Q of (6)
Theorem 1 (The distribution solving the regularized GAN problem is always biased). Under the
assumptions of Lemma 2 assume that the first variation (Santambrogio 2015 Definition 7.12) of
the functional Q' + D(O Q at the point Q' = Q exists and is equal to zero. This means that D(Q + eAQ) = D(Q, Q) + o(c) for every signed measure AQ of zero total mass and e 2 O such
that + cQQ E P()). Then, If + Q then Q' Q does not deliver the minimum to
Before proving Theorem I we highlight that the assumption about the vanishing first variation of Q' D(Q' Q) Q
reasonable. In Appendix A we prove that this assumption holds for the popular GAN discrepancies D(Q', Q). e.g., f-divergences (Nowozin et al. 2016)), Wasserstein
distances (Arjovsky et al. 2017). and Maximum Mean Discrepancies (Li et al. 2017).
Proof. Let AQ = P Q denote the difference measure of IP and Q. It has zero total mass and eP+ (1- e) is a mixture distribution of probability distributions
Ve [0. 1] it holds that Q +Q+cA0
P and Q. As a result, for all E [0, 1J. we have
F(C + (AQ) = D(Q + <QQ. Q) + A Cost [P Q + <4Q) =
D(Q. Q) + o(e) + A Cost(PP cP + (1 -1 e) <
(9)
(c)) + + Cls.c.Cos(PPPPp - A- - c) Cost(PP Q) = o(c) + \ c) Cost( Q) =
(10)
X- Cos ! - X Cost(P Q +o(c),
=F(() where in transition from 99 to 100, we use D(Q. ? 0 and exploit the convexity of the OT cost
(Villani 2003, Theorem 4.8). In (10). we use Cos P) = O. We see that F(Q + F(Q+ +eQ is smaller
then F for sufficiently small i.e., Q' Q does not minimize F.
4

---

Under review as a conference paper at ICLR 2023
Corollary 1. ie., it does not satisfy T Q and does not transform LR images to true HR ones.
Under the assumptions of Theorem L
the solution T of regularized GAN 5 is biased.
Additionally, we provide a toy example that further illustrates the issue with the bias.
Example 1. Consider X' = y R! Let P 6. + 1 62, Q 1 2,03 3 be distributions concentrated at (0, and {1, respectively. Put c(x, v) x yl to be the content loss. Also, let
2
DD to be the OT cost for [x - y Then for X = 0 there exist two maps between IP and Q that deliver
the same minimal value for ) namely T(0) = 1, (2) 3 and T(0) 3, T() = 1. For À 0,
the optimal solution of the problem (5) is unique, biased and given by T(0) = 2: T(2) 3-
Proof. Let T(0) = to and T(2) 12. Then T# 2000 202. and now becomes
1
min min ( (4o - 1)2 + 2 (/2 - -3) (40 - 3) + 2 (12 1)²} M 00 tol + 2 12 t0.32
where the second term is (T) and the first term is the OT cost D(T Q)) expressed as the
minimum over the transport costs of two possible transport maps to 1; + 3 and + 3; 22 The minimizer can be derived analytically and equals to = 2 12 3
In Example T @^ never matches Q exactly for > In
3 7711 conduct evaluation of
maps obtained via minimizing objective (5) on the synthetic benchmark by (Korotin et al. 202T). We
empirically demonstrate that the bias exists and it is indeed a notable practical issue.
Remarks. Throughout this section, we enforce additional assumptions on (5). c.g., we restrict our analysis content losses c(.,)) -), which are powers of Euclidean norms || _| This needed make
the derivations concise and to be able to exploit the available results in OT. We think that the provided
results hold under more general assumptions and leave this question open for future studies.
6 UNBIASED OPTIMAL TRANSPORT SOLVER
In I we derive algorithm to compute OT maps. Importantly, in 66.2 we detail its differences and similarities with regularized GANs, which we discussed in $ Our algorithm is suitable for general costs and generalizes the OT algorithm for the quadratic cost by (Rout et al. 2022).
6.1 MINIMAX OPTIMIZATION ALGORITHM
We derive a minimax optimization problem to recover the optimal transport map from P to Q. We expand the dual form (4) To do this, first note that
xx / inf {c(x, v) (w) d(P(x) yEy
/ (((()) dP(() (11)
(')))P(x)) x
inf T:X T-X-y
Here we replace the optimization over points E y with an equivalent optimization over the functions T X +>>) y. This is possible due to the Rockafellar interchange theorem (Rockafellar 1976,
Theorem 3A). Substituting (11 to (4), we have
Cost(P, Q = sup inf TXX+y
I, f(v)d(())
S
dP(w))
(12)
We denote the expression under the sup inf by L(J,T). Now we show that by solving the saddle point problem [12 one can obtain the OT map T*
Lemma 3 (OT maps solve the saddle point problem). Assume that the OT map T between P. Q for cost c(2, y) exists. Then, for every optimal potential fo E sup, [infr:x L(f, (ff,TT T)] of (2))
T E Era inf / T
- (T( - d(((x).
(13)
Proof. Since fr is optimal, we have inf (( Cost(I, Q) We use TzP Q and the = J (y))dQ Substituting this equality into 12 we obtain L(f",T") fx c(.,TT dx(x)) = Cost((P @), i.e., L3 holds.
change of variables = (x) to derive fx (T
Our Lemma 3 that one solve saddle point problem (12) and extract OT map 7 between P, Q from the optimal pair (*,T"). Analogous result but only for the (Q-embedded) quadratic cost
can be found in (Rout al. 2022 For general IP, Q. the arg inf, set for optimal f might contain
5

---

Under review as a conference paper at ICLR 2023
Algorithm 1: OT solver to compute the OT map between IP and Q for transport cost c(t, v).
Input :distributions P. Q accessible by samples; mapping network To X' - )
potential fw X' + IR: transport cost e: X IR; number Kr of inner iters; Output: sntput:approximatt OT map (Ta) P & Q: repeat
Sample batches X ~ P. Y
Σ NEY
(v)
E (_(T(())); ø
X
Update by using OC; for ky 1,2,.... ..... K do
maximize Ly;
Sample batch ~~.,, P;
CT + [ Σ [c(,,Te(x)) - Iw (Te( #EX
Update 0 by using de
DCr to minimize Cr;
until not converged;
not only OT map T but other functions as well. However, our experiments (77 show that this
not serious issue in practice. To solve the optimization problem (I2), we approximate the potential
and map with neural networks fo and Tg. respectively. We train the networks with stochastic gradient ascent-descent by using random batches from P, Q
The practical optimization procedure is detailed in Algorithm 1 Transport Solver (OTS).
We call this procedure an Optimal
6.2 REGULARIZED GANS VS. OPTIMAL TRANSPORT SOLVER
In this subsection, we discuss similarities and differences between our optimization objective (2)
and the objective of regularized GANs (5) We establish intriguing connection between GANs that use integral probability metrics (IPMs) as D. discrepancy D: (PV) R + is an IPM if
D(Qi, (2) = sup / (v) SEF
(v)dQ1
(14)
where the maximization is performed over certain class F of functions (discriminators) f -> RR. The most popular example of 'D the Wasserstein-1 loss (Arjovsky & Bottou 2017 ()))).
where F is class of 1-Lipschitz functions. For other IPMs, see (Mroueh et al. [017]] Table
1).
Substituting [4p to yields the saddle-point optimization problem for the regularized IPM GAN:
(W)) ((T(2)) + S clx, ((,,T()))dP()))
inf T: y
sup JEF
J (W)) + / (A. (A. - ((((m)) aP(2)) TX++y fer
inf
sup
(15)
We emphasize that the expression inside (15) for x is similar to the expression in OTS optimiza- tion (12. Below we highlight the key differences between 122 and 15
First, in OTS the map T a solution to the inner optimization problem, while in IPM GAN the generator T is solution to the outer problem. Swapping inf7 and sup, is prohibited and, in general, yields different problem, e.g., 1= inf, sup, cos(x+y) * sup, inf, cos(x+ty)=-1.
Second, in OTS the optimization to F, some certain restricted class of functions. For example, when D the Wasserstein- 1 (W1) additional penalization, e.g., the gradient penalty (Gulrajani et al., (017). This
potential f is unconstrained, while in IPM GAN it must belong
IPM, one has to further complicates the optimization and adds hyperparameters which have to be carefully selected.
Third, the optimization of IPM GAN requires selecting parameter À that balances the content loss R. and the discrepancy D. In OTS for all costs À o(x,y) y) with > O, the OT map T the same.
To conclude, even for = 1, the IPM GAN problem does not match that of OTS. Table summarizes the differences and the similarities between OTS and regularized IPM GANs.

---

Under review as a conference paper at ICLR 2023
Optimal Transport Solver (Ours)
Regularized IPM GAN
Minimax optimization objective
sup 1 T-N-Y
[ f(y)
inf sup T-X fet f())d((()))
Sx (()) - s(T(x))) dP(x)
Sxe (A
- (()) dF(x)
Transport map (generator)
T solves the inner problem (for optimal f");
T solves the outer problem:
it is an OT map from P to Q (Lemma
it is a biased OT map $$ 7.1
Potential f (discriminator)
Constrained f E J C (Q)
Unconstrained f (Q)
A method to impose the constraint is needed.
Regularization weight A
N/A
Hyperparameter choice required
Table I: Comparison of the optimization objectives of OTS (ours) and regularized IPM GAN.
7 EVALUATION
In 37.T] we assess the bias of regularized IPM GANs by using the Wasserstein-2 2 benchmark (Korotin cet 2001). J In 7.2 we evaluate our method on the large-scale unpaired AIM-19 19 dataset from (Lugmayr et al. 2019b). Appendix [ we test it on the CelebA dataset (Liu et al. 2013). The code
written in PyTorch. We list the hyperparameters for Algorithm I in Table 4 of Appendix C
Neural network architectures. We use WGAN-QC's (Liu et al. 2019) ResNet (He et al. 2016)
architecture for the potential fu. In $ where input and output images have the same size, we use UNe (Ronneberger et al., 2015) as transport map Tg. In $77.2 the LR input images are 4 times smaller than HR, so we use EDSR network (Lim et al., 2017).
Transport costs. In 77.1 we use the mean squared error (MSE), i.e., c(x, v) ---v|l It is d
equivalent the quadratic cost but more convenient due to the normalization. In 77.2 we consider c(z; y) b(Up(ar), v). where b is a cost between the bicubically upsampled LR image rP' Up(#) and HR image y. We test b defined as MSE and the perceptual cost using features of pre-trained VGG-16 network (Simonyan & Zisserman 2014), see Appendix C for details.
7.1 ASSESSING THE BIAS IN REGULARIZED GANS
In this section, we empirically confirm the insight of $ that the solution T of (5) may not satisfy TAP Q. Note if T map from IP to Q for c(z, y). Thus, to access the bias, it is reasonable to compare the learned map T with the ground truth OT map 7" for IP, Q
Q. then by our Lemma] 1 we conclude that T = 7", where 7 is an OT
For evaluation, we the Wasserstein-2 benchmark (Korotin al., 1021). It provides high- dimensional continuous pairs P, Q with an analytically known OT map T for the quadratic cost c(z, y) ||æ - 3//². We use their "Early" images benchmark pair. It simulates the image deblurring setup, i.e., X y' the space of 64 x 64 RGB images, blurry faces, Q is clean faces satisfying Q = T. where T is an analytically known OT map, see the 1st and 2nd lines in Figure 5
To quantify the learned maps from TP to Q we use PSNR, SSIM, LPIPS (Zhang et al. 2018a), FID (Heusel al. 2017) metrics. Similar to (Wei et al. (021). we use the AlexNet-based (Krizhevsky ct al.] 2012) LPIPS. FID and LPIPS are practically the most important since they better correlate with the human perception of the image quality. We include PSNR, SSIM as popular evaluation metries, but they are known to badly measure perceptual quality (Zhang et al., /20 8a Nilsson & Akenine Möller 2020). Due to this, higher PSNR, SSIM values do not necessarily mean better performance. We calculate metrics using scikit -itiiiimmigeeee for SSIM and open source implementations for PSNR LPIPS and FIL In this section, we additionally use the - (Korotin et al. 2021 84.2) metric.
On the benchmark, we compare OTS 12 and IPM GAN ( We use MSE as the content loss c(x, y). In IPM GAN, we use the Wasserstein- (W1) loss with the gradient penalty AGP = 10 (Gulrajani ct al. 2017) D. We do 10 discriminator updates per 1 generator update and train the model for 15K generator updates. For fair comparison, the rest hyperparameters match those of our algorithm.
gi Ttthub..co/mmeetzzrr///ttooorore
7

---

Under review as a conference paper at ICLR 2023
IPM (A-0) (x)
IPM (A=10-1) (x)
IPM
IPM
IPM
IPM
IPM
IPM
OTS
Input 50
(A-10%)
(A=10')
(A=10*)
(A-10")
AA 100)
(A=10")
[ours] Tix
y=Tb)
(2)
('
(x)
(2()
(*
('(x)
Figure 5: Comparison of OTS (ours), regularized IPM GAN on the Wasserstein-2 benchmark. The 1st line shows blurry faces x ~ P., the 2nd line, clean faces y = T (x), where T is the OT map from IP to Q. Next lines show maps from Q fitted by the methods.
Metricsl Method
Regularized IPM GAN (WGAN-GP. AGP = 10)
OTS
X = 0
X 10
X = 10
X 10
X == 10 A = 10 A = 10
A = 10
(ours)
CP-UVP
4
25.2%
16.7%
17.7%
12.0%
4.0%
14.0%
28.5%
30.5%
1.4%
FID)
57.21
16.23
40.04
12.89
24.25
187.95
332.7
331.7
15.65
PSNRT
17.90
19.76
19.34
20.81
25.58
19.91
16.90
16.52
30.02
SSIM
0.565
0.655
0.656
0.689
0.859
0.702
0.520
0.498
0.933
LPIPS)
0.135
0.093
0.099
0.081
0.031
0.172
0.429
0.446
0.013
Table 2: Quantitative evaluation of restoration maps fitted by the regularized IPM GAN,
OTS (ours) using the Wasserstein-2 images benchmark (Korotin et al.) 2021).
We train the regularized WGAN-GP with various coefficients of content loss À {0, 10 109) and show the learned maps T and the map π obtained by OTS in Figure
Results. The performance of the regularized IPM GAN significantly depends on the choice of the content loss value A. For high values À > 10², the learned map is close to the identity expected. For small values À S 10², the regularization has little effect, and WGAN-GP solely struggles to fit a good restoration map. Even for the best performing X 10 all metries are notably worse than for OTS. Importantly, OTS decreases the burden of parameter searching as there is no parameter X.
7.2 BARGE-SCALLE EVALUATION
For evaluating our method at large-scale, we employ the dataset by (Lugmayr et al. 2019b) of AIM 2019 Real-Worid Sesor-Resolution Challenge (Track 2). The train part contains 800 HR images with up to 2040 pixels width or height and 2650 unpaired LR images of the same shape. They are constructed using artificial, but realistic, image degradations. We quantitatively evaluate our method on the validation part of AIM dataset that contains 100 pairs of LR-HB images.
Baselines. We compare OTS AIM dataset with the bicubic upsample, FSSR (Fritsche al. 2019) and DASR (Wei et al. 202T) methods. FSSR method is the winner of AIM 2019 Challenge; DASR is current state-of-the-an method for unpaired image SR. Both methods utilize the idea of frequency separation and solve the problem in two steps. First, they train a network to generate LR images. Next, they train super-resolution network using generated eseudo-pairs. Differently to FSSR, DASR also employs real-world LR images for training SR network taking into consideration the domain gap between generated and real-world LR images. Both methods utilize several losses, e.g., adversarial and perceptual, either on the entire image or on its high/low frequency components. For testing FSSR and DASR, we use their official code and pretrained models.
Implementation details. We train the networks using x28x 128 HR, 32x 32 LR random patches of im- ages augmented via random flips, rotations. We conduct separate experiments using EDSR as the trans- port map and either MSE or perceptual cost, and denote them as OTS (MSE), OTS (VGG) respectively. Metrics. We calculate PSNR, SSIM, LPIPS, FID. FID is computed on 32 x 32 patches of LR test images upsampled by the method in view w.r.t random patches of test HR. We use 50k patches to compute FID. The other metrics computed the entire upsampled LR test and HR test images.
8

---

Under review as a conference paper at ICLR 2023
Bicubic upsample
GT
OTS (MSE)
OTS (VGG)
FSSR
DASR
TATE BANK O
MARKET BATE BANK O
MARKET BATAT BANK C
AAE BANK O
MARKET BA BANK O
MARKET BE AAE BANK O
MARKET BE
MARKET BR
Figure Qualitative results of OTS (ours), bicubic upsample, FSSR and DASR AIM 2019 dataset x350x350 crops).
Experimental results given in Table Figure 6 The results show that the usage perceptual cost function in OTS boosts performance. According to FID, OTS with perceptual cost function beats DASR. On the other hand, it outperforms FSSR in PSNR, SSIM and, importantly, LPIPS. Note that bicubic upsample outperforms all the methods, according only to PSNR and SSIM, which have issues stated in 7 According to visual analysis, OTS with the perceptual cost better deals with noise artifacts. Additional results are given in Appendix E We also demonstrate the bias issue of FSSR and DASR in Appendix B
Method
FID L
PSNR
SSIM
LPIPS
Bicubic upsample
178.59
22.39
0.613
0.688
OTS (MSE)
139.17
19.73
0,533
0.456
OTS (VGG)
89.04
20.96
0.605
0.380
53.92
0.514
0.390
DASR
124.09
21.79
0,577
0,346
Table 3: Comparison OTS (ours) with FSSR, DASR on AIM19 dataset. The 1st, 2nd, 3rd best results are highlighted in green, blue and underlined, respectively.
DISCUSSION
Significance. Our analysis connects content losses in GANs with OT and reveals the bias issue. Content losses are used wide range tasks besides SR, e.g., in the style transfer and domain adaptation tasks. Our results demonstrate that GAN-based methods in all these tasks may a priori lead to biased solutions. In certain cases it is undesirable, c.g., in medical applications (Bissoto al., 2021). Failing to learn true data statistics (and learning biased ones instead), e.g., in the super-resolution of MRI images, might lead to a wrong diagnosis made by doctor due to SR algorithm drawing inexistent details the scan. Thus, we think is essential emphasize and alleviate the bias issue, and provide a way to circumvent this difficulty.
Potential Impact. We expect our OT approach improve the existing applications of image super- resolution. Importantly, it has less hyperparameters, uses smaller number of neural networks than many existing methods (see Table in Appendix [ comparison), and end-to-end this should simplify its usage in practice. Besides, our method generic and presumably can be applied to other unpaired learning tasks as well. Studying such applications is promising avenue for the future work.
Limitations. Our method fits a one-to-one optimal mapping (transport map) for super-resolution which, in general, might not exist. Besides, not all optimal solutions our optimization objective are guaranteed to be OT maps. Moreover, might encounter training issues similar to those of the GAN-based approaches. These limitations suggest the need for further theoretical analysis and improvement of our method for optimal transport.
method requires solving the saddle point problem. Thus,

---

Under review as a conference paper at ICLR 2023
REFERENCES
Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. arXiv preprint arXiv: 01777001.462, 2017.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214-223. PMLR, 2017.
Yogesh Balaji, Rama Chellappa, and Soheil Feizi. Robust optimal transport with applications in generative modeling and domain adaptation. Advances in Neural Information Processing Systems, 23:12934-12944 2020.
Alceu Bissoto, Eduardo Valle, and Sandra Avila. Gan-based data augmentation and anonymization for skin-lesion analysis: A critical review. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 1847-1856, 2021.
Olivier Bousquet, Sylvain Gelly, Ilya Tolstikhin, Carl-Johann Simon-Gabriel and Bernhard Schoelkopf. From optimal transport to generative modeling: the vegan cookbook. arXiv preprint arXiv:1705.07642. 2017.
Adrian Bulat, Jing Yang, and Georgios Tzimiropoulos. To learn image super-resolution, use a gan to learn how do image degradation first. European Conference on Computer Vision, 09 2018.
Jiaojiao Fan, Shu Liu, Shaojun Ma, Yongxin Chen, and Haomin Zhou. Scalable computation of monge maps with general costs. arXiv preprint arXiv:2106.03812 2021.
Manuel Fritsche, Shuhang Gu, and Radu Timofte. Frequency separation for real-world super- resolution. In IEEF/CVF International Conference on Computer Vision (ICCV) Workshops, 2019.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural informa- tion processing systems. pp. 2672-2680, 2014.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of Wasserstein GANs. In Advances in Neural Information Processing Systems, pp. 5767-5777, 2017. 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in neural information processing systems, pp. 6626-6633 6637, 2017.
Xun Huang, Ming- Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-image translation. In Proceedings of the European conference on computer vision (ECCV), pp. 172-189, 2018.
Leonid Kantorovitch. On the translocation of masses. Management Science, 5(1):1-4, 1958.
Gwantae Kim, Jaihyun Park, Kanghyu Lee, Junyeop Lee, Jeongki Min, Bokyeung Lee, David K. Han, and Hanseok Ko. Unsupervised worldw super resolution with cycle generative adversarial network and domain discriminator. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition Workshops (CVPRW), pp. 1862-1871, June 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1rX.v:1412.6980, 2014.
Alexander Korotin, Lingxiao Li, Aude Genevay, Justin M Solomon, Alexander Filippov, and Evgeny Burnaev. Do neural optimal transport solvers work? a continuous wasserstein-2 benchmark. Advances in Neural Information Processing Systems, 34, 2021.
10

---

Under review as a conference paper at ICLR 2023
Alex Krizhevsky, Ilya Sutskever, and Geoftrey E. Hinton. Imagenet classification with deep convolu- tional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems Volume 1, NIPS' 12, pp. 1097-1105, Red Hook, NY, USA, 2012. Curran Associates Inc.
Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. Deep laplacian pyramid networks for fast and accurate super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 624 632, 2017.
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo-realistie single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681 2017.
Chun-Liang Li, Wei- Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. MMD GAN: Towards deeper understanding of moment matching network. In Advances in Neural Information Processing Systems, pp. 2203-2213 2017.
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 136-144, 2017.
Huidong Liu, Xianfeng Gu, and Dimitris Samaras. Wasserstein GAN with quadratic transport cost. In Proceedings of the IEEE International Conference on Computer Vision, pp. 4832-4844 2019.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In Proceedings of International Conference Computer Vision (ICCV), December 2015.
Guansong Lu, Zhiming Zhou, Jian Shen, Cheng Chen, Weinan Zhang, and Yong Yu. Large-scate scale optimal transport via adversarial training with cyclecconsistency. arXiv preprint 0rXiv:2003306635 2020.
Andreas Lugmayr, Martin Danelljan, and Radu Timofte. Unsupervised learning for real-world super- resolution. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3408-3416, 2019a.
Andreas Lugmayr, Martin Danelljan, Radu Timofte, Manuel Fritsche, Shuhang Gu, Kuldeep Purohit, Praveen Kandula, Maitreya Suin, AN Rajagoapalan, Nam Hyung Joon, et al. Aim 2019 challenge real-world image super-resolution: Methods and results. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3575-3583 IEEE, 2019b.
Shunta Maeda. Unpaired image super-resolution using sseudo-supervision In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 291-300, 2020.
Anton Mallasto, Jes Frellsen, Wouter Boomsma, and Aasa Feragen. (q. p)- aasserttnn GANs: Comparing ground metrics for Wasserstein GANs. arXiv preprint 102.64490. 2019.
Youssef Mroueh, Chun-Liang Li, Tom Sercu, Anant Raj, and Yu Cheng. Sobolev gan. arXiv preprint arXiv:1711.04844 2017.
Jim Nilsson and Tomas Akenine-Möller. Understanding ssim. arXiv preprint arXiv:2006.1338 13846 2020.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: using variational divergence minimization. In Advances in neural information processing systems, pp. 271-279, 2016.
Training generative neural samplers
Tyrrell Rockafellar. Integral functionals, normal integrands and measurable selections. In Nonlinear operators and the calculus of variations, pp. 157-207. Springer, 1976.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241 Springer, 2015.
11

---

Under review as a conference paper at ICLR 2023
Litu Rout, Alexander Korotin, and Evgeny Burnaev. Generative modeling with optimal transport maps. In International Conference on Learning Representations, 2022. URL [tLl/tttp:://ppeeeeeee net/ mut//Eorunin Btte//Eouunnn1155s.22s..
Filippo Santambrogio. Optimal transport for applied mathematicians. Birkäuser, NY, 55(58-63):94, 2015.
Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, and Kenji Fukumizu. Equivalence of distance-base based and rkhs-basee statistics in hypothesis testing. The annals of statistics, pp. 2263- 2291,2013.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
Yaniv Taigman, Adam Polyak, and Lior Wolf. Unsupervised cross-domain image generation. arXiv preprint arXiv:1611.0221 0.22000000022 2016.
Cédric Villani. Topics in optimal transportation. Number American Mathematical Soc., 2003.
Cédric Villani. Optimal transport: old and new, volume 338. Springer Science & Business Media, 2008.
Wei Wang, Haochen Zhang, Zehuan Yuan, and Changhu Wang. Unsupervised real-world super- resolution: A domain adaptation perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4318-4327, 2021.
Yunxuan Wei, Shuhang Gu, Yawei Li, Radu Timofte, Longeun Jin, and Hengjie Song. Unsupervised real-world image super resolution via domain-distance aware training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13385-13394, June 2021.
Yujia Xie, Minshuo Chen, Haoming Jiang, Tuo Zhao, and Hongyuan Zha. On scalable and efficient computation of large scale optimal transport. volume 97 of Proceedings of Machine Learning Research, pp. 6882 6892, Long Beach, Californi, USA, 09 15 Jun 2019. PMLR. URLhttp //proceedings mlr press Setress/v97/xiee wttp/x/ee9....a..e.... html
Yuan Yuan, Siyuan Liu, Jiawei Zhang, Yong bing Zhang, Chao Dong, and Liang Lin. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 814-81409, 2018.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unrcasonable effectiveness of deep features perceptual metric. In CVPR, 8a.
Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang. Bineng Zhong, and Yun Fu. Image super-resolution using very deep residual channel attention networks. In ECCV, 2018b.
Yuanbo Zhou, Wei Deng, Tong Tong, and Qinquan Gao. Guided frequency separation network for real-world super-resolution. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 428-429, 2020.
Jun Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired amage-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017.
12

---

Under review as a conference paper at ICLR 2023
A FIRST VARIATIONS OF GAN DISCREPANCIES VANISH AT THE OPTIMUM
We demonstrate that the first variation of Q - D(O', Q is equal to zero at Q = Q for common GAN discrepancies D. This suggests that the corresponding assumption of our Theorem I is relevant.
To begin with, for a functional G : P()) + R foo}, we recall the definition of its first variation. measurable function 8G [Q] y - I u [xc} is called the first variation of 9 at point Q P()). if, for every measure AQ on y with zero total mass (ʃy 1 dd)(() = 0),
(( = + e I/, (g(Q(()) daQ(U) + o(c)
(16)
for all 2 0 such that Q <AQ is a probability distribution. Here for the sake of simplicity suppressed several minor technical aspects, see (Santambrogio (015) Definition 7.12) for details. Note that the first variation is defined up to an additive constant.
Now we recall the definitions of three most popular GAN discrepancies and demonstrate that their first variation zero at an optimal point. We consider f fddivergences (Nowozin al., 2016). Wasserstein distances (Arjovsky et al. 2017). and Maximum Mean Discrepancies (Li et al. 2017).
Case (fvdeeergence). Let f R+ RR be convex and differentiable function satisfying f(1) O. The f-divergence between Q Q E
P()) is defined by
def J,/
(10'(w)
(Q)(v) (Q(
D2
(Q(v).
(17)
The divergence takes finite value only if Q' < Q i.e., C' is absolutely continuous w.r.t. Q. Vanilla GAN loss (Goodfellow et al. 2014) is a case of f-divergence (Nowozin et al. 2016 Table 1).
We define ((Q)) def Dj DQ( Q). For Q = Q and some AQ such that Q + eQQ E P(V) we derive ((Q+cAQ)= / / dQ(v) d3Q(v)
dQ(()) dQ(u)) dQ((v) ( -J= 1+ dAQ(v)
(
dQ(2))
(18)
dQ(s)
= =(DDRR + / (1) dAQ(v) dQ(y) o(c) = =Q(Q)+ + / ((I)dAQ())) o(e), (19)
dQ(
where in transition from 18 to 19 we consider the Taylor series w.r.t. at E = 0. We see that (g(()(() = (1) is constant, i.e., the first variation of Q' + (Q( . @) vanishes at Q' Q
Case 2 (Wasserstein distance). If in OT formulation B the cost function c(x, y) equals ||x y/I
with p 2 1, then [Cost(P, Q)] 1/r
is called the Wasserstein distance (W,). Generative models which typically called the Wasserstein GANs (WGANs). The most popular case is 1 (Arjovsky et al. 2017; Gulrajani et al. 2017). but more general cases appear in related 2019
use wp the discrepancy
work as well, (Liu et al.
Mallasto et al. 2019).
The first variation of (QQ') del wp(Q', Q) at point Q is given by (v) (v), where f is the optimal dual potential (provided it is unique up to a constant) in 44) for a pair (Q', Q). see (Santambrogio 2015 57.2). Our particular interest is compute the optimal potential (f*) at 0 to derive
V = Q We recall 4 and use Wp( Q)
[/ ((N)
QQ = 0= = sup
f
One may see that / = 0 attains the supremum (its c-transform (ʃ* " is also zero). Thus, if (f+) 0 a unique potential (up to a constant), the first variation of Q + wg(Q' Q) at Q' = vanishes.
Case 3 (Maximum Mean Discrepancy). Let k: y x y - RR be positive definite symmetric kernel. The (square) of the Maximum Mean Discrepancy between Q' Q is given by
del [ JYZ
2
k(m, (dn, Q') x (Q - -²) (v), v2)
= =f (2) ()) v2) - 2 S (()1.)2)d((' x @21, U2) + Const( QQ) (20)
13

---

Under review as a conference paper at ICLR 2023
Equation 3.3). The first variation of the quadratic in Q term is given by y 1 2 Jy ky, y2 2 (22). ). see (Santambrogio, 2015 $7.2). The second term is linear in Q and
(Sejdinovic et al. 2013
its first variation is simply + (-2) Jy k(y V/2 (QQ When Q = QQ the sum of these terms is
zero. That is, the first variation of the functional Q MMD
vanishes Q Q
B ASSESSING THE BIAS OF METHODS ON AIM19 DATASET
Test
Test
Bicubic Upscale
(a)
(b)
(c)
Green
Green
Red
Red
Red
OTS (ours)
FSSR
DASR
(d)
(e)
Green
Green
Red
Red
Red
Dataset
Test HR
Test LR
Bicubic
OTS (VGG)
FSSR
DASR
0.24
0.17
0.15
0.20
0.17
0.15
Variance
+0.01
+0.01
+00.02
+00.03
0.02
+0002
Figure Color palettes and their variance for Test HR, LR datasets and solutions Bicubic Upscale, OTS, FSSR, DASR methods on AIM19.
We additionally demonstrate the bias issue by comparing color palettes of HR images and super- resolution results of different methods, see Figure 7 We construct palettes by choosing random image pixels from dataset images and representing them
RGB point cloud in [0, 1] RR. Figure 7 shows that OTS (d) captures large contrast of HR (a) images (variance of its palette), while FSSR (e), DASR (f), Bicubic Upscale (c) palettes are less contrastive and closer LR (b). We construct palettes 100 times to evaluate their average contrast (variance). The metric quantitatively confirms that our OTS method better captures the contrast of HR dataset, while GAN-based methods (FFSR and DASR) are notably biased towards LR dataset statistics (low contrast).
C TRAINING DETAILS
Perceptual cost. In 7.2 we test following perceptual cost as b:
b(2 MSE(MEEEL y) +1/s MAE(x y) +/500 Σ MSE(ff(2 fa())):
k6(3,8,15,22)
where ft denotes the features of the kth layer of pre-trained VGG-16 network (Simonyan & Zisserman, 2014 MAE the mean absolute error MAE(s, v)
|---|l dim
Dynamic transport cost. In the preliminary experiments, we used bicubic upsampling as the "Up" operation. Later, found that the method works better if gradually change the upsampling. We start from the bicubic upsampling. Every k. iterations of f. (see Table we change the cost to c(x, y) (x)) v). where T% is fixed frozen copy of the currently learned SR map To.
14

---

Under review as a conference paper at ICLR 2023
Hyperparameters. For EDSR, set the number of residual blocks to 64, the number of features to 128, and the residual scaling to For UNet, we set the base factor to 64. The training details given in Table 4 We provide comparison of the hyperparameters of FSSR, DASR and OTS (ours) in Table In contrast to FSSR and DASR, our method does not contain a degradation part. This helps notably reduce the amount of tunable hyperparameters.
Optimizer. We employ Adam (Kingma & Ba 2014).
Computational complexity. Training OTS with EDSR the transport map and the perceptual transport cost on AIM 2019 dataset takes Ra days on single Tesla V100 GPU.
Experiment
dim(2t)
dim()/)
f
T
key
Irs
Irr
Initial cost
Total iters (f)
Cost update every
Batch
size
Benchmark
(7.1)
64
UNet
MSF
10K
64
Copta
Bilincar UNet
100K
23K
64
16
ResNet
EDSR
10
10
Bicubic MSE
100K
25K
AIM
32 32
128 (patches)
128
EDSR
50K
25K
7.2
(patches)
EDSK
Bicubic VGG
50K
20K
Table 4: Hyperparameters that we in the experiments with our Algorithm
Method
Degradation part
Super-resolution part
Total
2 neural networks;
2 neural networks;
4 neural networks;
2 optimizers; schedulers;
2 optimizers;
4 optimizers; schedulers;
FSSR
2 schedulers;
adversarial loss:
adversarial loss:
adversarial losses;
content loss (pprreppt))al) content loss (E +perceeptual content losses (& perceeptull)
2 neural networks;
2 neural networks;
neural networks;
2 optimizers;
2 optimizers;
optimizers;
DASR
2 schedulers;
2 schedulers;
4 schedulers;
adversarial loss:
adversarial loss:
adversarial losses;
content loss (& +perceptual)
content loss (&1 +perceptual) content losses (E +perceptual)
2 neural networks;
neural networks;
OTS (ours)
2 optimizers; cost (f2+ +perceptaa))))
optimizers:
cost (f2+L) +perccptu)))))
Table 5: Comparison of hyperparameters used in FSSR, DASR and OTS (ours) methods.
D IMAGE SUPER- -HEE-RESOOOTION OF FACES
We conduct experiment using CelebA (Liu et al. 2015) faces test the applicability of OT for unpaired SR. We test our Algorithm with MSE as the cost and UNet or EDSR as the transport map.
Method
FID J
PSNR
SSIM 1 LPIPS L
Sicubic
Bicuhic
130.72
22.73
0.756
0.303
upsample
OTS (ours) UNet
OTS (ours)
12.32
22.10
0.740
0.058
UNet
OTS (ours) EDSR
OTS (ours) EDSR
15.87
22.33
0.747
0.054
Table 6: Comparison of OTS (ours) with the bicubic upsampling on CelebA dataset. The 1st and 2nd best results are highlighted in green and blue, respectively.
Figure 8: Qualitative results of OTS (ours) CelebA.
Pre-processing and train- test split. We resize images to 64 x 64 px. We adopt the unpaired train-test split from (Rout et al. 2022, $5.2). We split the original HR dataset in parts A, B, C containing 90K, 90K, 22K samples, respectively. We apply the bicubic downsample to each image and obtain the L.R dataset (16x 16 faces). For training, we use LR part A, HR part B. For testing,
parts C.
15

---

Under review as a conference paper at ICLR 2023
Metrics. We compute PSNR, SSIM, LPIPS and FID metries on the test part, see Table
E ADDITIONAL RESULTS ON AIM19
Bicubic upsample
OTS (MSE)
OTS (VGG)
FSSR
DASR
Bicubic upsample
OTS (MSE)
OTS (VGG)
FSSR
DASR
Figure 9: Additional qualitative results of OTS (ours), bieubie upsample, FSSR and DASR on AIM 2019 x800 crops).
16

---

Under review as a conference paper at ICLR 2023
Bicubic upsample
GT
OTS (MSE)
OTS (VGG)
FSSR
DASR
Bicubic upsample
OTS (MSE)
OTS (VGG)
FSSR
DASR
Figure 10: Additional qualitative results of OTS (ours), bicubic upsample, FSSR and DASR on AIM 2019. The sizes of crops on the st and 2nd images are x50 350 and 800x respectively.
17

---

Under review as a conference paper at ICLR 2023
F CONNECTION BETWEEN GAN OBJECTIVES AND EQUATION
5
Typical objectives of GAN-based approaches consist of multiple losses - usually one adversarial and several content losses. To make the exposition simple, in our paper, we represented all the content losses as a single loss .). Below we provide several examples showing how the objectives of popular RAN-based approaches to unpaired image SR could be viewed [ For all of these methods, our Lemma applies without any changes. We include in brackets the number of papers citations according to Google Scholar to show that chosen methods are widely used.
FaceSR (2018, 284 citations) The paper of (Bulat et al. 2018) presents one of the first GAN-based approaches unpaired image SK problem. The method is composed of two steps. First, it learns a degradation between unpaired HR and LR images, Then it employs a second GAN to learn a supervised mapping between paired generated LR and corresponding HR images. The objective of the unpaired step (see their Eq. (1)) is as follows:
I
olgixel
AlGAN-
content loss
adversarial looss
Here (pixet is the MSE loss between the generated LR image and downsampled HR. Thus, the objective of this method exactly follows Equation 5
CinCGAN (2018, 400 citations) The method of (Yuan et al. 2018) is an other pioneering GAN-based approach unpaired image SR problem, which establishes different to FaceSR group of two-step methods. First, it uses one CycleGAN to learn a mapping between given noisy LR images and downsampled HR ("clean LR") images. Then, second CycleGAN fine-tunes a mapping between real LR and HR images. The objective for the first GAN (see their Eq. (5)) is as follows:
C
CR
+ Clye + W22 LR + W/ CTV
GAN
adversarial loss
content loss 1, identity loss, and CTB total variation loss.
Here fly is the cycle-consistency loss
idt
FSSR (Winner of the AIM Challenge on Real-Worlh SR (Lugmayr et al. 2019b). 2019, 127 citations) FSSR (Fritsche et al. 2019) method employs similar to FaceSR strategy. It firstly learns a mapping between downsampled HR images and given unpaired LR images, and then uses the generated pairs to learn supervised SR model. The objective of the unpaired step (see their Eq. (6)) is defined by:
L 0.005 + Cool. + 0.001
content loss where the texture (adversarial) loss Liex, a and the color (I1 identity) loss Lool a are applied to low frequencies of the images, while the perceptual loss Lper, d to the features of the full images.
adversarial loss
DASR (2021, 51 citations) DASR (Wei et al., 2021) structure is also based on the similar to FSSR principles and its two-step structure. In contrast to FSSR, a SR network is trained in a partially supervised manner using not only generated, but also real LR images. The objective of the fully unpaired degradation learning step (see their Eq. (4)) is as follows:
CDSs = aLeon + 3.pr
content loss
adversarial loss
Here the adversarial loss CG, is defined on high frequencies of the image, while the content Ccon (4 identity) and the perceptual Eper losses are defined on full images and their features respectively.
ESRGAN-FS (2020, 13 citations) ESRGAN-FS is an other two-step approach based on the principle of learning the degradation, see (Zhou et al. 2020). The objective of its unpaired degradation learning step (see their Eq. (4)) is as follows:
Liotal
Liow + Xa2 Eper + dra Lnigh
content loss
adversarial loss
Here Lrow (4 identity) loss is applied to low frequencies of the images, the perceptual loss Lper to the features of the full images, while Lhigh (adversarial loss) - high frequencies of the images.
$ Sc is detined as the MSE loss between given LR image and Gz(Gi(z)), where G1 leams to map real LR images to "clean" ones and G2 learns opposite mapping. For a fixed G2 this loss can be considered as part of the content loss.
18


Reviews:
[{"review_id": "WSWFhJ0QIIm", "reviewer": "Reviewer_mWTP", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces  an algorithm for unpaired  super-resolution which learns an unbiased OT map for the perceptual transport cost.  Unlike the existing GAN-based alternatives, this proposed  algorithm has a simple optimization objective reducing the need for complex hyperparameter selection and an application of additional regularizations.\n\nContributions: \nThis paper provides an algorithm to fit an unbiased OT map for perceptual transport cost and apply it to the unpaired image SR problem. In addition, it investigate the GAN optimization objectives regularized with content losses.", "summary_of_the_review": "This paper introduces  an algorithm for unpaired  super-resolution which learns an unbiased OT map for the perceptual transport cost.  This proposed algirithom extends the applications of existing SR practice.\nThe approach is technically relatively sound with some small flaw.  the paper is well-organized and clearly written. \nMore experimental evaluations are helpful to prove the effectiveness of the algorithm.", "strength_and_weaknesses": "This paper has a sufficient theoretical analysis, but its weaknesses is that the empirical evaluation is not sufficient.\nOnly two methods FSSR (Fritsche et al., 2019) and DASR (Wei et al., 2021) have been compared on the dataset AIM 2019, and both methods use two GANs. There are many existing SR methods based on a single WGAN. What is the reason why this paper does not compare with them?\nIn addition, the following issues still need to be further explained.\n1. As shown in equation 12, the proposed method fits a one-to-one optimal mapping (transport map) for super-resolution which, in general, might not exist.  To solve this problem, what special settings are required in real world super-resolution tasks?\n2. The author declares that the proposed OTS is different from the regularized IPM GAN, Because in OST,  the OT map T is a solution to the inner optimization problem, while in IPM GAN the\ngenerator T is a solution to the outer problem.  But from algorithm 1, there is no difference between finding inf first and finding sup first.\n3. The author declares  ' in OTS the optimization over potential f is unconstrained, while in IPM GAN it must belong to F '.  \n     This is not fair.  On the one hand, both potential functions need to be realized by neural networks. On the other hand, for  regularized IPM GAN, distance measure is also optional, but the two algorithms referred to in this paper are  Wasserstein-1.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "T7zyudZx9Mc", "reviewer": "Reviewer_GsaW", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces some new ideas and findings, by analyzing GAN-based models that learn constrained mappings between domains, through the perspective of optimal transport. This is important, since such GANs are used extensively for a variety of tasks, e.g. 'inverse' problems like super-resolution, denoising and debluring.\n\nSince the setting is an 'unpaired' one, in addition to the goal of having the 'generated' target distribution close to the 'real' one (this is enforced by the discriminator), there is a need to constrain the generator to keep the output close to the input, which is achieved by a 'cost' (e.g. 'content') function for minimizing such differences.\n\nIn this setting, they show that the learned generator (T) is an optimal transport map between the input distribution (e.g. LR images) and the image of T (the generated distribution), under the 'content' cost function. Furthermore, they show that the generator (or transport map) is biased, in the sense that perfect optimization will necessarily not lead to the true target distribution, but rather will be 'biased' towards the input distribution.\n\nAs an alternative, they suggest directly optimizing for an optimal transport map between source and target distributions, under the content cost. Several derivations and reasonable assumptions, lead to a formulation and GAN-type optimization scheme, which is verified empirically in several settings. ", "summary_of_the_review": "The paper was very inspiring and interesting to read, especially in terms of the OT point of view on GAN.\nHowever, I am not convinced that the analysis leads to a result that is significant enough in terms of its ability to overcome GAN's weaknesses, especially since the algorithm itself does not seem to differ significanlty from GAN (see above).\nThe bias of the generator is certainly not the only important factor in GAN quality and the different formulation might effect performance in other ways. The bias claim is empirically condsidered only in Section B of the appendix, only in terms of the color distribution variance, while the qualitative and quantitave results on that data (Aim19 dataset) are not better than those of the other methods.", "strength_and_weaknesses": "Strengths:\n1) The relations between the standard GAN formulation/optimization and different versions of optimal-transport schemes with respect to the 'content' cost are very interesting and mostly new. Most importantly, in my opinion, is that they give a new way of thinking about and analyzing different approaches under this setup.\n2) The introduction is written very well, in term of background, context and motivation. Also, the main derivations, while being very formal, are clearly and well explained.\n\nWeaknesses:\n1) The first observation regarding GAN (Lemma 1 - that it is a particular OT map) is pretty trivial, although interesting. I wouldn't agree that the main observation (Thm1 - that map/generator is biased towards the input) is surprising. In fact, the addition of the 'content' cost of closeness between the input and output obviously constrains the output in such a way, which is probably inevitable in the 'unpaired' setup.\n2) There is a problem with the logic that follows Lemma 3 - It wrongly claims that \"Lemma 3 states that one can solve a saddle point problem (12), obtain an optimal pair (f∗, T∗), and use T ∗ as an OT map from P, Q\". It was shown that the true T* can be chosen as optimal along with the true f*, but this doesn't show that the map obtained along with f* is actually that one and not another. The following sentence \"For general P, Q, the arg infT set for an optimal f∗ might contain not only OT map T ∗ but other functions as well. However, our experiments (M7) show that this is not a serious issue in practice\" only acknowledges this problem, but doesn't explain how it is dealt with.\n3) It seems like the final obtained algorithm is not truely different from the standard GAN optimization. There are 3 main differences that are highlighted between the optimization problems, but it seems that in the optimization is the same in terms of the same alternating generator and discriminator loss updating, exept that the 'content' loss has a weight (lambda) of 1.\n4) In terms of the results, overall - the method is shown to work, but is in par with other methods including the baseline GANs. The claim that one does not have to pick the lambda parameter isn't very influencial, since the GAN results work well with a particula choice of lambda.\n5) In terms of framing, I didn't find anything that was specific to super-resolution (and in fact the method is also demonstrated on a deblurring task). The analysis and solution are relevant to a much wider set of tasks over the unpaired setting of two domains. \n6) GAN training suffers from know stability issues. There should be some reference to this aspect of the optimization as well.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Gx028eSCoGP", "reviewer": "Reviewer_nWwQ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work theoretically investigate the unpaired techniques based on Generative Adversarial Networks (GANs) which arise in  super-resolution tasks, and find two observations: first the learned SR map is always an optimal transport map; second, the learned map is biased. The observations are supported by theoretical proofs and empirical experiments. The work proposes an algorithm for unpaired SR which learns an unbaised OT map for the perceptual transport cost. This method reduces the need for complex hyperparamter selection and additional regularizations. The proposed model provides a nearly state-of-the-art performance on unpaired AIM19 datasest.\n\nThe contributions are:\n1. the work investigate the GAN optimization objectives regularized with content loss, and prove the solutions are always OT maps, but biased;\n2. the work provide an algorithm to fit an unbiased OT map for perceptual transport cost and applied it for the SR problem, establishes connections between the proposed model and regularized GANs using IPMs as a loss.\n", "summary_of_the_review": "This work theoretically investigate the unpaired techniques based on Generative Adversarial Networks (GANs) which arise in  super-resolution tasks, and find two observations: first the learned SR map is always an optimal transport map; second, the learned map is biased. The observations are supported by theoretical proofs and empirical experiments. The work proposes an algorithm for unpaired SR which learns an unbaised OT map for the perceptual transport cost. This method reduces the need for complex hyperparamter selection and additional regularizations. The proposed model provides a nearly state-of-the-art performance on unpaired AIM19 datasest.\n\nThese observations are novel, it discovers the intrinsic connection between SR problem and OT theory. The work is very well written, the problem statement, the mathematical formulation, key lemmas and main proofs are very clean the elegant, the deduction is easy to follow, the experimental results are well analyzed and convincing.  The authors provide source codes, it is easy to reproduce. In summary, the work is inspiring and promising, it has both theoretical and practical values.", "strength_and_weaknesses": "The strength of the work are:  it has unique observations on conventional GANs for unpaired SR problem, and proves the solutions of GANs are OT maps, but biased, then provides an unbiased solution; The mathematical model and formulations are elegant and powerful, the algorithm is effective; the experimental results show the proposed method achieves SOTA in SR applications; it is a good example of combing rigorous mathematical model with real world applications.\n\nThe weakness is that the minmax optimization proposed by the work can find the saddle points, which may not correspond to optimal solutions, this step needs further exploration.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Ehs10b435_2", "reviewer": "Reviewer_xhtM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper analyzed the results of optimal transport regularized GANs, and claimed that the optimal transport map for optimal transport regularized GANs are biased. The authors proposed to use the method in Rout et al. 2022 (OTM) for the image Super Resolution (SR) problem. Experiments are conducted on a face dataset (CelebA?) and the AIM dataset. ", "summary_of_the_review": "This paper lacks novelty and the experimental results are not good. ", "strength_and_weaknesses": "## Strength: \n\n1. This paper provides details to analyze the transport plan of optimal transport regularized GANs. The observation is that the solution is biased. \n2. The connection and difference between the Regularized IPM GANs and the OTM are analyzed. \n3. The authors applied the OTM to the image Super Resolution (SR) problem. \n\n## Weakness\n\n1. The theoretical results are not novel. \n    1. In Sec. 5, the authors proved that adding a regularization term, the $\\mathcal{R}\\_c(T)$ in Eq. (5) will not be able to recover the true target distribution $\\mathbb{Q}$. This does not seem that novel to me because regularization technique is a well-known technique in machine learning. Adding a regularization term to a loss will prevent the loss term ($\\mathcal{D}$) from decreasing to zero, such that the $T^*_{\\\\#}\\mathbb{P}$ does not equal to $\\mathbb{Q}$. This is expected. In general, the purpose of the regularization term is to prevent overfitting or stabilize training GANs. \n\n     2. The theoretical results in Sec. 6 are not novel. Almost all results could be found in [1]. If one retain the constants of Eqs. (8) - (11) in [1] and replace $\\frac{1}{2} || x - y ||^2$ by $c(x, y)$ and replace $\\psi(y)$ by $v(y)$ in [1], one will find that Lemma 3 in this paper is equivalent to Eq. 12 (Lemma 4.1) in [1]. As also mentioned in [1], \"we focus on the quadratic ground cost $\\frac{1}{2} || x - y ||^2$. Nevertheless, our approach extends to other costs $c(\\cdot,\\cdot)$\". In adddition, Algorithm 1 in this paper is almost the same as Algorithm 1 in [1] by using the identity embedding $Q$, and a general cost $c(\\cdot,\\cdot)$ in Algorithm 1 of [1].  Unfortunately, the authors did not even mentioned [1] in the whole Sec. 6. \n\n2. The authors made incorrect claim “The solution of the regularized GAN is an OT map” in Lemma 1. The authors tend to mix the optimal transport GAN or Wasserstein GAN as the whole GAN family. However, these GANs are just part of the GAN family. The authors analyzed Biased Optimal Transport in GANs in Sec. 5. However, I want to mention that not all GANs use the optimal transport or the Wasserstein distance, nor do all the GANs uses the optimal transport regularizer as wrote in Eq. (5). Therefore, it is inappropriate to claim that minimizer of a regularized GAN problem is always an OT map as mentioned in the abstract and Lemma 1. \n\n3. The experimental results are not good. On the AIM dataset, the proposed OTS variants do not perform well as the DASR method in Table 3. Also, why not compare with other SR methods such as DASR on the face (CelebA?) dataset?\n\nReferences: \n\n[1] Rout et al., Generative modeling with optimal transport maps, ICLR 2022. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject