QRGAN: Quantile Regression Generative Adversarial Networks
Authors: Sunyeop Lee
Tuan Anh Nguyen
Dugki Min
Venue:  (2021)
Keywords: Quantile Regression
Generative Adversarial Networks (GANs)
Frechet Inception Distance (FID)
Generative Neural Networks

Abstract: Learning high-dimensional probability distributions by competitively training generative and discriminative neural networks is a prominent approach of Generative Adversarial Networks (GANs) among generative models to model complex real-world data. Nevertheless, training GANs likely suffer from non-convergence problem, mode collapse and gradient explosion or vanishing. Least Squares GAN (LSGANs) and Wasserstein GANs (WGAN) are of representative variants of GANs in literature that diminish the inherent problems of GANs by proposing the modification methodology of loss functions. However, LSGANs often fall into local minima and cause mode collapse. While WGANs unexpectedly encounter with inefficient computation and slow training due to its constraints in Wasserstein distance approximation. In this paper, we propose Quantile Regression GAN (QRGAN) in which quantile regression is adopted to minimize 1-Wasserstein distance between real and generated data distribution as a novel approach in modification of loss functions for improvement of GANs. To study the culprits of mode collapse problem, the output space of discriminator and gradients of fake samples are analyzed to see if the discriminator guides the generator well. And we found that the discriminator should not be bounded to specific numbers. Our proposed QRGAN exposes high robustness against mode collapse problem. Furthermore, QRGAN obtains an apparent improvement in the evaluation and comparison of Frechet Inception Distance (FID) for generation performance assessment compared to existing variants of GANs.

Under review as a conference paper at ICLR 2021
QRGAN: QUANTILE REGRESSION GENERATIVE AD- VERSARIAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Learning high-dimensional probability distributions by competitively training generative and discriminative neural networks a prominent approach of Gener- ative Adversarial Networks (GANs) among generative models to model complex real-world data. Nevertheless, training GANs likely suffer from non-convergence problem, mode collapse and gradient explosion or vanishing. Least Squares GAN (LSGANs) and Wasserstein GANs (WGAN) are of representative variants GANs in literature that diminish the inherent problems of GANs by proposing the modification methodology of loss functions. However, LSGANs often fall into local minima and cause mode collapse. While WGANs unexpectedly encounter with inefficient computation and slow training due to its constraints in Wasserstein distance approximation. In this paper, we propose Quantile Regression GAN (QR- GAN) in which quantile regression is adopted to minimize 1-Wasserstein distance between real and generated data distribution as a novel approach modification of loss functions for improvement of GANs. To study the culprits of mode col- lapse problem, the output space of discriminator and gradients of fake samples are analyzed to see if the discriminator guides the generator well. And we found that the discriminator should not be bounded to specific numbers. Our proposed QRGAN exposes high robustness against mode collapse problem. Furthermore. QRGAN obtains an apparent improvement in the evaluation and comparison of Frechet Inception Distance (FID) for generation performance assessment com- pared to existing variants of GANs.
INTRODUCTION
Deep learning-basse -baed data generation techniques have proved their successes in many real-world applications. Thanks the rising of generative models, the generation of audio, images and videos, either unconditionally or conditionally, has achieved remarkable advancements in recent years. Text or structured data can be generated easily as well many recent studies. Data generation techniques bring about efficency and creativity in human activities at every conner of the world.
Among the most influent and successful methods for data generation experiments, Variational Au- toencoders (VAE) Kingma & Welling (214) and Generative Adversarial Networks (GANs) Good- fellow al. 2014) are the fundamental representatives generative models.
Variational Autoencoders (VAE): VAEs regularize the encoder output to be a known distribution. This regularization is applied to each sample. For latent variable and input x, p(2[z), not p(z), is pushed to the prior distribution. With the additional reconstruction loss, the two objectives may conflict each other. Usually, mean square error (MSE) loss is used for the reconstruction loss. Because if the global minimum MSE loss is at the expected value of the distribution, the decoders generate blurry outputs. PixeIVAE Gulrajani al. (2016) fixes VAE's blurry output by replacing MSE loss by PixelCNN van den Oord et al. (016) decoder.
Generative Adversarial Networks (GANs): GANs regularize the entire decoder input distribu- tion to be known distribution. In other words, GANs regularize p(2). not p p(z[z). Therefore, generators can generate sharp outputs without the conflicting objectives in VAEs. A number of GAN variants has been presented in literature to improve the data generation including Least-square

---

Under review as a conference paper at ICLR 2021
GANs (LSGANs) Mao et al. 2019), Wasserstein GANs (WGANs) Arjovsky et al. (2017). Latent-
GAN Prykhodko et al. (2019). Adversarial Autoencoders (AAEs) Makhzami et al. (2016). LSGANS and WGANs improved the objective function for better generation quality. LatentGAN generates latent variables for the given autoencoder using GAN methods. AAES replace the KI. regularization from VA by discriminator that distinguishes the encoder output distribution (generated data) and known distribution (real data). Although GANs can obtain better generation output, they are diffi- cult train because of the framework architecture which two networks compete. Mode collapse is common failure in GAN frameworks that generator outputs very similar samples only. Mode collapse is caused by unstable training and improper loss function.
Recent work targets to find the best method to train GANs with better quality and less mode collapse. The work WWittakk et al. (020)) shows huge number of existing studies that put an effort to improve GANs. The work also explains GANs' recent successes and problems. Accordingly, several GAN variants are introduced stabilize the GAN training such DCGAN
Variants of GANs:
Radford et al. (2016), LSGAN, WGAN-GP Gulrajani et al.] (2017), and Fisher GAN MMoueh & Sereu 2017). In Least Square Generative Adversarial Networks Mao al. 2017);[Mao et al. al. (2019) (LSGANs), the authors propose to use mean square error (MSE) which does not saturate. Also, they found that training generator make samples near the decision boundary instead of trying to overwhelm the discriminator results better models. Alternatively, Wasserstein GANs Arrovsky al. (2017) applies Lipschitz condition to approximate 1-Wasserstein distance. In the original WGAN, they implement the constraint by clipping weights of critic network into range specified by hyperparameters. WGAN-GP is introduced to improve training of WGAN. WGAN-GP give gradient penalty to push weight gradient norm of critic to 1. Thus, the generator can easily escape from local minima. However, it takes more time for computation of gradients for random inputs.
Reinforcement Learning (RL): is to learn the optimal policy in a given environment. The agent read an observation (state) from the environment, calculate the optimal action, and take the action. The environment returns the next state with the reward by the action. By trial and error, the agent learns the optimal action for each state and gradually reach to the optimal policy. For given time t, state s. action a, decay factor % and reward at è is R., Q value is defined as Q(s, a) EI 7 |s, a). which in the other words is the expected value of cumulative sum of decaying rewards. Previously, Q-Learning used table to store every Q-value for each state. This method cannot be used for complex environments which have a great deal of states.
Quantile Regression Deep Q Network (QR-DQN:) To learn a generalized state-value function, Deep Q Network (DQN) successfully applies deep neural network to predict Q-value Mnihh et al. (203)). With some tricks for stability, DQN succeeds to beat human level performance in over 29 of
49 Atari 2600 games. Mcanwhile, there are attempts to learn a distribution of Q-value: C51 Belle- Q Network (QR-DQN) Dabney et al. (2017). ). C5I
mare et al. (2017). Quantile Regression-Dest
introduces the importance of distributional reinforcement learning (distributional RL) which learns Q-value distribution instead of the expected value. Distributional RL improves training stability and performance. However, the KL divergence used in C51 is not mathematically guaranteed to con- verge. In economics, the mean predicting quantile values is not only important. The work Koenker (2005) proposes a method called quantile regression to predict quantile values. QR-DQN demon- strates solution that guarantees mathematical convergence by quantile regression that minimizes 1- W-Wassrsstein distance without bias.
Adropting the above-mentioned quantile regresstion approach in this work, we propose QRGAN, a GAN-based generative model adopting quantile regresstion, to minimize 1- I-Wasserstein distance between real samples and generated samples using quantile regression. We train the discriminator to predict quantile values of realisticity using quantile regression. Then, we train our generator to minimize the difference of quantile values of real and fake samples to minimize 1- I-Wasserstein distance between the two. We analyze and compare the discriminator output space for each method to find out if discriminator can guide the generator well. Discriminators whose target is specified tend not to make ambiguous outputs. Those discriminators create sharp minima, so generators may not learn from them. For example, when a generator generates fake samples near a real sample A, a discriminator lowers the probability of samples near A (which can be seen as "realisticity"". Then, generator should generate fake samples near another real sample B. However,
Our contributions:
2

---

Under review as a conference paper at ICLR 2021
discriminator detects fake samples between A and B almost completely. Although the generator should move to B. the generator chooses to stay near A because the way from A to B is too risky decrease). In other words, bounded discriminator creates sharp local minima,
(the realisticity which prevents the generator from learning. This also stops the networks from escaping from the mode collapse. Not to let the discriminator get bounded, we desire minimize 1-Wasserstein distance. Wasserstein GAN AArjovsky et al. (2017) applies Lipschitz condition to discriminator approximate I-Wasserstein distance causing a slowness in training. Our approach of quantile regression can obtain relatively faster training speed in comparision to existing variants of GANs.
The paper is organized as follows. In Section [ we brief current state of the and motivation of this work. In Section we comprehensively present the development of QRGAN. And experimental demonstrations are shown in Section 3 inncluding experiments on the mixture of gussian dataset and image generation experiments. Lastly, the paper is concluded in Section
2 QUANTILE REGRESSION GAN
GANs train generative model by minimizing Kullback-Libeler (KL) divergence between real sam- ples and generated samples. However, KL. divergence is not mathematically guaranteed to converge. Thus, the discriminator often creates sharp local minima and even cause mode collapse issue. Al- though WGANs instead minimize --Wasserttein distance to fix this problem, the training is slow because Lipschitz condition is added to the discriminator (critic) network for Wasserstein distance approximation. We propose QRGAN, a GAN-based generative method, to minimize 1- I-Wasserstein distance between real samples and generated samples using quantile regression. We train dis- criminator to predict quantile values of realisticity using quantile regression. Then, we train our generator to minimize the difference of quantile values of real and fake samples to minimize 1- Wasserstein distance between the two. First, we explain the relation of quantile regression and I-Wasserstein distance in Section 2.I In Section [.2] we explain how to learn quantile values by applying quantile regression to the discriminator, and how the generator minimizes I-Wasserstein distance. Finally, we compare QRGAN with other state of the GAN methods in Section 2.3
2.1 QUANTILE REGRESSION FOR MINIMIZATION OF WASSERSTEIN DISTANCE
For given quantile fraction 7 inverse CDF function F, distributions P. and P. g. p W Wasserstein distance is defined by,
(P,,P.P)) = (fr Fr. (T) (F, dr 1
Then I-Wasserstein distance is,
W (P,, Pg) (") (T) - (F dr
(2)
Here, minimizing 1- I-Wasserstein distance is same to minimizing distance between quantile values. This method is used to indirectly minimize 1- J-Wasserstein distance in QR-DQN. Quantile regression done by training the network using quantile loss. is crafted function to have minimum value the given quantile fraction т. Given quantile fraction E [0, 1] and error u, quantile loss is formulated as follows,
[u.(T - 1), (U.T,
u 1
Pr
(3)
u 0
2.2 QRGAN
We define the number of quantile values N, then quantile fractions are To = 0, NII: TN
where, 1, ..... N. W Wasserstein distance can be represented better by using middle points Dabney et al. 2017) f = I-1+T QR-DQN Dabney et al. 2017). the authors apply Huber loss to quantile loss smoothen it. However, the solution of quantile Huber loss is different from the true quantile value. For accurate quantile regression, we trade off the converge-ability by not apply Huber loss unlike QR-DQN. For given target y and prediction error is defined by, y-n We
2
3

---

Under review as a conference paper at ICLR 2021
apply quantile regression to discriminator to learn a distribution of realisticity. We modify DCGAN Radford et al. (2016) architecture to output vector of N dimension instead of scalar for prediction of quantile values. The target of real samples is and the of fake samples is b. The output of discriminator is the realisticity distribution of the input batch. We simply define by the mean along the batch dimension of each result did in Wasserstein GANs. For given batch size M and discriminator output o, Dr(batch) = M Σ Oi,r- Objective functions of discriminator can be formulated as the following:
Σ ( (+((D((treat) a) + p -b) (4)
min VORGAN (D) D
N
PE(f1....T))
Intuitively, "realisticity" should not be bounded to the specific number. In that sense, we find that works best to use a = +00, b = -oo with regularization (explained below). The quantile values will have gradients of for positive changes and gradients of (7 - 1) for negative changes. Still, the quantile values can be viewed as pessinistic/optinistee neurons that are mentioned in Dabney et al.) (2020). The discriminator output will grow forever and thus the discriminator will not converge when we use targets = +oo and b = -o without regularization. We add L1 square penalty term of the discriminator outputs for convergence.
min VORGAN (D) = N
Σ +E(fI
p+ (+((e(rreat) - a) + pt ( -6)
(5)
M
M
Σ
Σ
loi +
loi,
N
2M
Where, À and are hyperparameters. When added 1.2 penally term, the discriminator outputs
eventually saturate (gradients 0), and the generator training gets slowed. For generator minimize W ateerstein distance,
J. (F7 F (w)
W (U,
we compute the quantile values D+ (xveat) and D= (a and minimize the differences using mean absolute eитоr loss.
min VORGAN
Σ
DD((reett) - D3 (T
(7)
N
=E(FI....NN)
We replace D+(2real quantile values of fake batch are often greater than that of real batch.
prevent it updating to decrease the discriminator output when some
min VORGAN (G) G
Σ
100 D
(8)
N
FE(AI..,,T))
Minimizing this objective function is same maximize the following function, thus:
max VORGAN(G) =
Σ
D (T
99)
N
FC(FI....Tv)
The proposed QRGAN is developed as shown in Algorithm
2.3 COMPARISON TO OTHER GANS
Derivative of discriminator loss when a 00, b= to the derivative of the half of the critic loss of Wasserstein GANs. The two even regularizes the discriminator//critic outputs to prevent endless growth. In addition, the derivative generator loss is same as Wasserstein GANs. This fact may mean QRGANs an upgraded version of Wasserstein GANs that do not need to meet the Lipschitz condition for approximating I- I-Wasserstein distance thanks to quantile regression.
= 1, 0.5 which is proportional

---

Under review as a conference paper at ICLR 2021
Algorithm 1: QRGAN
Input: 7, M, N, A, h; Ty, 0p. 0g
> (learning rate), M (batch size), N (number of quantile values), (regularization coefficient), & (norm target)
TN (quantile values), Op (initial discriminator weights), lg (initial generator weights)
while (Parameters NOT converged) do
2:
Sample {ri P, batch of real data
3: Sample {2e i ~ p(2) batch the prior distribution
9D V0D
(+((((s:00):0D) ( +
M.N Ee EtE(t
N E+E(F
Ee loi + Ei 0 (0i,
2,30
E E+e(t....)) D=(G(z):06)
5:
96 - Vea
M.N
6:
0G DD - 1) Al.Admm(0), p,
0a 0g Afam(fg: 9G)
LSGAN
QRGAN (Ours)
d
mean
do.50
da.ro
menn
mean
0:28 01.0
00.5
00.2 06.00
00.75
1
Loss
2
Genan
Loss
N
Σ de
Figure 1: Comparison of LSGAN and QRGAN generator loss. LSGAN generator minimizes the difference of the means of distributions. QRGAN generator minimizes the difference of the quantile values of distributions.
3 EXPERIMENTS AND RESULTS
In order to investigate the QRGAN's robustness against mode collapse problem, we performed ex- periments using several GAN methods (NSGAN, LSGAN, QRGAN) on mixture of gaussian dataset (Ring-8, Grid-25) in the works Section 3.1 We also analyzed the discriminator output space to see the discriminator guides the generator as expected. Also, we pinpoint the reasons why we should not use L2 penalty term in QRGANs. Furtheremore, to examize the QRGAN's generation capabil- ity, we performed different image generation experiments (CIFAR-10 10, LSUN-Bedrood evaluated the GAN methods by Frechét Inception Distance Heusel et al. (2018). We also conducted analogous analyses of QRGAN with regard WGAN-GP using ww-.aach-nommalizatiiii discrimina- tor. For all image generation experiments, only take training set for training and evaluation. All model weights of convolutional and transposed convolutional layers are initialized from (0, 0.02) and biases of them
Cats) We
initialized to zeros. We picked Adam Kingma & Ba (201)) optimizer with 3 0.5 and B2 = 0.99 for optimization purposes. For WGAN-GP, we use (gradient penalty coefficient) of 1 for CIFAR-10, and 10 for LSUN-Bedroom.

---

Under review as a conference paper at ICLR 2021
3.1 MIXTURE OF GAUSSIAN DATASET
To test the robustness against to mode collapse, we did the same experiments which are done in Unrolled GAN (Ring-8) and VEEGAN (Grid-25) Metz al. (2017) Srivastava al.] (2017). We define two training dataset generator, Ring-8 and Grid255. Ring-8 dataset contains samples from mixture 8 gaussians arranged in ring shape. The ring has radius of 2 and each gaussian distribution has standard deviation of 0.02. Meanwhile, Grid-25 dataset contains samples from mixture of 25 gaussians arranged in grid (5x55 shape. The centers of gaussian distributions range from [-4, -4 to [4, 4]. Each distribution has standard deviation of 0.05. The MLP-basce model architecture used for MoG experiments is shown in Fig. Z Hyperparameters used for Ring-8 8 and Grid-25 experiments presented in Table 2 The experiment results are shown in Fig. 2
True Data
NSGAN
LSGAN
ORGAN (Ours)
Figure 2: (Top) Ring-8 experiment result. (Bottom) Grid-25 experiment result.
(a) LSGAN
(b) NSGAN
(c) WGAN-GP
(d) QRGAN
(e) QRGAN 000000
(regulanized by ponaity term)
Figure Contour plots of discriminator output space
Mode collapse occurred on NSGAN and LSGAN both Ring-8 and Grid-25 experiments. How- ever, mode collapse is not observed on our proposed method (QRGAN) experiments. Mode collapse occurs by various reasons. One of the most decisive reason is local minima in the space created by discriminator. To analyze the discriminator output space whelms generator, we trained discriminator by various GAN methods with dummy generator which outputs samples from uniform distribution. The color is the discriminator output value (higher is more realistic). Blue dots are dummy generator output, sampled from uniform distribution. We pick some blue dots and draw red arrows (force) by normalized computed gradients by generator loss. The contour plots of discriminator ourput spaces are shown in Fig. 3 As we can see from the contours, very steep slope appears only near real samples and very gentle slope appear for samples outside. Thus. gradient norm is very small (the arrow length is short). Most gradient directions are not desirable for fake samples. Looking the colors, discriminator guides proper Wasserstein distance minimization. Slopes are not too gentle fake samples. Red arrows consistently head to real samples. Like LSGAN case, gradients are too gentle for fake samples and gradient directions are decided by noise, thus not desirable. Generator is likely to suffer from local minima. In com- mon, discriminator is modeled to predict probability. For the optimal discriminator, there are no output difference between real samples. This behavior is same to fake samples, too. In real, noise makes difference, thus gradient directions are noisy. Then, generator not trained to make realistic samples. This works very similarly for LSGANS and bounded (which use finite targets) QRGANS,
situation which discriminator over-

---

Under review as a conference paper at ICLR 2021
too. Instead, model a discriminator to predict distance. If discriminator predicts distance, it should be less affected by noise.
3.2 IMAGE GENERATION EXPERIMENTS
3.2.1 EVALUATION METRICS
We use Fréchet Inception Distance Heusel et al. (2018) (FID) for evaluation of image generation experiments. FID is an evaluation metric for image generation models. Although Inception Score (IS) is used previously, there are cases where IS evaluates worse than FID because does not reflect training data. FID calculates 2048-dimension feature vector from Inception-v3 model. It calculates mean and covariance matrix from the feature vector, and then compute Fréchet distance from the statistics. Lower FID value means that the two statistics are closer. Given mean μı and μ2g, and covariance matrix D1 and Σ2, the Frechet inception distance could be represented as following, FID [1 H2 |2ll2 + Tr()1 + Σ> 2g), We used the dataset and approach presented in Heusel et al. (2018) for FID value calculation. We generate 50000 images and then calculate FID value for each 4k iterations and compare the minimum Fréchet Inception Distance (FID) value.
3.2.2 ARCHITECTURE FOR EXPERIMENTS
Deep Convolutional Generative Adversarial Networks Heusel et al. (2018) (DCGANs) introduce rules for stable adversarial training of convolutional networks. Addi- tionally, the authors have shown vector arithmetic operations (addition, subtraction) on the latent code representations and it means that they successfully vectorized image. For image generation experiments, we use OCGAN-based architectures as shown in Fig. g
Deep Convolutional GANs:
Checkerboard artifacts and Resize-convolution: In DCGAN, generator outputs with checker- board artifacts when the kernel size is not divisible by stride Odena et al. (2016). These artifacts generated because the outputs can be unevenly overlap. This phenomenon also occurs in back- ward pass of convolution operations. To fix the checkerboard artifacts, we use convolution-averae pooling and resize-convolution layers. For downsample pass in the discriminator, we use kernel size of 3, stride of 1, padding of for convolutional layers and following average pool 2d layer. For up- sample pass in the generator, we upsample the input by nearest neighbor method and then pass it to following convolutional layer. The checkerboard artifacts is presented in Fig. 9 The parameters of different architecture variants are shown in Table 3 Hyperparameters are presented in Table The results of computed minimum FID are as in Table I QRGAN has shown better FID values than NSGAN and LSGAN and competitive score to WGAN-GP. By applying the regularization method learned from the discriminator output space analysis, gradients keep good norm and direction. Al- though WGAN-GP shows great performance in several experiments, the computation is very costly and take much more iterations. The analysis results for comparison of QRGAN against NSGAN, LSGAN and WGAN-GP are shown in Fig. Fig. and Fig. 6 for the experiments on CIFAR-10, LSUN-Bedroom and Cats, respectively.
TID (Lower is better))
FID (Lower better)
50.0
ORCAR (nsi
CRGAN Ovin-331 933
47.5
4/.3
WGG/ GP (min- 20.30
45.0
42.5
40.0
37.5
37.5
35.0
35.0-
12.5
20000
40000
50000
2000E Enaniing stpp
100000 120000 140000
200000 <OCO0S
GCDO00 DOOCDO
1000000 1200200
trinking dtep
(a) QRGAN compared to NSGAN and LSGAN
(b) QRGAN compared to WGAN-GP
Figure 4: CIFAR-10 generation experiment results
7

---

Under review as a conference paper at ICLR 2021
FID (Lower is better)
50
50
GIGAN 0mii-22.44
WGANKO (mte-
43-
40-
42-
35-
33
30
32-
25-
DB oninu 451
50000
200000
150000
200000
250000
50000
100000
150000
200000
250000
500000
Tramingg step
(a) QRGAN compared to NSGAN and LSGAN
(b) QRGAN compared to WGAN GP
Figure
generation experiment results
better)
better)
GRG (mm-
200
33.0-33.001
25-
130
100
50-
20080
50000
1000O Branting stop
80000
180000
1000C
20000
300CO
40000
50000
10000
20000
traniing
(a) QRGAN compared to NSGAN and LSGAN
(b) QRGAN compared to WGAN
Figure 6: Cats generation experiment results
Table 1:
The minimum FID summary
Minimum FID
CIFAR-10 10
LSUN-Bedroom
Cats
NSGAN 34.16
28.29
11.37
LSGAN
34.53
30.39
13.68
WGAN-GP
30.36
19.76
229.8
QRGAN
30.93
22.45
10.46
4 REMARKS
We presented the proposed QRGAN, new method to train GANs that minimize 1-Wasserstein distance between real data distribution and fake data distribution using quantile regression. By modeling a discriminator to predict quantiles, more details could be learnt. We explained how mode collapse is created, and what is the desired characteristics of discriminator output space. We showed that I.SGAN discriminator creates sharp local minima, thus it can get stuck in mode collapse. On the contrary, WGAN-GP did not create any sharp local minima. We found an appropriate regularization method from this analysis. The result has shown improved generation quality than previous state of the art GAN methods.
8

---

Under review as a conference paper at ICLR 2021
ACKNOWLEDGMENTS
This research was partially supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(No. 2020R A6A1A0304881
This research was partially supported by the MSTT(Minnstry Science, ICT), Korea, under the ITRC(Information Technology Research Center) support Program(IITP- 1000-2016-0-04465 supervised by the ITTP(Institute for Information communications Technol- ogy Planning & Evaluation)
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. volume 70 of Proceedings of Machine Learning Research, pp. 214-223, International Convention Centre, Sydney, Australia, 06-11 Aug 2017. PMLR. URL http: ptttp://proceedings .mlr.
Marc G Bellemare, Will Dabney, and Rémi Munos. A Distributional Perspective on Reinforcement Learning. volume 70 of Proceedings of Machine Learning Research, pp. 449 458, International Convention Centre, Sydney, Australia, 2017. PMLR. URL http::/proceeedngg mlr.
press/v70/be emarel 7a html
Will Dabney, Mark Rowland, Marc G. Bellemare, and Rémi Munos. Distributional reinforcement learning with quantile regression, 2017.
Will Dabney, Zeb Kurth-Nelson, Naoshige Uchida, Clara Kwon Starkweather, Demis Hassabis, Rémi Munos, and Matthew Botvinick. A distributional code for value in dopamine- re- inforcement learning. Nature, 577(7792)x61--775 1038/ 01133//41586-019110
2020. ISSN 1476-4687.
doi: 10.1038/
15586-019-1924-6 URL
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. In Z. Ghahramani, M Welling, C Cortes, N D Lawrence, and K Q Weinberger (eds.), Advances in Neural Information Processing Systems 27, pp. 2672-2680. Curran Associates, Inc., 2014. URL http:ttppp/pppae cecppersniiss.. wwi.scc...... cc/ pepper/543--eenerttte-e-ee
pdf
Ishaan Gulrajani, Kundan Kumar, Faruk Ahmed, Adrien Ali Taiga, Francesco Visin, David Vazquez, and Aaron Courville. PixeIVAE: A Latent Variable Model for Natural Images. nov 2016. URL
http://arxiv. wwt.oorg//aa..... abs 1 611 .05013
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Im- proved training of wasserstein gans, 2017.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to local nash equilibrium, 2018.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
Diederik P Kingma and Max Welling, Auto-encoding variational bayes. In 2nd International Con- ference Learning Representations, ICLR 2014 Conference Track Proceedings. International Conference on Learning Representations, ICLR, dec 2014. URL http:://aaattitt wwwiiii.rrgg/bo. arg/abs/
1312.6 6114
Roger Koenker. Quantile Regression. Cambridge University Press, 2005. econpapers. reper.eee...... wwwpo//eeee....coooooooeeo.oooee http///Pccccooroooooooe777772333 978002184072777
URL https://
E. Levina and P. Bickel. The earth mover's distance is the mallows distance: insights from statistics. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, volume 2. pp. 251-256 vol.2, 2001.
Alireza Makhzani. Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. Adversarial autoencoders, 2016.

---

Under review as a conference paper at ICLR 2021
X. Mao, Q. Li, H. Xie, R. K. Lau, Z. Wang, and S. P. Smolley. On the effectiveness of least squares generative adversarial networks. IEEE Transactions on Pattern Analysis and Machine In- telligence, 41(12):2947-296
Dec 2019. ISSN 1939-3539. doi: 0..1109/TPAM.2211255770000
Xudong Mao, Qing Li, Haoran Xie, Raymond Y. K. Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks, 2017.
Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks, 2017.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Mare G Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wier- stra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning- Nature, 51 3187554):559-533. feb 2015. ISSN 1476-4687 (Electronic). doi: 10. 10.1/38/a 4226..
Youssef Mrouch and Tom Sercu. Fisher GAN. In I Guyon, U V Luxburg, S Bengio, H Wallach, Fergus, Vishwanathan, and R Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 2513 2523. Curran Associates, Inc., 2017. htLhhtt:/// //apers//ppp....pp nips.
cc/paper/ 68 fisher-iseee gan.gann.......
Augustus Odena, Vincent Dumoulin, and Chris Olah. Deconvolution and checkerboard arti- facts. Distill, 2016. doi: 10.s.155/
URL http://distilll http:///diit111...p//2200... pub/2016.
deconv-checkerboard
Oleksii Prykhodko, Simon Viet Johansson, Panagiotss-Chrisoo Kotsias, Josep Arús-Pous, Es- ben Jannik Bjerrum, Ola Engkvist, and Hongming Chen. A de novo molecular generation method using latent vector based generative adversarial network. Journal of Cheminformat- doi: 10.118661322-009-00997
ics, 11(1):74, 2019. ISSN 1758-2946. //doi.org/10. www.oorg/......... 1186/ 1ttp://001116611331111 1113221-199111 0397-9
URL https:
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks, 2016.
Akash Srivastava, Lazar Valkov, Chris Russell, Michael U. Gutmann, and Charles Sutton. Veegan: Reducing mode collapse in gans using implicit variational learning, 2017.
Aaron van den Oord, Nal Kalchbrenner, Lasse Espeholt, koray Kavukcuoglu, Oriol Vinyals, and Alex Graves. Conditional Image Generation with PixeICNN Decoders. In D D Lee, M Sugiyama, U V Luxburg, Guyon, and R Garnett (eds.), Advances in Neural Information Processing Sys- tems 29, pp. 4790-4798. Curran Associates, Inc., 2016. URL http: /tttt::/ppapeee http://ppperrs.ii.. cc. paper/ 6527 cooodiitional image generation with withh-pieeehnee wwppppieepieeee decoders.
pdf
Maciej Wiatrak, Stefano V. Albrecht, and Andrew Nystrom. Stabilizing generative adversarial net- works: A survey, 2020.
10

---

Under review as a conference paper at ICLR 2021
APPENDIX
A FIGURES AND TABLES RELATED TO EXPERIMENTS
Due to the space limitation, we present related figures and tables of our experiments in appendices.
A.1
THE MLP-BASED MODEL ARCHITECTURE USED FOR MoG EXPERIMENTS
DISCRIMINATOR
GENERATOR
Input
Input 128
2
Linear(in_dim-22 jut_dimm122) LeakyReeU(0.22)
Limear(in_dim= 128, ReLU
128
(inear(in_dim=110 out dim=128) (00)))
Linear(in_dim=128 1288 out out_dim=128) ReLU
128
128
Linear(in_dim=128, out_dim=64 or 1)
Linear(in_dim- 1iciiin__im112211 128, out out_dim=2)
1
Figure 7: The MLP-based model architecture used for MoG experiments

---

Under review as a conference paper at ICLR 2021
A.2
CIFAR-10 ARCHITECTURE
Generator
Discrtminator
Dape
padding:))
Loakydel L8G33)
Constr.irwest oc.din SereLLttw)) striewd, pad8ing-11
Anghol
Cenejn, dimiii suadd stident, pablinge))
Baichhor Looky (ib2)
Upertse padding-) Babcikern
Upaans
Connjn dinndddnnta 128, out diniiss, kamed alam-,, stidem1, puodingg)) Baldheorm
Babrkkrn
Aughtcal
Coenda .i Fibd rihhl kondl suzend stiddet, paolinge))
Dekthorn
Figure 8: Architecture for CIFAR-10
12

---

Under review as a conference paper at ICLR 2021
A.3 CHECKERBOARD ARTIFACTS AND CENNLTICONVOLUTTON
Figure 9: Checkerboard artifacts. (Top) Checkerboard artifacts has caused because kernel size is not divisible by stride. (Bottom) No checkerboard artifacts appeared by using resize-convolution.
13

---

Under review as a conference paper at ICLR 2021
A.4
TABLES OF PARAMETERS
Table 2: Hyperparameters used for Ring-8 and Grid-25 experiments
Hyperparameter Name
Value
Batch size
500
Dimension for latent vector
128
Discriminator Model
MLP without BN
Generator Model
MLP without BN
Main dimension for discriminator
128
Main dimension for generator
128
Learning rate for discriminator
le-3
Learning rate for generator
le-3
Optimizer
Adam
B1 (Adam)
0.5
B2 (Adam)
0.99
Number of quantiles
(L1-square penalty)
0.1
(Norm target)
Table 3: Architecture variants summary
Task
Number of hidden layers
Image size
CIFAR-I0
32x32
MUUN-Berroo
64x64
Cats
128x
Table 4: Hyperparameters summary
Hyperparameter Name
CIFAR-10 10
LSUN-Bedrood
Cats
Batch size
64
64
64
Image size
32x32
64x64
128x
128
Dimension for latent vector
100
100
100
Main dimension of discriminator
64
64
64
Main dimension of generator
64
64
64
Discriminator Model
DCGAN
DCGAN
DCGAN
Generator Model
DCGAN
DCGAN
DCGAN
Learning rate for discriminator
2e-4
2e-4
5e-5
Learning rate for generator
2e-4
2e-4
5e-5
Optimizer
Adam
Adam
Adam
B
0.5
0.5
0.5
B2
0.99
0.99
0.99
Number of quantiles
64
64
64
(L1-squee (LI-square penalty)
0.1
1.0
0.1
K (Norm target)
0.0
1.0
0.0
14

---

Under review as a conference paper at ICLR 2021
B BACKGROUND AND PRELIMINARY
We present the background and preliminary for this study as in the following appendices for clearer understanding of the proposed QRGAN.
B. DISCRIMINATION AND GENERATION
Generally, generation tasks are known to be complex than classificatonnreegresiion tasks because the model learns joint probability distribution: Given an observable variable X and a target variable Y:
p(X,Y) p(X(x))p(X
(10)
while discriminative model learns only p( (XIV).
Discriminant Model
Generative Model
Figure 10: Discriminant model generative model
There are many attempts to train neural-network-basee generative models like Variational Autoen- coders (VAE), Generative Adversarial Networks (GANs), and Adversarial Autoencoders. Kingma
& Welling 2014 Kingma & Welling 2014) Makhzani et al. (2016)
B.2 THE KULLBACK-LEIRLER (KL) DIVERGENCE
The Kullhack-Leibler (KI. divergence is defined as follows,
(kI( = / p(z))og
p(x)) ((x))
dr
(11)
where both P and Q are assumed be continuous, an integral probability metric (IPM) measure diversity between P and Q. The KI. divergence can be infinite when there are points where P(x) and (xx) > 0. that case, gradients calculated by KI. divergence tends to zero, then training
gets slow.
For GAN discriminator loss
vo = 0 where (zx) = 0 or (x)
(12)
For NSGANs, Non-saturating GANs, whose generator loss defined as:
min VNSCAN(G) = -logo (((esake))
(13)
Then, log0- -c, where (I:) 00
15

---

Under review as a conference paper at ICLR 2021
B.3 WASSERSTHIN DISTANCE
To fix the non-convergence problem of the KL divergence, there are attempts to minimize Wasser- stein distance in recent work. The p Wasserslein metric W,, p. for p [1, oo], also known as
the Earth Mover distance when 1 Levina & Bickel] (2001), is an integral probability metric (IPM) between distributions. The p W asserstein distance L metric on inverse cumulative distribution functions between distributions U and Y.
W,(U (((15"()) (F
(14)
where for random variable Y, the inverse CDF F7* of Y is defined by,
(W) = R: S (1)
(15)
where, (2) Pr(Y < y) is the CDF of Dabney et al. (017). Wasserstein distance is difficult compute because of its complexity. In Wasserstein GANs, the authors apply Lipschitz condition which is defined by,
ds (1(1))) S (dg x2)
(16)
for given metrics A and da apply approximation method for the W-Wasersttein distance. This constraint limits the neural network capacity because the model architecture and weight value should be restricted.
Compare in this direction
Not in this direction
Figure 11: 1- Wasserrstni distance is also called Earth Mover's distance
B.4 VARIATIONAL AUTOENCODER
B.4.1 AUTOENCODER
Autoencoder is neural network architecture used to learn latent representations in unsupervised manner. consists of encoder and decoder networks, and usually encoder's output dimension is smaller than input dimension, called bottleneck.
Objective function of autoencoders can be formulated as follows:
min JAE(O) = [ ( D(E(X;0):0)
(17)
where encoder network outputs code and D is decoder network which predicts input to the en- coder. To generate samples from the decoder, should generate appropriate input for the decoder. However, it is very difficult to generate the input due to the intractability the latent distribution of plain autoencoders.
16

---

Under review as a conference paper at ICLR 2021
DECODER Output Leyer
ENCODER
Inpul Layer
Code
Real
X
X
Sample
Hepresentation
Figure 12: Autoencoder architecture
B.4.2 VARIATIONAL AUTOENCODER
In VAEs, they make the encoder to output parameters of a tractable distribution such normal dis- tribution. By doing so, we can calculate Kullback-Leible (KI.) Divergence of the code distribution and prior distribution. For given encoder E, decoder D, prior distribution p, the objective is:
1 1 (T-DDZZ)) + (L())))) 2
min JvA
(18)
where, 14, 0( and Z ~ N(u,ol).
ENCODER Inpul Layer
Cutput Layer
Code Desibbutir
Sample
data
Lstent Representation
Figure 13: Variational Autoencoder architecture
In addition, reparameterization trick is applied for backpropagation. 2 μ + a.e, where e ~ p. In VAEs, the KL divergence regularization assumes every code distribution to be even when different input is given. This leads to input aliasing to the decoder and worsen blurry outputs.
B.5 GENERATIVE ADVERSARIAL NETWORKS
B.5.1 GAN
Recently, Generative Adversarial Networks (GANs) have been introduced. Discriminator leams whether the input is real or fake, and generator learns to fool discriminator to generate realistic samples help of diseriminator. GANs successfully solved problems of VAEs. Unlike VAEs, GANs can hypothesize the entire code distribution. Adversarial autoencoders (AAEs) take this idea to make generative autoencoder which regularizes the entire code distribution. GANs can be formulated as the following:
min max V(D.G) Bz~pa(()) (log (2) D(G())))
(19)
Because gradients could very small at the beginning, alternative loss function for generator, namely NSGAN is proposed.
max V(G) E[logD(G(2))
(20)
17

---

Under review as a conference paper at ICLR 2021
Noise N(0,
~~ (z)): Input noise distribution
Generator (MLP)
G(=; 0a) differentiable with Ag
Fake sample
Real sample
Sample from Generator,
Sample training data, Zreat Pr
ffol ~ Pg
Discriminator (MLP)
D(2; 0p)
Realisticity score (scalar)
Figure 14: The Generative Adversarial Networks framework architecture
B.5.2 DIFFICULTY OF TRAINING GANS
3
76
b
1019
4
(19161)
2
3 396767
LV 1
0557
Figure 15: (Upper, Unrolled GAN) the GAN training is successful without mode collapse. (Lower, Standard GAN) the GAN training is failed, and this type of GAN failure called mode collapse that generator outputs very similar images Metz et al. (2017)
GANs are difficult to train because the networks compete each other. Because discriminator is used for generator loss calculation, the loss can oscillate and be unstable. addition, cross entropy loss used in the original GANs often saturates. When failed to train GANs, mode collapse problem often occurs.
B.6 REINFORCEMENT LEARNING
Reinforcement Learning is to find a policy that maximizes given rewards by environment. Given state and action, environment outputs next state and reward. Agent should try various action
18

---

Under review as a conference paper at ICLR 2021
state (ebservation)
Agent
reward
action
reward next state
Environment
Figure 16: Reinforcement learning
sequence to find best policy. Q Learning is learning Q-values iteratively. Q - value is defined
as,
s 3 Ra s.
Q a)
Σ
(21)
and this can be represented by Bellman equation,
= R =
(22)
where is current state, is the action, is reward, a' is the next action, and y is hyperparameter called discount factor to discount future rewards exponentially.
B.6.1 DEEP Q NETWORK (DQN)
Deep Q Network Mnih et al. (2015) successfully applics deep learning to solve the Bellman equa- tion. The training unstable because the Bellman equation is recursive. Target network, cloned network only for calculation of target Q value, is used to stabilize the training. Moreover, replay memory is applied randomly sample the training data.
B.6.2
C51
attempt to learn value distribution instead of expected value of it. In C51, the authors define three hyperparameters: the number of supports, low and high for Q-value. By making bins by the three hyperparameters, the regression task has been turned a classification task. Unlike DQN, target is not scalar value but a distribution. To optimize predicted distribution to target distribution, they projected supports properly to fit bins then minimized the KIL divergence be- tween the two. Learning value distribution stabilizes training and improved prediction performance. Although they prove that -assserstein distance should be minimized because KL divergence not mathematically guaranteed to converge, they could not find way to minimize it.
Moreover, there
vs. DQN
# Games Superior
C51 HUMAN
DQN HUMAN
50
100
150
200
Training Frames (millions)
Figure 17: C51's performance compared to human and DQN Bellemare al. (0117)
19

---

Under review as a conference paper at ICLR 2021
B.6.3 QUANTILE REGRESSION DEEP Q NETWORK (QR-DQN)
QR-DQN Dabney al. 2017) is proposed fix the non-convergence problem of C51. The authors hypothesize the probability distribution function (PDF) consists of N dirac delta functions. We can optimize the PDF by changing the supports of the dirac delta functions. Likewise, the CDF can be also optimized by quantiles.
73
72
22
11
91
TV
F (f:)
2/
Figure 18: The CDF which is represented by Dirac delta functions. Area filled by red represents the I-Wasserstein distance. The 1- W-Wassesttein distance is proportional to the sum difference between quantile values at each quantile fractions. Therefore, quantile regression is the equivalent to W-Wasserstein distance minimization.
20


Reviews:
[{"review_id": "uCE0xjCy9eR", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "– Summary – \n\nThe paper proposes a new GAN method that applies the quantile regression of reinforcement learning into GAN and aims to show this helps to estimate the 1-Wasserstein distance better without gradient regularization. The idea of quantile regression presented in the paper is a way to match two distributions like WGAN-GP yet at a more grained level and need no regularization like WGAN-GP. The experiments are conducted on 2D-toy examples (Ring-8, Grid-25) as qualitative results and three other image datasets (CIFAR-10, LSUN-Bedroom, Cats) with FID scores. The proposed method is compared with some GANs baselines: SNGAN, LSGAN, and WGAN-GP. \n\n\n– Strength –\n\nS1 - The paper proposes a new idea to apply quantile regression into GANs.\n\n\n– Weakness –\n\nW1 - The paper is not well-written, and the paper representation is not good.\n\nW2 - The performance of the  proposed method does not look outperforming the WGAN-GP even though the paper strongly claims the robustness of this method. As shown in Fig. 4, 5, the proposed method converges faster, but is not necessarily better than WGAN-GP at the end. It looks WGAN-GP converges much more stable than the proposed method.\n\nW3 - It does not make sense why WGAN-GP is so bad on Cats dataset as shown in Fig. 6. It could be just the problem of parameters-tuning?\n\nW4 - It's unclear why Fig. 2 misses the WGAN-GP?\n\nW5 – The paper does not convince me why the proposed method is better than WGAN-GP in either theoretical and empirical results. In addition, the paper does not provide sufficient theoretical content to show the 1-Wasserstein distance is the same as minimizing quantile values as claimed.\n\nW6 - Many mathematical notions are not explained, e.g., What is $\\rho_{\\hat{\\tau}}$ in Eq. 4? How  do the authors implement with $a = \\infty$ and $b = -\\infty$?. How is $o_{i, \\tau}$ computed?\n\nW7 - FID scores alone may be biased, the combination with IS is required in the experiments.\n\nW8 – The experimental results are not sufficient, e.g., the results are with only standard DCGAN architecture, and the paper would need more ablation studies on some selected hyper-parameters, e.g., $a, b, N, k$ ... in the method.\n\nOverall, I think the paper is far to meet the conference's standard, e.g., at paper presentation, strong empirical or theoretical evidence to justify the claims. It also would need substantial revision to improve in writing. I tend to reject the paper.", "extra_scores": {}, "extra_text": {}}, {"review_id": "QWcD-Tnwj3D", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary. The paper proposes to use quantile regression as an alternative GAN loss. The idea is to evaluate the quantiles of discriminator outputs instead of just using a single scalar discriminator value: while all quantile discriminator outputs for real/fake data is pushed to +infty/-infty, the generator's cost function tries to push the quantiles of fake data to be as realistic as possible (+infty). As the regression loss leads to runaway values, a L2 regularizer is put on the discrimantor output quantiles. Experimental results show very good mode coverage on Gaussian Mixtures. Further experiments demonstrate it's in the ballpark of WGAN-GP on CIFAR-10, best on CATs, and second on LSUN-Bedroom.\n\nReasons for score: While the paper presents a reasonable and (to me) novel idea in GANs, the experiments fall short in comparing to the state of the art, and also the ingredients in the method are not sufficiently dissected to attain a sufficient understanding. Hence, I suggest to reject the paper in its current form.\n\nPro.\n- the paper proposes a simple and novel target function for GAN training, that makes intuitively sense\n- the mode coverage on the toy Mixture of Gaussian experiments look very solid\n\n\nCon.\n- experiments:\n   - on CIFAR-10 etc. lack comparison to state-of-the art (sota) methods, which is necesary to put the work in context. E.g. it should be added SN-GAN (Miyato et al.), StyleGAN(2).\n   - similar for the Mixture of Gaussian: comparison to other standard methods in the literature that tackled this problem are missing (e.g. Unrolled GANs). \n   - also, please check again WGAN-GP on cats - why is WGAN-GP seemingly not training at all (and starting at a much higher level from the start in Figure 6)?\n- the regularizer in equation 5 pushes both discriminator for real and fake towards the same values - hence potentially counteracting stability issues in training. This should be investigated independently to understand the effect of this regularizer in isolation (can even be formulated also for standard GAN KL losses, e.g. by pushing the average to 0.5). Otherwise it remains unclear if the benefits of the methods are attributable to the quantile regression or this regularizer\n- the exposition should be improved: \n  - pg 2: too much stuff on RL - this is not needed in the paper and should be shortened to a minimum\n  - the method could be written to be understandable more easily (e.g. give a high-level description of the intuition similar to my Summary above before diving deep into the formulae)\n\n\nRebuttal:\n- please address the points on the Con side.\n\n\nMinor issues:\n- the relation to standard divergence minimization remains unclear; i.e. in particular it remains theortically unclear if this really converges to the target distribution (the empirical results seem encouraging though)\n- many typos and language needs improvement - please check the document carefully again\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "MFj2Hhes3kZ", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose the Quantile Regression GAN (QRGAN) to minimize the 1-Wasserstein distance between the real and generated data distributions. The proposed method avoids the mode collapse problem and obtains an improvement in the FID score compared to some existing GANs.\n\n-  Pros:\n  - Compared with NSGAN and LSGAN, the proposed method avoids mode collapse and achieves better performance than them in the FID score.\n  - In addition, compared to WGAN-GP, it achieves better FID with less training iterations while maintaining comparable performance.\n- Cons:\n  - Overall, there are so many typos and grammatical errors in this paper that they make it difficult to understand the content of the paper. The authors must look for these errors and correct them.\n  - Also, the way of citing figures and tables is inappropriate. For example, Fig. 1 is not cited in the main text, and figures and tables in appendixes such as Fig. 7 and Table 2 are cited without specifying that they are in appendixes. Reviewers don't need to read the appendixes, so the content should be complete in the main text.\n  - The authors state that the relationship between quantile regression and 1-Wasserstein distance is shown in section 2.1, but this is not explicitly shown. In particular, the authors state in section 2.1 that \"Here, minimizing 1-Wasserstein distance is same to minimizing distance between quantile values\", but Eq.1 and Eq.2 are simply p-Wasserstein distance and 1-Wasserstein distance, so it is unclear which equation represents the relationship. Also, the period in Eq. 3 should be a dot (multiplication).\n  - In Eq. (4), you state that a and b are set to +∞ and -∞ respectively, but how were these infinities implemented in practice?\n  - Why is there no WGAN-GP result in Figure 2? My understanding is that QRGAN minimizes 1-Wasserstein like WGAN-GP, so the result is almost the same. And why didn't the authors include unrolled GAN and VEEGAN results for comparison, even though they performed the same experiments as these papers?\n  - If the authors claim that WGAN-GP is computationally expensive, they should show how much less expensive it is in QRGAN. QRGAN also requires the sum of multiple quantile values, so the more of them, the longer it should take to compute them. Also, as far as I read, there is no indication in the paper of how the number of quantile values was set up in the experiment.\n  - Looking at Figure 4 and Figure 5, GRGAN appears to be less stable than WGAN-GP. Why is this?\n  - In section 3.2, the authors should show the image actually generated by GANs.\n\n- Minor comments:\n  - It is difficult to read because the author's citation is not enclosed in parentheses.\n  - Some parts of the random variables are in bold type and others are not. These notations should be consistent.", "extra_scores": {}, "extra_text": {}}, {"review_id": "CnE9EdL8aQT", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1. This paper contains many typos, grammar mistakes, and format problems, which make it very hard to read. For most equations, there is no ending period. The citation format seems wrong.\n\n2. The equations (1) and (2) are for 1-dimensional random variables. They may not be suitable for high dimensional random variables. The $N$ quantile values defined in this paper are for which random variable?\n\n3. For the neural network for $D_\\tau$, does it mean the input is $(\\tau, X)$? So for different $\\tau$, they share the same weights. What is the meaning of this output?\n\n4. The \"$+\\infty$\" and \"$-\\infty$\" notation is confusing. Does this mean we choose a very big value or a very small value in practice? But this is very subjective now. Did you perform some sensitive analysis on the choices of $a$ and $b$?\n\n5. The authors claim that the WGAN training is slow. For WGAN-GP, I don't see why the training is much slower than the training of QRGAN. Did you perform some analysis on the training time?\n\n6. The arguments for QRGAN  to overcome mode collapse is quite vague. Many GANs with the encoder structure can solve the mode collapse well. It may benefit to compare  QRGAN with these methods.\n\n7. What is the implication of Table 1? Does that mean WGAN-GP is better than QRGAN? The generative images are not demonstrated. Other dataset such as CelebA can be applied to check the performance. Image interpolation can also be demonstrated for mode collapse situation.\n\n8. The Appendix B is completely not necessary. It contains only well known results.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "6DDc5KD-s0X", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "# General statements\nthe core contribution of this paper is to train GAN by designing the discriminator so that it outputs a whole distribution instead of a point estimate for \"realism\". This distribution is instantiated through its quantiles, and the whole approach is thus framed in a quantile-regression framework. The nice feature is that using losses over quantiles means using a wasserstein distance, which has strong properties in a GAN setting.\n\nThe idea is interesting and is definitely worth investigating. I am not 100% sure it was not presented previously. But I trust the authors on this.\n* All in all, the paper is average in terms of english usage: totally acceptable in the beginning, there is a strong degradation for the experimental section, that has been either written in haste, or by another author than the rest.\n* The actual performance is rather disappointing, since the authors do not clearly manage to demonstrate any superiority of their proposed approach vs author (classical) methods, except on toy data. I don't see this as a real issue though, because the fact that the paper is inspiring is what I believe is most important.\n* The quantile regression loss is not presented in a way that allows people not knowing it already to understand the paper. This must be changed.\n* you didn't study the impact of the number of quantiles, although it looks like something you definitely should have done, since it's the core contribution of the paper.\n\n\n# Detailed comments\nBelow are remarks and typos found along the way:\n\n## Abstract\n* \"And we found that he discriminator should not be bounded to specific numbers.\" is unclear here.\n\n## Introduction\n * \"Text or structured data [...] of the world \" awkward\n * \"p(z|x), not p(z)\" awkward\n * \"Mode collapse is caused by unstable training and improper loss function\" : reference ?\n * \"propose to use mean square error (MSE) which does not saturate\": at this stage, you didn't introduce this concept of \"saturation\". And you don't tell where the MSE is applied\n * \"results better models\" : results in better models ?\n * \"Reinforcement Learning (RL):is to learn\": awkward. And we don't understand why you're mentioning RL at this stage. Above, you only introduced VAE and GAN for your purpose. This paragraph comprises no reference.\n * \"and gradually reach to the optimal policy.\": typo\n * The reason why you introduce RL becomes clearer after you introduced QR-DQN. I think that it's however a bit weird as it is framed currently. You should mention that he inspiration and motivation of your work originates from RL and some of its recent developments. \n * \"Discriminators whosetarget is specified\": what do you mean ?\n * \"mixture of gussian\"\n \n## Quantile Regression GAN\n * one really needs to know the trick already to understand equation (3). You must provide a reference here and explain the relationship between quantile regression and (3) as a loss.\n * \"D_{\\tau(batch)}\" instead of \"D_\\tau(batch)\" ? above equation (4)\n* it reads rather uncommon to me to write that infinity (negative or positive) is the objective of the discriminator, with a regularization that constraints its magnitude. I suspect the reason for this to work is: you don't actually provoke some strong collapse of the discriminator output to a specific value (a or b), but rather enforce that it stays somewhere in the approximative range [-k/2M k/2M]. This looks like a nice trick. But I would have appreciated some discussion about it.\n* Is there a reason why you picked 1-wasserstein rather than 2- or p-wasserstein ?\n* We replace Dτ(xreal) by ∞ to prevent it updating to decrease the discriminator output\": awkward sentence. \n* Actually, I don't really understand (7). What is \"x_real\" in the setting of trainng the generator ? You just have fake samples at this stage, and you're indeed using your discriminator for computing your los. I would have written min |a-D(x_fake)|. \n* maybe I'm missing something, but in Alg. 1, I don't clearly understand the difference between your notation o_{i,\\tau} and D_\\thau(x_i). Aren't them the same ? I understand that in your definition of D, you average over the batch. but here in this algorithm box you use a notation D(x_i), which makes it identical with o_i as far as I understand.\n\n## Experiments and results\n\n### toy\n\n* \"arranged in grid (5x5)\": the `x` doesn't render well.\n* \"by normalized computed gradients by generator loss\": awkward\n* \" ourput spaces\"\n* \"steep slope appears\": slopes appear. \"very gentle slope appear\": gentle slopes appear ? \"gentle\" reads awkaward to me.\n* This whole paragraph is extremely badly written and must be written completely, from \"As we can see\" to \"less affected by noise\". The english there is very bad, I don't understand what happened out of a sudden.\n* I don't understand what is depicted for (d) and (e) in figure 3: since the output of your discriminator is a whole distribution, what is it exactly that you decided to plot ? Did you pick a specific quantile ? \n* \"Instead, we can model a discriminator to predict distance. If discriminator predicts distance, itshould be less affected by noise.\" what should I understand here ? I am sorry but I really don't understand the discussion here. are you eventually discarding your model and changing it for something else that would predict a \"distance\", whatever it means ?\n\n### image\n\n* \"the checkerboard artifacts is\" \n* how many quantiles are you using ?\n* Inspecting your results on figures 4-6, I'd say they don't look particularly favorable. i/ For CIFAR10, they look kind of similar with NSGAN and LSGAN, and eventually WGAN-GP gets better. ii/ same result for LSUN, although the proposed method looks better than NSGAN and LSGAN at the end, after much instability. still catched up by WGAN-GP eventually. iii/  for cats, your method looks totally similar to NSGAN and LSGAN, and there was apparently some problem in the finetuning of WGAN-GP, that just didn't train for some reason you should have investigated. It really doesn't look like what happend for it with the other datasets. table 1 hence should not be taken too seriously, unless you really can tell that this WGAN-GP could not be made better on this \"cats\" dataset.\n\n## Acknowledgments: should that be part of a double blind review ?\n\n## References:\n* are not consistent. Sometimes full names, sometimes just initials. please make consistent.  \n\n## Apendixes\nI am not sure appendix B is necessary\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All the reviewers agree that this paper was poorly written, which I agree upon my own reading of this paper. Section 1 is rather telegraphic and difficult to comprehend. Section 2 is cryptic in several respect, including what space of probability distributions the authors consider the Wasserstein distance, what QR task the objective function (5) for the discriminator corresponds to, especially after letting $a=+\infty$ and $b=-\infty$, and so on. The numerical experiment results do not seem convincing enough to demonstrate advantage of the proposal over existing methods. The authors did not respond to the reviews, so that many concerns raised by the reviewers have not been resolved. I would thus recommend rejection of this paper.
Decision: Reject