UniGAN: Reducing Mode Collapse in GANs using a Uniform Generator
Authors: Ziqi Pan
Li Niu
Liqing Zhang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Mode Collapse
GANs

Abstract: Despite the significant progress that has been made in the training of Generative Adversarial Networks (GANs), the mode collapse problem remains a major challenge in training GANs, which refers to a lack of diversity in generative samples. In this paper, we propose a new type of generative diversity named uniform diversity, which relates to a newly proposed type of mode collapse named $u$-mode collapse where the generative samples distribute nonuniformly over the data manifold. From a geometric perspective, we show that the uniform diversity is closely related with the generator uniformity property, and the maximum uniform diversity is achieved if the generator is uniform. To learn a uniform generator, we propose UniGAN, a generative framework with a Normalizing Flow based generator and a simple yet sample efficient generator uniformity regularization, which can be easily adapted to any other generative framework. A new type of diversity metric named udiv is also proposed to estimate the uniform diversity given a set of generative samples in practice. Experimental results verify the effectiveness of our UniGAN in learning a uniform generator and improving uniform diversity.

UniGAN: Reducing Mode Collapse in GANs using a Uniform Generator
Ziqi Pan, Li Niu; Liqing Zhang MoE Key Lab of Artificial Intelligence Department of Computer Science and Engineering Shanghai Jiao Tong University, Shanghai, China
{panziqi_as ust.cnewly1@s (esjtt..die edu. en, zhang-lq@es sj edu. cn
Abstract
Despite the significant progress that has been made in the training Generative Adversarial Networks (GANs), the mode collapse problem remains major chal- lenge in training GANs, which refers to a lack of diversity in generative samples. this paper, we propose new type of generative diversity named uniform diversity, which relates to a newly proposed type of mode collapse named u-mode collapse where the generative samples distribute nonuniformly over the data manifold. From a geometric perspective, we show that the uniform diversity is closely related with the generator uniformity property, and the maximum uniform diversity is achieved if the generator is uniform. To learn a uniform generator, we propose UniGAN, a generative framework with Normalizing Flow based generator and simple yet sample efficient generator uniformity regularization, which can be easily adapted any other generative framework. A new type of diversity metric named udiv is also proposed to estimate the uniform diversity given set of generative samples in practice. Experimental results verify the effectiveness of UniGAN in learning uniform generator and improving uniform diversity.
1 Introduction
Generative Adversarial Networks [1 (GANs) are a technique for unsupervised learning generative models to capture unknown data distribution. Given training samples, two modules, a generator and discriminator, contest with each other in form of a zero-sum game, where the generator aims to produce realistic samples and the discriminator aims to distinguish generative samples from real data. Though originally proposed as generative model for unsupervised learning, GANs have also been widely used in variety of learning scenarios 2 3 due not only their ability to learning highly structured probability distributions but also their thcoretical implications [5 6).
GANs training is notoriously difficult and is very sensitive almost every aspect of its setup due to the dynamic training paradigm λ. Despite significant progress p
B 14 have been made in the training of GANs, the mode collapse [15] problem remains major challenge in training GANs, which refers to a lack of diversity in generative samples [5]. Existing methods reduce mode proposing novel optimization
10
collapse by modifying model architectures 3 16
15
17
18 19
20
algorithms [2122 22 23 24] (55 and regularizations 26
27
28
29 etc. Several works also provide
empirical explanations or formal analysis for mode collapse, including catastrophic forgetting (30 of
the discriminator, sample weighting [31] and binary hypothesis testing [15].
In this paper, adopt the manifold hypothesis [32 33
formalize our analysis and develop framework named UniGAN for reducing mode collapse. The manifold hypothesis assumes that high
"Corresponding author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

dimensional data is concentrated on a lower dimensional manifold. First, this assumption provides us prior knowledge about the unknown ground-truth data distribution p. Specifically, from the manifold hypothesis, it is natural to adopt the uniform distribution over the data manifold M where training samples come from as the ground-truth data distribution p. since samples on M are equally accepted as real samples for training GANs. Given that p is the uniform distribution over M, we propose the uniform diversity of a generative distribution g with formalized definition that measures how close is to by computing the differential entropy of q. considering that the uniform distribution gives the minimum entropy. Recent work [34] [29] also deem that a diverse generator should yield a generative distribution with lower entropy. To quantify the extent of mode collapse with respect to the uniform diversity, we further propose the u-mode collapse, which is a new type of mode collapse compared to the existing (s, --mode collapse [15]. We also present theoretical analysis showing that the u-mode collapse is more generalized than the (e, m-mmde collapse and hence is more robust and efficient in capturing mode collapse. Second, to maximize uniform diversity, based on the manifold hypothesis, we derive the uniformity property of the generator, providing both formalized definition and intuitive geometric interpretation. We also relate the generator uniformity the uniform diversity, showing that the maximum uniform diversity is achieved if the generator is sufficiently uniform.
To maximize uniform diversity, we propose framework named UniGAN, which consists of a normal discriminator, a Normalizing Flow [35 36 37 (NF) based generator, and a simple yet sample efficient regularization for pursuing uniform generator. NFs generative models with specifically designed architectures that are bijective and mosy-to-inverr Hence we use an Na-bd generator since these favorable properties not only make the generator intrinsically satisfy the theoretical prerequisites for maximizing uniform diversity, but also facilitate us to develop a generator uniformity regularization in sample efficient manner. Moreover, the regularization is simple and thus can be easily integrated into any models, which makes our UniGAN easy to adapt to any generative frameworks. To realize the computation of uniform diversity in practice, we also propose a new metric named udiv estimating the uniform diversity of a given set of generative samples. Experimental results show that UniGAN is effective on improving generator uniformity and uniform diversity. Our contributions are threefold:
We propose a new type of generative diversity named uniform diversity and the associated u-mode collapse that measures how close generative distribution is to the data distribution which we chose to be the uniform distribution over the data manifold based on the manifold hypothesis. Theoretical analysis shows that u-mode collapse is generalization of existing (e, - collapse and is more robust and efficient in capturing mode collapse.
We derive the generator uniformity property that is closely related to uniform diversity, and provide analysis showing that the maximum uniform diversity is achieved if the generator is sufficiently uniform. A geometric interpretation is also provided.
We propose UniGAN, a generative framework with an NE-based generator and a simple yet sample efficient regularization on generator uniformity, which can be easily adapted to any other generative framework. We also propose new diversity metric named udiv estimating the uniform diversity of given set of generative samples. Experimental results verify the effectiveness of our UniGAN in improving generator uniformity and uniform diversity.
2 Related Work
Reducing Mode Collapse A variety of methods have been proposed mitigate the mode collapse problem in the training of GANs, including modified architectures
3
16, 38 [5]. optimization
algorithms 21
22
23 39), and loss functions 34 40
26
27 28 Several methods aim to maintain
a bijection between the latent and data space by learning an encoder that projects data back into the
42 38), For example. BiGANs/ALI [3] and VEEGANs 44 42 propose to Iearn an encoder implicitly by making the discriminator to distinguish not only in data space but jointly in data and latent space, and BicycleGAN [16] explicitly encourages reconstruction of both latent codes and training data by combining VAEGAN [1 42 and LRGAN [6]. Several methods
latent space 3 4
20 for mode collapse problem. For example, MAD-GAN [1 employs multiple generators to enforce that different generators capture diverse modes, and PacGAN (i5] reduces mode collapse by modifying the discriminator to make decisions based on multiple rather than single samples from the same class, either real or artificially generated. PacGAN also provide analysis on mode collapse by proposing a formal definition of mode collapse, namely (e, 8) mode collapse from the view of binary hypothesis
make hypothesis of weak generators [8 19 and weak discriminators
45 22
2

---

testing. Similarly, VirtualGAN 45 is proposed to merge multiple samples into one before training the discriminator. Another line of research reduces mode collapse by proposing novel optimization algorithms. Inspired by boosting algorithms, AdaGAN [2I] is proposed as an iterative meta-algorith where many potentially weak generators are greedily aggregated to form a strong composite generator. The unrolled GAN 22 aims to reduce mode collapse and stabilize training of GANs by defining the
generator's ' objective with respect to an unrolled optimization of the discriminator. The progressive growing of GAN [23] aims to grow both the generator and discriminator progressively: starting from a low resolution, then add new layers that model increasingly fine details as training progresses. The BourGAN 24 replaces Gaussian with Gaussian mixture for the prior latent distribution, since the authors assume that the incompatibility between the prior latent distribution and the mode structure is cause of the mode collapse problem, which is also supported by the analysis of [46] from the view of optimal transport. There are also some works propose to reduce mode collapse by proposing novel regularizations. For example, MSGAN 40 26 maximizes the ratio of the distance between samples with respect to the distance between the corresponding latent codes to reduce mode collapse.
Isometry Learning For ease of practical implementation of our regularization on generator unifor- mity, we turn to a special case of uniform generators, namely isometric generators whose Jacobians orthonormal (up constant) 47
(see Section 3.3) which is related to isometry learning. Several methods are proposed to learn isometric generators. Following the definition of an isometry, namely J J = I where is the Jacobian of the generator and 7 is an identity matrix, the authors of LGAN [50] propose to constrain v, 2/ d.g where 0ú is the approximated i-th column vector of J and 6ig the Kronecker delta. In StyleGAN2 [51]. regularization on path lengths of the generator is proposed to encourage that fixed-size step in latent space results in a mnggiimge change in the generative image, which implicitly encourages the generator be an isometry. For the task of manifold learning, an algorithm named Riemannian relaxation is proposed to construct an isometric mapping in 47]. However, this algorithm does not deal with high-dimensional input such as images, as mentioned by 44). In RaDOGAGA [49]. for an autoencoder, an isometric decoder is learned by injecting random noise to the latent code before reconstructing the input data. There are also several methods 522 learn isometric mappings aided by traditional dimensionality reduction algorithms [3
48
49
3
Methodology
We firstly propose the formal definitions of uniform diversity and the associated u-mode collapse in Section 3.1, then relates the uniform diversity with the generator uniformity property in Section 3.2
which motivates us to develop the UniGAN framework for reducing mode collapse in Section 3.3
3.1 The Uniform Diversity and w-Mode Collapse
Let PM be the set of all distributions over finite domain M. The uniform diversity of a distribution q E PM measures how close q is to the uniform distribution over M, which defined as
Definition 1 (Uniform diversity). The uniform diversity of E PM is given as Ug A c2c where M. is the measure of M and Hy is the differential entropy of 9. namely
E [0. 1).
mm
Hgg A - f Jauppte) log of (x) dg (x),
(1)
where supp (4) (o E Mlg' (x) + 43 is the support of For H. = 00, we define Un 2 0.
Note that given distribution p E PMM we use p' to denote its Probability Density Function (PDF). We employ differential entropy H. in Definition since it measures the uniformity of distribution q. Specifically, from information theory [54] we know that Hl could be 0o which corresponds to the case where is extremely nonuniform (e.g., q' is a Dirac delta function where all probability mass is concentrated on single point), and H. is upper bounded by log m.M since M is finite, where H, = log m.M achieves only if q is the uniform distribution over M (see supplementary). The more uniform the q, the higher the Hg. The uniform diversity Ug is further defined to normalize the range of H. to [o, 1]. Based on uniform diversity, the u-mode collapse is formally defined as
Definition (u-mode collapse). We say that q E PM exhibits u-mode collapse for 0 < u < 1 if uniform diversity Uy is equal to u.
3

---

u
Feasible Region of (u, E)
1.0
pu=
0.35. 0.50
0.8
0.6
pv 0.56
0
mild (z, 6)-mode collupse,
P 0.25.
0.30
0.90
severe u-mode collapse
severe (e, 8)- mode collapse
0.2
9.10 Q 0.95. 5 0.99
pu
0.02
0.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.8
0.9
1.0
severe u-mode collapse
The more severe (c, 8)-ode collupse. more severe u-mode collapse.
T.ste)
Ro-o
My o.u(E)
no =o=0000 0
(a) Feasible regions of (u, e) given different 6.
(b) The Venn diagram.
In Fig. (aa we illustrate the feasible region of (u, E) for E {0.5, 0.8, 0.9, 0.99) respec- tively. Given 6, the corresponding (s, 5) curve with respect is plotted, and the corresponding feasible region of (u, E) is shown as the shaded region under the curve. We also select four pairs of (c,5) with increasing ō and decreasing e, and then plot points PH, that gives the maximum for the corresponding (s,6) 6) to see how u-mode collapse changes as (s, -mood collapse gets more severe. We also select a point Q² to show that (e, 8-mmode collapse could be mild while u-mode collapse is severe. Different is represented by different color. In Fig. [(b] a Venn diagram is used to show the relationship between (e, 6m-ode collapse and u-mode collapse.
Figure 1:
The proposed u-mode collapse measures how far distribution g PM deviates from the uniform distribution over M. Lower u indicates more severe mode collapse, and for 24g = 1, is sufficiently uniform and does not exhibit u-mode collapse.
Connection to (e, -mooee collapse Recently, another formal definition of mode collapse named (c, 8) mooo collapse is proposed in [15], which leads to a wwo-dimensional representation of the mode collapse region of the pair of the data distribution p and generative distribution q. We discuss how it relates to our proposed u-mode collapse. Recall that (s, m collapse is formally defined as
Definition ((E, -mode collapse, [5]). Given data distribution E PM, a generative distribution q E PM exhibits (s, -modde collapse for < 5 < 1 if there exists S M such that (S) ō and q (S) < e.
Given that the data distribution p is chosen as the uniform distribution over M, we show that u-mode collapse is more generalized than (s, momee collapse, and is more robust and efficient in capturing mode collapse. We first present the following Theorem (see supplementary for proof):
Theorem 1. For q E PM that exhibits (e, 6)-mdde collapse, 0 < Un & (s,6), where (s,
(1-6) E 1). That is, generative distribution that exhibits (e, 8)-mode collapse at least exhibits π (s, 5)-mode collapse.
(9)
1-6 (1-8)
Theorem fundamentally specifies the feasible region three-dimensional representation (u, E, 8). which we illustrate in Fig. [(a) using two-dimensional diagram that shows the respective feasible we then present the following corollaries:
regions of (u, 8) for different 8. Based on Theorem |
Corollary 1. The following claims can be deduced from Theorem 7
(i) Let u² (E,8) be the maximum uniform diversity that can be achieved for generative
distribution that exhibits (e, 5)-mode collapse. Then u² (s' 6"'88) < u² (s, holds for 8' > and el E. Moreover, vg > 0, 3 v such that (E,8) < Ef < μ and 8 v.
(ii) Let q E PM be generative distribution that exhibits u-mode collapse for any E [0, 1), then for any 8 E (0, 1) and & > 0, g can exhibit (E, 8) -)-mode collapse for 5 < E.

---

(0.01, mooec collapse, 0.01-mode collapse
(0, m.00)-mmode collapse, O-mode collapse
1.0
14
0.0
0.0
0.0
0.01
0.59
0.905
99.0
wxet
9% se 0.014a.wclx)
(a) Both (e, 8)- and u-mode collapse severe.
(b) Mild (e, and severe u-mode collapse.
Figure 2: Illustrative distributions r. q over domain [0, 1]. For r and q, the corresponding PDFs r' and of are plotted using colored solid lines, where the PDFs restricted to different sub sub-domains are represented by different colors for each distribution. The PDF p' of the uniform distribution p over [0, 1] is plotted using black dashed line, where p' = 1. In Fig. 2(b) use A0.995 (x) to denote the Dirac delta function at 0.995. The mode collapse exhibited by each distribution are indicated.
See supplementary for proof. We provide intuitive explanations to aforementioned claims. In terms of (c, -mode collapse, larger and smaller & indicate more severe mode collapse [15. therefore, we can learn from Claim (i) that as (E, - collapse becomes more severe, the maximum uniform diversity that can be achieved decreases, Le., u-mode collapse also becomes more severe. Moreover, Claim ( states that w-mode collapse can be sufficiently severe (i.e,, u 0) as (e, m-modde collapse becomes sufficiently severe (i.e., E + 0,
1). An illustrative example is given in Fig. [(a] where P are points in feasible regions of (u, E) given ö such that P", gives the maximum that can be achieved for the given (). We can intuitively see that
increases and decreases, decreases for P and u can be sufficiently small as + 1 and = - 0. Meanwhile, Claim (ii) indicates that distributions that exhibit severe u-mode collapse can exhibit mild (e, 5) -mode collapse (i.e., e Rs 6), illustrated by the point Q" shown in Fig. [(a] Hence in summary, the u-mode collapse severe as long as the (s, 5) moode collapse is severe, however, the (s, -omoe collapse could be mild while the u-mode collapse is severe. The relationship between (E, 8) -mode collapse and u- u-mode collapse can be depicted using the Venn diagram as shown Fig. [(b) Specifically, while severe (c, 5mmode collapse is covered by severe u-mode collapse, the (e, ô -mode collapse may not be able to identify severe u-mode collapse effectively. Therefore, u-mode collapse can be regarded as generalization of (e, mm-dd collapse, and is more robust and effective in capturing mode collapse.
We also provide distributions r, q in Fig. 2 as illustrative examples of different mode collapses. Both r and q are defined over domain [0, 1]. i.e., q E P|u.1]- We take r shown in Fig. 2(a) as an example that exhibits severe (c, -mode collapse and severe u-mode collapse. By denoting S - 00.1, 1), can observe that r (S) = 0.01 meanwhile p (S) = 0.99, namely exhibits (c, m collapse for 0.01 and 6 0.99, which corresponds a severe (e, 6mmmde collapse since
and 1. Regarding u-mode collapse, we can compute that u 0.01, which corresponds to a severe u-mode collapse since и s 0. In Fig. 2(b) by denoting S 2 [0.99, 0.995) U (0.995, 1], we can observe that q (S) meanwhile p (S) 0.01, hence q exhibits (e, mmode collapse for 0 and 8 0.01. Regarding u-mode collapse. due to the infinity of Dirac delta function A0.995 (I) at 0.995, we can compute that u 0, since Hy -0. Therefore, q exhibits severe u-mode collapse but mild
(s, 0) -ode collapse, since N RJ 0 and u = 0.
3.2 The Generator Uniformity Property
We now relate the uniform diversity to the property of the generator which we refer to as uniformity.
Let Z C K be the latent space and g Z - D be the generator with K < D. From the manifold hypothesis [32 33 the image of g, M 2 (Z) {g (z) |a E Z), is assumed be K -immenniional manifold residing in the ambient space IR and pushes forward the prior distribution p. E P2 to a
generative distribution E PM that is denoted by g#P.. According to the definition of PDF, for x g (2) we have that
(x) P/ (z)
(2)
dVMlz
5

---

2
Area
(g(z)
0g:r)
det((r/(z))
dg(z²
dz:
021
02;
dzz
(2)
d2z
dzy
(z(2)
dx1
d21
dz,
((z)
g(x))
M g(2), 9 uniform.
Latent Space
g(2), , nonuniform.
Figure 3: Intuitive illustrations of the generator uniformity. We set up two generators g 2 M and ÿ M with different uniformity, where both the latent space Z and the manifolds M, are 2-dimensional. The coordinate lines of Z, M, M are plotted, where the correspondence between Z and M (resp., MM for g (resp., 9) is indicated by the colors of coordinate lines.
where dV2|= and dV are the volume elements 55 of 2 at and M æ. respectively. Intuitively, the probability density is the ratio of the probability mass p', (z) dVz|= to the volume dVMlz of the region where that probability mass is concentrated. Assume that g local diffeomorphism [55], the volume element dVMls la is given according to the Riemannian geometry [55] as
dVMlz det G (z)dV2l2
where (z) A J (
R K is the matrix form the Riemannian metric tensor [55], and Jg (2) E R is the Jacobian of at z. Therefore, by plugging Eq. (3) into Eq. (2) we arrive at
(=)
o (x) det J Jz)
= (2),
which is also known the change of variables formula 36 37 35
As aforementioned, the uniform diversity of generative distribution q E PM is maximized only if is the uniform distribution over M, which is equivalent to q' being constant over M. For ease of analysis, we assume that Ps is the uniform distribution over Z, i.e., p'g constant over 2. In such case, is obvious that q' constant over M only if det Jj Jy is constant over Z. On this account, we formalize the definition of the generator uniformity property as
Definition (Generator uniformity). Given a generator g 2 - M that is local diffeomorphism between Z R and M C R.. with K < D, we say that is uniform, if det Jg J is constant.
Obviously, the connection between the generator uniformity and maximized uniform diversity can be formalized the following Theorem 2 (see supplementary for proof)
Theorem 2. Assume that the generator g Z + M is local diffeomorphism between Z C R and M C TR with K < D. and the prior distribution pz E Pz is the uniform distribution over Z. The uniform diversity of the generative distribution q 9#P= maximized, if g is uniform.
We provide intuitive illustrations for a better understanding of the generator uniformity property. In Fig. we set up generators g, with different uniformity. The latent space Z is 2-dimensional Euclidean subspace, and we divide into even grid blocks by drawing coordinate lines along axes z and 22, respectively. The generator y (resp., 9) pushes forward coordinate lines of Z those of M fresp...), Since the prior distribution ps E Pz is the uniform distribution over Z and the area of each grid block is equal, we know that each grid block of Z contains the same probability mass. So each grid bock of M (resp., M) also contains the same probability mass, since the coordinate lines between and M (resp., MM) correspond. For M. because the area of each grid block is equal, ratio of probability mass to the area, i.e., the probability density, for each grid block is equal too, i.e., the generative distribution is the uniform distribution over M. However, the probability density for each grid blocks of M is not equal, since the area of each grid block is not equal. Therefore, given that each grid block contains the same probability mass, the equality of areas of grid blocks matters regarding the uniformity of the generative distribution, and the "equality of areas of grid blocks" is essentially intuitive interpretation of the generator uniformity. We also explain how / J

---

relates to the area of a grid block. As shown in Fig. 3. for grid block of M, if we consider p1
821
and a di Ozz
as two adjacent sides of that grid block, then the computation of
det (JJJ) = P1 ||
P p1- = (Ileal leell sin 0)² = (Area p2))
(5)
implies that det (JTJ) actually gives the area of the parallelogram spanned by Pr and p2, namely the area of the grid block, where 9 is the angle between P1 and Pz. Hence if det (JJ is constant, the area of each grid block is equal, and the generator is uniform since the "equality of areas of grid blocks" is an intuitive interpretation of the generator uniformity, which coincides Definition 4
3.3 The UniGAN Framework
We introduce our UniGAN framework for reducing mode collapse based on aforementioned analysis from manifold view. Our goal is to maximize the uniform diversity of the generative distribution, which can be done by pursuing a uniform generator, as indicated by Theorem 2 However, to apply Theorem we have to satisfy its prerequisite, i.e., the generator g should be a local diffeomorphism. Moreover, constraining det J Jg to be constant involves not only computing the Jacobian Jg. but also computing the determinant, both of which are computationally expensive. Hence is nontrivial to implement the generator uniformity constraint in practice.
Regarding the above problems, we propose to employ an NF-based generative model. On one hand, NFs are intrinsically bijections, therefore, the diffeomorphism prerequisite of the generator can be satisfied. On the other hand, NFs are specifically designed architectures with tractable determinant of Jacobian [35 33 37), which may ease the practical implementation of the constraint. Unfortunately, the above advantages of NFs are at the cost of maintaining latent space of as high a dimensionality as the data space, which leads to over- peee-arameterized models and very slow training. To overcome this problem. we propose to use an NF- generator g : Z E with hierarchical architecture as follows
K + R for arbitrary K D
= fL ez ft-1 ..... f1oe1,
(6)
where ʃ :R - RKK is bijective and invertible NF module (e.g., coupling layers 36 37 56) with tractable determinant of Jacobian, e; IR K, - RRK boosts the dimensionality of the input feature x from K;-1 K, by simply padding K, K;-1 zeros at the end of [57]. and the dimensionalities
of hierarchical features satisfy K = Ko < K1 ... < K D. See supplementary for detailed introduction. However, a notable issue is that det JJ Jg is intractable despite tractable determinants of Jacobian for f.. Hence for case of implementation, we turn to pursuing a special case of uniform generators where the K singular values of Jg. { are equal and constant over 2, considering the fact that det J Jg = 0ī0? .... o This is further realized by constraining the minimum and the maximum singular values of Jg to be equal and constant over Z. Therefore, the main concern is how to obtain the minimum and the maximum singular values of Jg-
We propose the Linearized Transpose (LTT) technique to estimate the spectral norm of the Jacobian of a given network in a sample efficient manner. Due to space limitation, we use T.T G (z) to denote the estimated spectral norm of Jg at 2 E 2, and the detailed introduction of our proposed LT is provided in supplementary. Since the spectral norm essentially the maximum singular value, we know that LT, (2) is the maximum singular value of J, at z. To obtain the minimum singular value of Jg, we propose to utilize the invertible property of fi, which makes it tractable to obtain the pseudo inverse of g. g¹ RDD + KR defined
g¹ -cto oe.ffeo..ee..i. sT- o f
where f. RK R is the inverse of fú, and e/ RK - K.-1 is the pscudo inverse of that reduces the dimensionality of the input feature x from K; to K;-1 by simply dropping K; - K;-1 elements at the end of a. Given g, g", we can prove that (see supplementary)
Proposition 1. Vz 2, Jat (g (z)) Ju (2) E KxK holds, where I is the identity matrix.
Given the above Prop. I by denoting o* (J) (resp., a. (J)) as the maximum (resp., the minimum) singular value of given matrix J, we intuitively illustrate that o. (Jy (z)) (=)r((ot))))
by using
7

---

T M u = g0
u' gives the maximum s.v. of nn = V/u'll
u²= 11 -ー =
|w'll
Muuull
u=/ge, v=/2u
||2|||
11
= g()
Vu,
vv.
|||||
|1 2/
v" gives the minimum s.v. of
||0*||
Z
z = h(x)
Mg
||
||
Mnu*11
s
Figure 4: Illustration of the relationship between 0x (Jg) and of (Ji), where we use h é g'. Given
that u² E M gives the maximum singular value of Jn. we have oN (Jg) --.3..-
Fig. 4 However, the illustration of Fig- is only applicable in the case where the maximum singular value of J'' (g (z)) can be given by a vector u² E M C R2 that is on the tangent space of M at (z), see supplementary for illustration. To tackle this problem, we observe that by encouraging o- ()())((())) a² (Jg (z)), the maximum singular value of Jgt (g (=)) can be given by vector
"" TTo To see this, assume that ü" ERD gives the maximum singular value of Jg: (g (z)),
we have that (J,t,) 1 & J
|
1 ū" 3
& o (Jg), where the first equality
holds if and only u² _ Jo T E T C RDD gives the maximum singular value of Jgt- So in the case of ot ( (o(e))) = o² (Jg (2)). we have a. (Jg (2)) or (J (s(=))) from Fig. 4 and hence
ox (Jy (z)) = (Jg (z)). namely the minimum and the maximum singular values of Jg are equal. Therefore, by letting be the moving average of the maximum and the minimum singular values of Jg. the regularization for the generator uniformity implemented practice formalized
{ (I. (IT a)² +
(LT (( a
2
Lgunif Egups
-2) LT g' (z
which is simple yet sample efficient regularization due to the superior sample efficiency of LT (see supplementary). We then combine the above NF-based generator and a normal discriminator into a generative framework augmented using our generator uniformity regularization Lgunit, obtaining UniGAN whose objective function is as follows,
Lunigan Ggan Aganif Lgunif,
where Cyan is the original objective of GANs, and Agunit is the balancing hyper-parameter. Eq. (9) also implies that Lguntf can be easily integrated into any models by appending Lgunif to the original objective function, and UniGAN can be adapted to any other generative framework by replacing Cgan with the corresponding objective function.
Experiments
We show through experimental results that our proposed UniGAN framework is effective in learning a uniform generator and improving uniform diversity of the generative distribution. To demonstrate the adaptability of our UniGAN framework, we adapt UniGAN for a variety of existing models that reduce mode collapse, including GAN (I) BiGAN BVALI [4), MSGAN 440 26).. VEEGAN (38), PacGAN (15], MDGAN (34 and RegGAN B4. We also show that these models are inferior to ours regarding uniform diversity. For intuitive demonstration, we firstly provide results
toy synthetic data manifold, then provide results on simple datasets including MNIST (58). FashionMNIST (59) and their colored version (22. and CIFARIO [60] We also provide results on natural image datasets
including CelebA [61] FFHQ [62, AFHQ 63 and LSUN (44 To implement the computation of
uniform diversity of given set of generative samples in practice, we also propose a diversity metric named udiv. Due to space limitation, the implementation details about our proposed udiv metric and experiments can be found in supplementary.

---

MSGAN
VEEGAN
MDGAN
RegGAN
GAN
BiGAN
PacGAN
(a) Manc.
(b) Manifolds generated by different models.
Figure 5: For real samples Maec in Fig. ((a) we visualize training data in the top image, and show the three major modes using heatmap as the bottom image. We visualize manifolds generated by different models in Fig. [(b) where the top row visualizes results of baseline models, and the bottom
row visualizes results of UnGAN adapted to the corresponding baseline models.
MSGAN
VEEGAN
MDGAN
RegGAN
GAN
BIGAN
PacGAN
Visualization of probability densities (top row) and log det JTJ (bottom row) of samples generated by baseline methods. The probability densities are estimated by the method we introduced in the udiv metric, see supplementary. Magnitudes of values are reflected by using heatmap.
Figure 6:
We provide results in this section on synthetic datasets for the sake of intuitive understanding, while leaving more results on other datasets in supplementary due to space limitation. We use a 2D synthetic dataset sector which samples points from sector data manifold (ar, v) 1 & /x² + < in the first quadrant. We set up three major modes to simulate unbalanced distribution of modes in real data. Specifically, we use fr, 1.2, ar 0.1 obtain R, - (r(()) A (Hr. o,) (r(()) [1, 21] and use j4 = 7/4, og = 7/2 to obtain O & {0(+) N (no, σΦ) (0()) E r/]] 1=1 and then perform polar transform, obtaining M₁ - {x((0, (8) N as the first mode. Similarly, we use ,r = 1.8, i=1 with (0) = (() cos (+) and y (() sin (()
N
3 / 0g x/ /2 to obtain the second mode M2, and use μ4, 1.8, o, 0.1 and μg 7 220 do π /20 obtain the third mode
0.1 and
M3. Finally, we obtain Masc A U M See Fig. 5(a)
We train baseline models and the corresponding UniGAN adaptations on the proposed 2D synthetic manifold sector. The manifolds generated by different models are intuitively visualized in Fig. (5b) From Fig. (b)), we learn that for almost every bascline model, their generative samples lack uniform diversity because we can intuitively observe that they distribute nonuniformly. Meanwhile, samples generated by our UniGAN consistently distribute uniformly, hence our UniGAN outperforms baseline models terms of uniform diversity, and can better interpolate between training samples on the data manifold compared to baseline models. We also provide quantitative comparison between baselines and our UniGAN in terms udiv and generator uniformity as shown Tbl. We see that compared to baseline models, our UniGAN achieves much higher udiv score and maintains lower variance of log clt. JT J, therefore, our UniGAN consistently outperforms baseline models in terms generator uniformity and uniform diversity of generative samples.

---

Table 1: Quantitative comparison between different models on the uniform diversity and generator uniformity for the Maec manifold. The uniform diversity and generator uniformity is measured by the udiv score and the gumif score, respectively, where (resp., +) indicates that larger (resp.. lower) values are better. The gunif metric is gives as the Coefficient of Variation (CV) of log det. J J over latent space, which is a standardized measure of dispersion of a distribution. For each metric, the first row and the second row give the results of baseline models and UniGAN adapted to the corresponding baselines, respectively, and we also provide the p-value of significant test between results of baselines and the corresponding UniGAN to avoid randomness in the third row. For each baseline model, the better value among the baseline result and the corresponding UniGAN result is shown in bold.
Metric
MSGAN
VEEGAN
MIGAN
RegGAN
GAN
BiGAN
PacGAN
0.46
0.82
0.52
0.75
0.50
0.80
0.78
udiv f
0.91
0.93
0.93
0.91
0.90
0.93
0.93
2.07e-5
1.42e-10
8.62e-1
3.59e-3
6.43e
1.26e
2.90e
0.25
0.20
0.25
0.22
0.26
0.20
0.21
gunif +
0.01
0.02
0.02
0.02
0.01
0.01
0.02
1.65e
7.69e 13
4.02g-8
1.61e
6.91e
8.37e 9
6.46e-7
Table 2:
Correlations between probability densities and the corresponding log det JT J values for each baseline model. The first and second rows represent the mean and the 90% confidence interval of correlations over 10 runs with random initialization, respectively.
Metric MSGAN
VEEGAN
MDGAN
RegGAN
GAN
BiGAN
PacGAN
0..00
-0.85
-0.81
-0.82
-0.82
-0.85
-0.83
corr
+00088
:00.02
+.010
+.0.55
+00.99
+.0.0
#0015
We further experimentally demonstrate that the uniform diversity is closely related with the generator uniformity, which justifies our theoretical analysis in Sec. 3.2) Specifically, in Fig. [6] we visualize the correspondences between the values of log J/ and the probability densities on the generated manifold for different models trained on the 2D synthetic dataset sector. From Fig. 6 we learn that the probability densities of generated samples and the corresponding values of log det J are inversely correlated, i.e., the higher the probability density, the lower the values of log det J J. Quantitative results provided in Tbl. show that the correlation coefficients corr between the probability densities and the values of log del JJ J are lower than -0.8 across all baselines, which indicates that they are strongly negatively correlated and coincides with the intuitive illustration of the generator uniformity in Fig. 3 and the analysis on the relationship between the generator uniformity and uniform diversity.
2
5 Conclusion
In this paper, we propose the uniform diversity, which is associated with new type of mode collapse named u-mode collapse where generative samples distribute nonuniformly over the data manifold, and is closely related with the generator uniformity property. To maximize the uniform diversity, propose a simple yet effective generative framework named UniGAN with an NF-based generator and sample efficient regularization the generator uniformity, which can be adapted to any other framework. A new type of diversity metric named udiv is also proposed to estimate uniform diversity in practice. Experimental results verify the effectiveness of our UniGAN framework.
Acknowledgements
The work was supported by the National Science Foundation of China (62076162), and the Shanghai Municipal Science and Technology Major/Key Project, China (2021SHZDZX0102, 2051 195110 100300).
10

---

References
[1] lan Goodfellow, Jcan Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurlPS, 2014.
[2] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. In PAMI, 2013.
[3] Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell. Adversarial feature learning, In ICLR, 2017.
[4] Vincent Dumoulin, Ishmacl Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Arjovsky, and Aaron Courville. Adversarially learned inference. In ICLR, 2017.
[5] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR, 2016.
Masatoshi Uchara, Issei Sato, Masahiro Suzuki, Kotaro Nakayama, and Yutaka Matsuo. Generative adversarial nets from density ratio estimation perspective. In arXiv, 2016.
[7] Vaishnavh Nagarajan and J Zico Kolter. Gradient descent gan optimization is locally stable. In NeurlPS. 2017.
[8] Mario Lucie, Karol Kurach, Marcin Michalski, Sylvain Gelly, and Olivier Bousquet. Are gans created equal? a large-scale study. In NeurlPS, 2018.
[9] Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In NeurlPS, 2016.
[10] SC Martin Arjovsky and Leon Bottou. Wasserstein generative adversarial networks. In ICML, 2017.
[11] Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. In NeurlPS, 2017.
[12] Piotr Bojanowski, Armand Joulin, David Lopez-Paz, and Arthur Szlam. Optimizing the latent space of generative networks. In ICML, 2018
[13] Yan Wu, Jeff Donahue, David Balduzzi, Karen Simonyan, and Timothy Lillicrap. Logan: Latent optimisa- tion for generative adversarial networks. In CORR, 2019.
[14] Karol Kurach, Mario Lučić, Xiaohua Zhai, Marcin Michalski, and Sylvain Gelly. A large-scale study on regularization and normalization gans. In ICMI., 2019.
[15] Zinan Lin, Ashish Khetan, Giulia Fanti, and Sewoong Oh. Pacgan: The power of two samples in generative adversarial networks. In NewrlPS, 2018.
[16] Junyan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexci A Efros, Oliver Wang, and Eli Shechtman. Toward multimodal image-to-image translation. In NeurlPS, 2017.
[17] Amab Ghosh, Viveka Kulharia, Vinay P Namboodiri, Philip HS Torr, and Puneet K Dokania. Multi-agent diverse generative adversarial networks. In CVPR, 2018
[18] Quan Hoang. Tu Dinh Nguyen, Trung Le, and Dinh Phung. Mgan: Training generative adversarial nets with multiple generators. In ICLR, 2018.
[19] Wei Li, Li Fan, Zhenyu Wang, Chao Ma, and Xiaohui Cui. Tackling mode collapse in multi- generator gans with orthogonal vectors. In Pattern Recognition, 2021.
[20] Karttikeya Mangalam and Rohin Garg. Overcoming mode collapse with adaptive multi adversarial training. In arXiv, 2021.
[21] Tlya O Tolstikhin, Sylvain Gelly, Olivier Bousquet, Carl-Johann Simon-Gabriel,, and Bernhard Schölkopf. Adagan: Boosting generative models. In NeuriPS, 2017.
[22] Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein Unrolled generative adversarial networks. In ICLR, 2017.
[23] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In ICLR, 2018
[24] Chang Xiao, Peilin Zhong, and Changxi Zheng. Bourgan: Generative networks with metric embeddings. In NewrlPS, 2018
11

---

[25] Ricard Durall, Avraam Chatzimichailicis. Peter Labus, and Janis Keuper. Combating mode collapse in gan training: An empirical analysis using hessian eigenvalues. In VISIGRAPP, 2020.
[26] Dingdong Yang, Seunghoon Hong. Yunseok Jang, Tianchen Zhao, and Honglak Lec. Diversity-sensitive conditional generative adversarial networks. In ICLR, 2019.
[27] Shaohui Liu, Xiao Zhang, Jianqiao Wangni, and Jianbo Shi. Normalized diversification. In CVPR, 2019.
[28] Kanglin Liu, Wenming Tang, Fei Zhou, and Guoping Qiu. Spectral regularization for combating mode collapse in gans. ICCV, 2019.
[29] Pirazh Khorramshahi, Hossein Souri, Rama Chellappa, and Soheil Feizi. Gans with variational entropy regularizers: Applications in mitigating the mode-collapse issue. In arXiv, 2020.
[30] H Thanh-Tung and T Tran. On catastrophic forgetting and mode collapse in generative adversarial networks. In CORR, 2018.
[31] Aksel Wilhelm Wold Eide, Bilif Solberg, and Ingebjørg Kásen. Sample weighting as explanation for mode collapse in generative adversarial networks. In CORR, 2020.
[32] Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. In JAMS, 2016.
[33] Hang Shao, Abhishek Kumar, and P Thomas Fletcher. The riemannian geometry of deep generative models. In CVPR Workshop, 2018.
[34] Tong Che, Yanran Li, Athul Paul Jacob, Yoshua Bengio, and Wenjie Li. Mode regularized generative adversarial networks. In /CLR, 2017.
[35] Ivan Kobyzev, Simon JD Prince, and Marcus A Brubaker. Normalizing flows: An introduction and review of current methods. In PAMI, 2020.
[36] Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation. In ICLR Workshop, 2015.
[37] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. In ICLR, 2017.
[38] Akash Srivastava, Lazar Valkov, Chris Russell, Michael U Gutmann, and Charles Sutton. Veegan: Reducing mode collapse in gans using implicit variational lcarning. In NeurlPS, 2017.
[39] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilihrium. In NeurlPS, 2017.
[40] Qi Mao, Hsinying Lec, Hungyu Tseng, Siwei Ma, and Minghsuan Yang. Mode seeking generative adversarial networks for diverse image synthesis. In CVPR, 2019.
[41] Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. Autoencoding beyond pixels using learned similarity metric. In ICML, 2016.
[42] Ming-Y Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In NeurlPS, 2017.
[43] Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang. Generalization and equilibrium in generative adversarial nets (gans). In ICML, 2017.
[44] Jerry Li, Aleksander Madry, John Peebles, and Ludwig Schmidt. Towards understanding the dynamics of generative adversarial networks. In CORR, 2017.
[45] Adel Abusitta, Omar Abdel Wahab, and Benjamin CM Fung. Virtualgan: reducing mode collapse in generative adversarial networks using virtual mapping. In IJCNN, 2021.
[46] Lei, Yang Guo, Dongsheng An, Xin Qi, Zhongxuan Luo, Shing-Tune Yau, and Xianfeng Gu. Mode collapse and regularity of optimal transportation maps. In CORR, 2019.
[47] James McQueen, Marina Meila, and Dominique Joncas. Nearly isometric embedding by relaxation. In NewrIPS, 2016.
[48] Amos Gropp, Matan Atzmon, and Yaron Lipman. Isometric autoencoders. CORR, 2020.
[49] Keizo Kato, Jing Zhou, Tomotake Sasaki, and Akira Nakagawa. Rate-distortion optimization guided autoencoder for isometric embedding in euclidean latent space. In ICML, 2020.
12

---

[50] Guo-Jun Qi, Liheng Zhang, Hao Hu, Marzieh Edraki, Jingdong Wang, and Xian-Sheng Hua. Global versus localized generative adversarial nets. In CVPR, 2018.
[51] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In CVPR, 2020.
[52] Gautam Pai, Ronen Talmon, Alex Bronstein, and Ron Kimmel. Dimal: Deep isometric manifold learning using sparse geodesic sampling. In WACV, 2019.
[53] Joshua B Tenenhaum, Vin de Silva, and John C Langford. A glohal geometric framework for nonlinear dimensionality reduction. In Science, 2000.
[54] Fazlollah M Reza. An introduction to information theory. Courier Corporation, 1994.
[55] Peter Petersen. Riemannian geometry. Springer, 2006.
[56] Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1xl convolutions. In NeurlPS, 2018.
[57] Johann Brehmer and Kylc Cranmer. Flows for simultancous manifold learning and density estimation. In NewrIPS, 2020.
[58] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffnnr. Gradient- learning applied to document recognition. In Proceedings of the IEFE, 1998.
[59] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-man minss:: a novel image dataset for benchmarking machine learning algorithms. In arXiv, 2017.
[60] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Citeseer, 2009.
[61] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015.
[62] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In CVPR, 2019.
[63] Yunjey Choi, Youngjung Uh. Jaejun Yoo, and Jung-Wos Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In CVPR, 2020.
[64] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construc- tion of a large-scale image dataset using deep learning with humans in the loop. In CORR, 2015.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See supplementary.
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] See supplementary.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes] We include code in the supplementary material, including the implementation of our NF-based generator, the LT technique, the generator uniformity regularization, and the udiy metric.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See supplementary.
13

---

(c) Did you report enror bars (e.g., with respect to the random seed after running exper- iments multiple times)? [Yes] We report p-value of significant test. See Tbl. | and supplementary.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See supplementary.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] See supplementary.
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "tNW-Gvk0YYR", "reviewer": "Reviewer_YpC8", "summary": "This paper proposes UniGAN a new approach to alleviate mode collapse in GANs. Assuming the manifold hypothesis, the authors motivate training a generator with uniform distribution over the data manifold M. They encourage the uniform distribution by arguing that samples on the manifold M are equally accepted as real samples for training GANs. UniGAN restricts the generator to be Normalizing Flow (NF) based to perform effective and simple regularization in pursuit of a uniform generator. Authors also propose a new measure of performance of GANs and show the effectiveness of their methods on several benchmark datasets experimentally.", "questions": "Please look at my above comments.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n\n1- the idea of the paper clearly has been explained.\n\n2- The idea is novel.\n\n3-The paper has solid theoretical results.\n\n4- A large set of experiments has been done.\n\nCons:\n\nOne of the main reasons GANs have been preferred to other generative models despite mode collapse is their sample quality. However, the current paper restricts the generator to an NF-based model, which restricts the flexibility of the generator and consequently lowers the quality of samples. This also can be shown by looking at the Inception score of the trained model on CIFAR; UniGAN achieve IS of less than 3.9; however, PacGAN archives IS of more than 6 (please refer to Self-Cond-GAN or PGMGAN) when used with proper architecture.\n\nAlso, I have not convinced necessarily a uniform distribution is necessarily better. For an instant, consider toy data of 1D mixture of normals distribution. The support of this distribution is the entire R, and it is not ideal to have a uniform distribution over the entire space. Instead, one may prefer some samples to others.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "l-KnXYFu58c", "reviewer": "Reviewer_cP8d", "summary": "This paper proposes a new type of generative diversity named uniform diversity, and thus introduce the corresponding $\\mu$-mode collapse. And the authors propose a new framework called UniGAN to learn the uniform generator. This framework is formulated with a normalizing flow based generator and a uniformity regularization. Moreover, the authors also propose a new metric called udiv to estimate the uniform diversity. ", "questions": "Please refer to the weakness.", "limitations": "More experimental results, especially the visual analysis can be provided.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThis paper proposes a new type of uniform diversity, and analyze the corresponding importance and behavior. And the proposed UniGAN shows great performance under the metric of udiv.\n\nWeakness:\n1. The experiments are conducted with small image resolution. And the uniform diversity on the high resolution images is not analyzed. The authors can combine the framework with some up-sampling layers to analyze the uniform diversity with the increase of image resolution.\n\n2. The analysis results of Fig.5 can be made on other datasets to demonstrate the correctness of the conclusion on different datasets, like FFHQ, AFHQ, etc.\n\n3. Will the proposed uniformity regularization suitable for dealing with other types of mode collapses? The relations between other types of mode collapses can be analyzed.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "_hIcZnLecnT", "reviewer": "Reviewer_cYPc", "summary": "The paper proposes a simple yet effective way to mitigate the mode collapse issue of GANs. To this end the authors introduce the generator uniformity property, which is utilized to regularize a flow-based generator. Lastly, a new form of diversity is introduced, labelled uniform diversity.", "questions": "I suggest the authors follow my suggestions regarding further evaluation as well as the inclusion of a discussion on ethical considerations.", "limitations": "The authors have provided adequate analysis of the limitations of the work. However, I would like to see a discussion on the societal impact of the work.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n\nAlthough I am not particularly experienced in this particular sub-area (i.e., mitigating mode-collapse in GANs), I appreciate the effort the authors put into the technical part of the motivation. The main idea is clear and well motivated. The authors also made sure to thoroughly evaluate their model against a number baselines and metrics, which highlight the efficacy of the method.\n\nWeakness: \n\nAlthough I don’t think the paper has any strong weakness, I would like to see how the approach performs on other mode-coverage benchmarks, e.g.,\n- Fashion-MNIST and partial MNIST of [1]\n- Stacked MNIST of  [2]\n\nLastly, I would also appreciate a discussion on the ethical considerations of this work, especially since it touches the task of generative modelling of faces.\n\n[1]. Rethinking Generative Mode Coverage: A Pointwise Guaranteed Approach\n\n[2] . VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "1nL94wTTmf6", "reviewer": "Reviewer_phM4", "summary": "In this paper, the authors propose the maximization of the uniform diversity of the generative distribution as a means for avoiding mode collapse. The main idea relies on the diffeomorphism between input uniform distribution and the samples that can be obtained using a normalizing flow -based generator.  The key insight is due to this property being preserved, one can maximize the uniform diversity. \n\nThe paper provides useful theoretical and empirical analysis results that validate their work", "questions": "It will be useful if the authors could comment on the weaknesses mentioned above. ", "limitations": "The negative social impact are not particularly applicable.\n\nThe limitations to some extent are addressed in the supplementary material. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1) The use of uniform diversity as a measure to evaluate mode collapse in GANs is interesting. The ideas are also related to the Epsilon-Delta analysis of PacGAN\n2) Thorough evalution of their work by considering various existing techniques to avoid mode-collapse and showing that incorporating uniform diversity improves their performance\n3) Detailed theoretical analysis of their work with detailed results being provided in the supplementary material.\n\nWeakness:\n1) The main observation is that the existing uniform diversity can be achieved if one obtains the diffeomorphism between the distribution and the sample space. However, this exists only in cases such as normalizing flows. In general, normalizing flows have invertible properties but suffer from being inferior in terms of sample generation. It is not clear, how exactly this limitation is addressed in this work.\n2) As observed, the normalizing flows are limited in terms of the size of the latent space being exactly the same as the original sample space. The method adopted to solve this by zero-padding appears to be a heuristic. This needs to be better understood.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept