Assisting the Adversary to Improve GAN Training
Authors: Andreas Munk
William Harvey
Frank Wood
Venue:  (2021)
Keywords: Generative Adversarial Networks
GANs

Abstract: Some of the most popular methods for improving the stability and performance of GANs involve constraining or regularizing the discriminator. In this paper we consider a largely overlooked regularization technique which we refer to as the Adversary's Assistant (AdvAs). We motivate this using a different perspective to that of prior work. Specifically, we consider a common mismatch between theoretical analysis and practice: analysis often assumes that the discriminator reaches its optimum on each iteration. In practice, this is essentially never true, often leading to poor gradient estimates for the generator. To address this, AdvAs is a theoretically motivated penalty imposed on the generator based on the norm of the gradients used to train the discriminator. This encourages the generator to move towards points where the discriminator is optimal. We demonstrate the effect of applying AdvAs to several GAN objectives, datasets and network architectures. The results indicate a reduction in the mismatch between theory and practice and that AdvAs can lead to improvement of GAN training, as measured by FID scores.


Under review as a conference paper at ICLR 2021
ASSISTING THE ADVERSARY TO IMPROVE GAN TRAINING
Anonymous authors Paper under double-blind review
ABSTRACT
Some of the most popular methods for improving the stability and performance of GANs involve constraining or regularizing the discriminator. In this paper we consider a largely overlooked regularization technique which we refer to as the Adversary's s Assistant (AdvAs). We motivate this using different perspective to that of prior work. Specifically, we consider common mismatch between theo- retical analysis and practice; analysis often assumes that the discriminator reaches its optimum each iteration. In practice, this is essentially never true, often leading to poor gradient estimates for the generator. To address this, AdvAs is a theoretically motivated penalty imposed on the generator based on the norm of the gradients used to train the discriminator. This encourages the generator to move towards points where the discriminator is optimal. We demonstrate the effect of applying AdvAs to several GAN objectives, datasets and network architectures. The results indicate a reduction in the mismatch between theory and practice and that AdvAs can lead to improvement of GAN training, as measured by FID scores.
INTRODUCTION
The generative adversarial network (GAN) framework (Goodfellow et al., 2014) trains neural network known as a generator which maps from random vector an output such as an image. Key to training is another neural network, the adversary (sometimes called a discriminator or critic), which trained to distinguish between "true" and generated data. This is done by maximizing one of the many different objectives proposed in the literature; see for instance Goodfellow et al. (2014); Arjovsky et al. (2017); Nowozin et (2016). The generator directly competes against the adversary: it is trained to minimize the same objective, which it does by making the generated data more similar to the true data. GANs are efficient to sample from, requiring a single pass through a deep network, and highly flexible, they do not require explicit likelihood. They are especially suited to producing photo-realistic images (Zhou et al., 2019) compared to competing methods like normalizing flows, which impose strict requirements et al., 2020; Rezende & Mohamed, 2015) and VAES (Kingma & Welling, 2014; Razavi et al., 2019; Vahdat & Kautz, 2020). Counterbalancing their appealing properties, GANs can have unstable training dynamics (Kurach et al., 2019; Goodfellow, 2017; Kodali et al., 2017; Mescheder et al., 2018).
the neural network architecture (Kobyzev
Substantial research effort has been directed towards improving the training of GANs. These en- deavors can generally be divided into two camps, albeit with significant overlap. The first develops better learning objectives for the generator/adversary to minimize/maximize. These are designed to have properties which improve training (Arjovsky et al., 2017: Li et al., 2017: Nowozin et al., 2016). The other camp develops techniques to regularize the adversary and improve its training dynam- ies (Kodali et al., 2017; Roth et al., 2017; Miyato et al., 2018). The adversary can then provide a better learning signal for the generator. Despite these contributions, stabilizing the training of GANs remains unsolved and continues to be an active research area.
An overlooked approach is to train the generator in a way that accounts for the adversary not being trained to convergence. One such approach was introduced by Mescheder et al. (2017) and later built on by Nagarajan & Kolter (2017). The proposed method is regularization term based the norm of the gradients used to train the adversary. This is motivated as a means to improve the convergence properties of the minimax game. The purpose of this paper is provide new perspective to why

---

Under review as a conference paper at ICLR 2021
this regularizer is appropriate. Our perspective differs in that view it promoting updates that lead to a solution that satisfies a sufficient condition for the adversary to be optimal. To be precise, it encourages the generator move towards points where the adversary's 's current parameters are optimal. Informally, this regularizer "assists" the adversary, and for this reason we refer to this regularization method as the Adversary's Assistant (AdvAs).
We additionally propose a version of AdvAs which is hyperparameter-free. Furthermore, we release library which makes it simple integrate into existing code. We demonstrate its application to standard architecture with the WGAN-GP objective (Arjovsky et al., 2017; Guirajani et al., 2017); the otate-of-the-art StyleGAN2 architecture and objective introduced by Karras al. (2020); and the AutoGAN architecture and objective introduced by Gong et al. (2019), We test these on the MNIST (Lecun al., 1998), CelebA (Liu et al., 2015), CIFARTO (Krizhevsky et al., 2009) datasets respectively. We show that AdvAs improves training on all datasets, as measured by the Fréchet Inception Distance (FID) (Heusel et al., 2017), and the inception score (Salimans et al., 2016) where applicable.
2 BACKGROUND
A generator is a neural network g 2 + X' C R which maps from a random vector Z to an output x E X' (e.g.. an image). Due to the distribution over 2, the function g induces a distribution over its output = g(2). If g is invertible and differentiable, the probability density function (PDF) over x from this "change of variables" could be computed. This is not necessary for training GANs, meaning that no such restrictions need to be placed on the neural network g. We denote this distribution (a(x) where 0 E θ [ I denotes the generator's 's parameters. The GAN is trained on a dataset 2...... NN, where each x; is in X. We assume that this is sampled i.i.d. from a matta-genrattng distribution Ptrue- Then the aim of training is to learn 0 so that po is as close as possible to Ptrue- Section 2.1 will make precise what is meant by "close."
The adversary 0g X + AA has parameters 6 E + S Rd which are typically trained alternately with the generator. receives as input either the data or the generator's outputs. The set that it maps to, A, is dependent on the GAN type. For example, Goodfellow et al. (2014) define an adversary which maps from æ E X to the probability that æ is a "real" data point from the dataset, as opposed to a "fake" from the generator. They therefore choose A to be [0, 1] and train the adversary by maximizing the associated log- liglliiel objective,
h(po, a4) Eg-pusee [log as (x:) + Exzps [log(1 (o()))]
Using the intuition that the generator should generate samples that seem real and therefore "fool" the adversary, the generator is trained to minimize h(po, ag). Since we find @ to minimize this objective while fitting & to maximize it, training GAN is equivalent to solving the minimax game,
min max h hpp., as).
Eq. (1) gives the original form for h(po, as)) used by Goodfellow al. (2014) but this form varies between different GANs, as we will discuss in Section 2.1. The minimization and maximization in Eq. (2) are performed with gradient descent in practice. To be precise, we define Tigen(( (0, Φ) h(pp. a.)) and Ladv (0, Φ) - -hipp....
These are treated as losses for the generator and adversary respectively, and both are minimized. In other words, we turn the maximization of h(pa, az) w.r.t. ø into a minimization of (e, Φ). Then on each iteration, 0 and are updated one after the other using gradient descent steps along their respective gradients:
V Legon (0. Φ) V eellpe as), de Land (0, = V Voh(po, ao).
(3)
(4)
2.1 GANS MINIMIZE DIVERGENCES
A common theme in the GAN literature is analysis based on what we call the optimal adversary assumption. This is the assumption that, before each generator update, we have found the adversary as which maximizes h h(pg, as) given the current value of 9. To be precise, we define a class of permissible adversary functions F. This is often simply the space of all functions mapping X A
2

---

Under review as a conference paper at ICLR 2021
(Goodfellow et al., 2014), but is in some GAN variants constrained by, e.g., a Lipschitz constant (Ar- jovsky et al., 2017). Then we call the adversary as optimal for a particular value of # if and only h(po; ap) m a).
In practice, the neural network as cannot represent every a E F and so it may not be able to parameterize optimal adversary for a given 8. As is common in the literature, we assume that the neural network is expressive enough that this is not an issue, i.e., we assume that for any 8, there exists least one & & resulting in an optimal adversary. Then, noting that there may be multiple such & E Q, we define $- (0) to be the set of all optimal adversary parameters. That is, (0) {o E $ h(pe, a3)) m h(pp. a)} and the optimal adversary assumptions says that before each update of 6 we have found & E (0). We emphasize that, in part due to the limited number of gradient updates performed on Φ, this assumption is essentially never true in practice. This paper presents a method to improve the training of GANs by addressing this issue.
The optimal adversary assumption simplifies analysis of GAN training considerably. Instead of being a two-player game, it turns into case of minimizing objective with respect to 0 alone. We denote this objective
M(po) = max h(po, a) = h(po, ag)) where x- E (0).
(5)
a
For example, Goodfellow et al. (2014) showed that using the objective presented in Eq. (1) results in M(pa) = 2 JSD |pe) ) log 4, where JSD is the Jensen Shannon divergence. By making the optimal adversary assumption. they could prove that their GAN training procedure would converge, and would minimize the Jensen-Shannon divergence between Ptrue and po-
A spate of research following the introduction of the original GAN objective has similarly made use of the optimal adversary assumption to propose GANs which minimize different divergences. For example, Wasserstein GANs (WGANs) (Arjovsky et al., 2017) minimize a Wasserstein dis- tance. MMD GANs (Li et al., 2017) minimize a distance known the maximum mean discrep- ancy. Nowozin et al. (2016) introduce f-GANs which minimize f-divergences, a class including the Mullback-Leibler and -aasnnnShannon divergences. We emphasize that this is by means haustive list. Like these studies, this paper is motivated by the perspective that, under the optimal adversary assumption, GANs minimize divergence. However, the GAN framework can also be viewed from a more game-theoretic perspective (Kodali et al., 2017; Grnarvv et al., 2018)
3 DOES AN OPTIMAL ADVERSARY LEAD TO OPTIMAL GRADIENTS?
As introduced above, the training of an adversary does not need be considered in any analysis if it is simply assumed to always be optimal. From this perspective, the goal of training GANs can be seen as learning the generator to minimize M(p)). This leads the question: assuming that we have an optimal adversary, can we compute the gradient required for the generator update, VoM(ps)) ? To clarify, assume that we have generator parameters 6. and have found Φ" E 5 (8') such that h(per, h(per.a.)) and M(per) are equal in value. We then want take a gradient step on 0' to minimize M. Virtually all GAN methods do this by assuming that M(prr) and h(pw, aa-) have equal gradients with respect to 0 at 0', That is, it is assumed that VeM(ps) le= is equal to the partial derivative' Dyh(pe, as-) 10=9r. It is not immediately obvious that this is true.
In the GAN literature this concern has largely been overlooked, with few treatments for spe- cific GAN types, see e.g. Arjovsky et al. (2017); Goodfellow et al. (2014). In particular, Arjovsky et al. (2017) invoke (but do not explicitly prove) an extension of Theorem 1 in Milgrom & Segal (2002) to prove that the Wasserstein GAN has optimal gradients the adversary is optimal, iee. Dih(po, a |-=-'= VoM(pe) le-er. We note that this extension can, in fact, be used to prove that GANS in general have this property under fairly weak assumptions:
Theorem 1. Let M(pa) = hipo, ag.) ) for any o' E xx (0), as defined in Eq. (5). Assuming that ( is differentiable w.rt. O and hipp, 0s) is differentiable w.rt. for all @ (0),,, then if oe E (0) we have
vaM(po) = Dyh(po, ag.) ).
(6)
We use D2 Dih(po, as) to denote the partial derivative of h(po, as) with respect to 0 with kept constant. Similarly, we will Dyh(po, aa) to denote the derivative of hipo, as) with respect to o with @ held constant.
3

---

Under review as a conference paper at ICLR 2021
See Appendix D.I for proof. We emphasize Theorem I applies only if the adversary is optimal. If this is not the case we cannot quantify, and so cannot directly minimize or account for, the dis- crepancy between VoM(po) and Di h(po, ag-). Instead of attempting to do consider an approach that drives the parameters towards regions where o* E (0) so that Theorem 1 can be invoked.
3.1
ADVERSARY CONSTRUCTORS
To see how we may impose the constraint that Eq. (6) is true, we consider a trivial relationship between any generator and the corresponding optimal adversary. If an optimal adversary exists for every E O then there exists some, possibly non-unique, function ʃ + + that maps from any generator
corresponding optimal adversary. That is, for all 0 θ, f(0) o* E (0) in which case h (p9, (f(o)) = m h(pe, a). We refer to such a function as an adversary con- structor. In an ideal scenario, we could compute the output of an adversary constructor, f(0), for any 0. We could then invoke Theorem 1 and the generator could be updated with the gradient VgM(pe) = Drh(pe a (())). In practice, computing (()) is infeasible and we can only approxi- mate the optimal adversary parameters with gradient descent. There therefore a mismatch between GAN theory, where Theorem 1 is often invoked, and practice, where the conditions to invoke it are essentially never satisfied. How then, can we address this problem? We look to the adversary con- structors, which provide condition that must be satisfied for the optimal adversary assumption to be true. Adversary constructors allow us to account for the influence total derivative Veh(pe, (()))- We prove in Appendix D.2 that comparison with the result of Theorem 1 leads to Corollary 1. In the next section, we motivate AdvAs as an attempt to fulfill a condition suggested by this corollary.
on é by considering the
Corollary 1. Let f ④ + be a differentiable mapping such that for all 0 O, MMpp) h(po. a.)))) ) If the conditions in Theorem I are satisfied and the Jacobian matrix of f with respect to 0, Jo( exists for all 0 E θ then
Dyh(po, a(1))) " Jf() = =0.
4 ASSISTING THE ADVERSARY
Corollary 1 tells us that D2h(po, ao) Jo will be zero whenever Theorem can be invoked. This makes Eq. (7) necessary. but not sufficient, condition for the invocation of Theorem 1. This suggests that the magnitude of Dyh(ps, Jo could be a measure of how "close" Dyh(po, a is to the desired gradient VaM(pa). However, the Jacobian Ja(J) is not tractable 80 D2h(po, az) )T Jo(f) cannot be computed. The only term we can calculate in practice is D2h(po, aa). exactly the gradient used to train the adversary. If D2h(po, aş) is zero, then Dyh(po, TO)TToo (f) is zero. The magnitude of Dyh as) could therefore be an approximate measure of "closeness" instead of (ph a.) aT) Je(f). This leads to an augmented generator loss, which regularizes generator updates to reduce the magnitude of D2h az). has a scalar hy- perparameter A= 2 0, but Section 4.3 provides a heuristic which can remove the need to set this hyperparameter.
L LAdvAs(0, (0, 6) Lgen (0, () (r) gen
(8)
with
r(0 x) = V a La (0, 0112
(9)
recalling that V Ladv - DDhh(po ag). We emphasize that r(8, Φ) is the same as that found in previous work (Mescheder et al., 2017; Nagarajan & Kolter, 2017).
Figuratively, AdvAs changes the generator updates to move in conservative direction that does not over-exploit the adversary's 's sub-optimality. Consider the generator and adversary as two players attempting to out-maneuver one another. From Eq. (2), we see that the generator should learn to counteract the best possible adversary, rather than the current adversary. If the current adversary is sub-optimal, allowing it to catch up would yield better updates to the generator. One way to achieve this is to update the generator in a way that helps make the current adversary optimal. This behavior is exactly what AdvAs encourages. In this sense, it assists the adversary, leading it's name, the Adversary's Assistant. We emphasize that using AdvAs involves making only a small modification to GAN training algorithm, but for completeness we include pseudocode in Appendix A.
4

---

Under review as a conference paper at ICLR 2021
15
Badv/Reg Reg type A/AdvAs 1/ Baseline 5/ Baseline
5
10000
20000
30000
40000
0.5
1.0
1.5
2.0
Iterations
Time (h)
Figure/ 1: FID scores throughout training for the WGAN-GP objective MNIST, estimated using 60000 samples from the generator. We plot up to a maximum of 000 iterations. When plotting against time (right), this means some lines end before the two hours show. The blue line shows the results with AdvAs, while the others are baselines with different values of Nadv-
4.1 ADVAS PRESERVES CONVERGENCE RESULTS
AdvAs has several desirable properties which support its use as a regularizer: (1) it does not interfere with the update on Φ and recall that perfectly optimizing leads to h(po, ag) ((p)). (2) Under mild conditions, vr( Φ) ld=ø- is zero for an optimal o² and so L x*) V MPM))
These properties imply that, under the optimal adversary assumption, optimizing ggon equivalent to optimizing Lygen See Appendix D.4 for proof. Therefore any convergence analysis which relies on the optimal adversary assumption is equally applicable when AdvAs is included in property (2), require that ø² be stationary point
As is fact
the loss. Regarding the mild conditions of h(pe, az). This is true as long as h is differentiable w.r.t. ó at (²)" and ø" does not lie on boundary of 4. The optimal adversary parameters, o², cannot lie on boundary unless, for example, weight clipping is used as in Arjovsky et al. (2017). In such cases, we cannot speak to the efficacy of applying AdvAs.
We make the additional observation that for some GAN objectives, minimizing r(f. Φ) alone (as Adv may match Pa and Ptrue- We show this in Appendix D.3 for the WGAN objective (Arjovsky al., 2017). particular, for all $ r(0, Φ) is zero and at global minimum whenever Pe Ptrue- Experimental results in Appendix C.3 support this observation. However, the results appear worse than those obtained by optimizing either Tggn or I I,AdAAs,
opposed to Leen or gen
'gen
4.2 ESTIMATING THE ADVAS LOSS
It is not always possible, and seldom computationally feasible, compute the AdvAs regularization term r(0, Φ) exactly. We instead use stochastic estimate. This computed by simply estimating the gradient V (0, Φ) with minibatch and then taking the squared L2-norm of this gradient estimate. That is, defining Ienav (0, () as an unbiased estimate of the adversary's loss, we estimate r(0, ()) with
F(0,
V V 7. (0, ())
(10)
Although the gradient estimate is unbiased, taking the norm results in biased estimate of r(0, Φ)- However, comparisons with a more computationally expensive unbiased estimate" did not reveal significant difference in performance.
"Computing an unbiased estimate can be done using following: consider two independent and unbiased
estimates of V. T. (0, Φ) denoted X, X'. Then T. T x'] R. [x]" [X] || Vo (0, d) This implies that multiplying two estimates using independent samples unbiased.
5

---

Under review as a conference paper at ICLR 2021
4.3 REMOVING THE HYPERPARAMETER
Eq. (8) introduces a hyperparameter, A, which we would prefer not to perform a grid-search on. Setting À to be too great can destabilize training. Conversely, setting it to be too small gives similar results to not using AdvAs. We therefore introduce a heuristic which can be used to avoid setting the hyperparameter. Our experiments suggests that this is often a good choice, although manually tuning À may yield greater gains. This heuristic involves consider- ing the magnitudes of three gradients, and so we first define the notation.
Table FID and IS scores on CIFARIO using Au- toGAN with and without AdvAs.
IS =
FID to
AdvAs (Radv 2) Baseline (Nadv = 5)
8.4 0.1 0.1
14.5 + 1.0
15.0 = 0.7
Tuud muav/RRe type
1/AdvAs
2/AdvAs
100
1/Baseline
FID
gorig (0, Φ) V o goggn o), gAdvAs (0, Φ) = Voi Φ),
2/ 2/Basoline
5/ B/Baseliue
Giotal (9, Φ, A) = VoLgen (0.
= gorig Φ) + AgAdvAs(0, o).
The heuristic can be interpreted choosing À at each iteration to prevent the total gradient, (O, Φ, A) from being dominated by the AdvAs term. Specifically, we ensure the mag- nitude of AgAdvAs (0, Φ) less than or equal to the magnitude of gorig by setting
2
4
6
Time (h)
FID scores on CIFARIO using Auto- GAN from baselines and AdvAs plotted with a log y-axis against running time for different val- ues of Nadv. We see that AdvAs with nadv = (11) yields the lowest FID scores at every point during training.
Figure 2:
Mgrig (0, 0)2
A min (".TAAvssc( 1)12
at every iteration. We then perform gradient descent along I1, (0, o, A). This technique ensures that À is bounded above by 1.
EXPERIMENTS
We demonstrate the effect of incorporating AdvAs into GAN training using several GAN architec- tures, objectives, and datasets. Our experiments complement those of Nagarajan & Kolter (2017) and Mescheder et al. (2017). In each case, we compare GANs trained with AdvAs with baselines that do not use AdvAs but are otherwise identical. We first demonstrate the use of AdvAs in conjunc- tion with the WGAN-GP objective (Gulrajani et al., 2017) to model MNIST (Lecun et al., 1998). In this experiment, we compare the performance gains achieved by AdvAs reasonable upper bound on the gains achievable with this type of regularization. We further support these findings with experiments on CIFARIO (Krizhevsky et al., 2009) using AutoGAN (Gong al., 2019), an architecture found through neural architecture search. We then demonstrate that AdvAs can im- prove training on larger images using StyleGAN2 (Karras al., 2020) on CelebA (Liu al., 2015). We quantify each network's 's progress throughout training using the FID score (Heusel et al., 2017). Since AdvAs increases the computation time per iteration, we plot training progress against time for each experiment. We also present inception scores (IS) (Salimans et al., 2016) where applicable. We estimate scores in each case with random seeds and report the standard deviation (a) as a measure of uncertainty.
AdvAs aims to improve performance by coming closer to having an optimal adversary. Another common way to achieve this is to use larger number of adversary updates (Radv) before each generator update. For each experiment, we show baselines with the value of Radv suggested in the literature. Noting that the computational complexity is O(neav) and keeping nadv low is desirable, we find that AdvAs can work well with lower values of nadv than the baseline. For a fair comparison, we also report baselines trained with these values of nadv-

---

Under review as a conference paper at ICLR 2021
baseline
AdvAs
baseline
AdvAs
200
AdvAs
Baseline
FID
150
100
20
40
60
50
100
Epochs
Time (h)
Figure 3: Bottom: FID scores throughout training estimated with 1000 samples, plotted against number of epochs (left) and training time (right). FID scores for AdvAs decrease more each iteration at the start training and converge to be 7.5% lower. Top: The left two columns show uncurated samples with and without AdvAs after 2 epochs. The rightmost two columns show uncu- rated samples from networks at the end of training. In each grid of images, each row is generated by network with a different training seed and shows 3 images generated by passing different random vector through this network. AdvAs leads to obvious qualitative improvement early in training.
For MNIST and CelebA, we avoid setting the hyperparameter by using the heuristic proposed in Section 4.3. We found for CIFAR10 that manually tuning À gave better performance, and so set À 0.01. Additionally, on MNIST and CelebA,
methods we consider use regularization in the form of gradient penalty (Gulrajani et al., 2017) for training the adversary. This is equivalent to including a regularization term Yadv (6) in the definition of Ladv. That is, Ladv (0, Φ) = V Vah(po. a²) + Tadv( Following Eq. (9) this regularization term is included in the AdvAs term r(0, o). Another practical detail is that AutoGAN and StyleGAN2 are trained with a hinge loss (Lim & Ye, 2017).
That is, when computing the adversary's 's loss Ludv (6, Φ), its output ag ax(xx) is truncated to be below +1 for real images, or above -1 1 for generated images. This prevents it from receiving gradient feedback when its predictions are both accurate and confident. Intuitively, this stops its outputs becoming too large and damaging the generator's ' training. However, this truncation is not present when updating the generator. This means that the generator minimizes a different objective the one maximized by the adversary, and so it is not exactly a minimax game. It is not clear that it is beneficial to calculate the As regularization term using this truncation. We found that better performance was obtained by computing r(0, Φ) without truncation, and do this in the reported experiments.
5.1 WGAN-GP ON MNIST
We use simple neural architecture: the generator consists of a fully-connected layer followed by two transposed convolutions. The adversary has three convolutional layers. Both use instance normalization (Ulyanov et al., 2017) and ReLU non-lincarities; see Appendix E for details. We compare using AdvAs with Badv = 1 against the baseline for Nadv E {1,5) where Nadvv = is suggested by Gulrajani et al. (2017). Fig. shows the FID scores for each method throughout training. We see that using AdvAs with nadvv = leads to better performance on convergence; even compared to the baseline with nadv = 5. the best FID score reached is improved by 28%.
5.2 AUTOGAN ON CIFAR10
We next experiment the generation of CIFAR10 (Krizhevsky et al., 2009) images. We use Auto- GAN (Gong et al., 2019), which has a generator architecture optimized for CIFARIO using neural architecture search. It is trained with hinge loss, as described previously, exponential moving

---

Under review as a conference paper at ICLR 2021
average of generator weights, and typically uses nadv 5. Figure 2 shows FID scores throughout training for various values of nadv, with and without AdvAs, each computed with 1000 samples. Ta- ble 1 shows FID scores at the end of training for the best performing value of Radv for each method, estimated with 50 000 samples. For a fixed aadv of either or 2, using AdvAs improves the FII) score. In fact, with Madv = 2, the performance with Adv is indistinguishable from the baseline with the suggested setting of nadv = 5. Unlike for MNIST, AdvAs does not outperform the baseline with high enough Aadv. We hypothesize that this is because, with an architecture highly optimized for Radv = 5, the adversary is closer to being optimal when trained with Radv = 5. Assuming this the case, we would not expect AdvAs improve training compared to a baseline with sufficient Madvy Still, our results show that applying AdvAs allows the same performance with lower aadv
5.3 STYLEGAN2 ON CELEBA
To demonstrate that AdvAs improves ofte-of-thheart GAN architectures and training procedures, we consider StyleGAN2 (Karras et al., 2020). We train this as proposed by Karras et al. (2020) with WGAN-like objective with gradient penalty (Gulrajani et al., 2017), an exponential moving average of the generator weights, and various forms of regularization including path length, R1, and style-mixing regularization. More detail on these can be found in Karras al. (2020), but we merely wish to emphasize that considerable effort has been put into tuning this training procedure. For this reason, we do not attempt further tune Radv, which is 1 by default. Any improvements from applying AdvAs indicate a beneficial effect not provided by other forms of regularization used.
Figure 3 compares the training of StyleGAN2 on CelebA at 64x64 resolution with and without the AdvAs regularizer. Using AdvAs has two main effects: (1) the generated images show bigger improvements per epoch at the start of training: and (2) the final FII) score is improved by 7.5%. Even accounting for its greater time per iteration, the FID scores achieved by AdvAs overtake the baseline after one day of training. We verify that the baseline performance similar to that reported by Zhou et al. (2019) with a similar architecture.
RELATED WORK
We motivated AdvAs from the perspective of the optimal adversary assumption. this sense, is similar to large body of work aiming to improve and stabilize GAN training by better training the adversary. AdvAs differs fundamentally due to its focus on the training the generator rather than the adversary. This other work generally affects the discriminator in one of two broad ways: weight constraints and gradient penalties Brock et al. (2019). Weight normalization involves directly manipulating the parameters of the adversary, such as through weight clipping (Arjovsky et al., 2017) or spectral normalization (Miyato et al., 2018). Gradient penalties (Kodali et al., 2017; Roth al., 2017; Guirajani al., 2017) impose soft constraints on the gradients of the adversary's output with respect to its input. Various forms exist with different motivations; see Mescheder et al. (2018) for a summary and analysis. AdvAs may appear similar to gradient penalty, as it operates on gradients of the adversary. However, the gradients are w.r.t. the ad adversary's 's parameters rather than its input. Furthermore, AdvAs added to the generator's loss and not the adversary's.
Regularizing generator updates has recently received more attention in the literature (Chu et al., 2020: Zhang et al., 2019; Brock et al., 2019). Chu et al. (2020) show theoretically that the ef- fectiveness of different forms of regularization for both the generator and adversary is linked to the smoothness of the objective function. They present a set of conditions on the generator and adversary that ensure smooth objective function, which they argue will stabilize GAN training. However, they leave the imposition of the required regularization on the generator to future work. Zhang al. (2019) and Brock et al. (2019) consider applying spectral normalization (Miyato al., 2018) to the generator, and find empirically that this improves performance.
7 DISCUSSION AND CONCLUSIONS
We have shown that AdvAs addresses the mismatch between theory, where the adversary is assumed to be trained to optimality, and practice, where this is never the case. We show improved training across three datasets, architectures, and GAN objectives, indicating that it successfully reduces this
8

---

Under review as a conference paper at ICLR 2021
disparity. This lead to substantial improvements in final performance. We note that, while applying AdvAs in preliminary experiments with BEGAN (Berthelot et al., 2017) and LSGAN (Mao et al., 2017), we did not observe either significant positive effect, significant negative effect other than the increased time per iteration. Nevertheless, AdvAs is simple to apply and will, in many cases, improve both training speed and final performance.
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, pp. 214-223, July 2017.
David Berthelot, Thomas Schumm, and Luke Metz. Began: Boundary equilibrium generative ad- versarial networks. arXiv: (rXiv:/733.1077 [cs, stat], May 2017.
Dimitri P Bertsekas. Nonlinear Programming. volume 48 of Journal of the Operational Research Society. Taylor & Francis, 1997.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv: 1809. 1096 [cs, stat), February 2019.
Casey Chu, Kentaro Minami, and Kenji Fukumizu. Smoothness and stability in gans. In Eighth International Conference on Learning Representations, April 2020.
Xinyu Gong, Shiyu Chang, Yifan Jiang, and Zhangyang Wang. Autogan: Neural architecture search for generative adversarial networks. In Proceedings of the IEEF/CVE International Conference Computer Vision, pp. 3224-3234, 2019.
Ian Goodfellow. Nips 2016 tutorial: Generative adversarial networks, arXiv:1701.00160 [cs]. April 2017.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Infor- mation Processing Systems 27, pp. 2672-2680. Curran Associates, Inc., 2014.
Paulina Grnarova, Kfir Y. Levy, Aurelien Lucchi, Thomas Hofmann, and Andreas Krause. An online learning approach generative adversarial networks. In International Conference on Learning Representations, February 2018.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of wasserstein gans. In Advances in Neural Information Processing Systems 30, pp. 5767-5777. Curran Associates, Inc., 2017.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local nash equilibrium. In Advances in Neural Information Processing Systems 30, pp. 6626-6637. Curran Associates, Inc., 2017.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyz- ing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8110 8110-8119, 2020.
Diederik Kingma and Max Welling. Auto-encoding variational bayes. arXiv: 14 [cs, stat), May 2014.
Ivan Kobyzev, Simon Prince, and Marcus Brubaker. Normalizing flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2020. ISSN 1939-3539. doi: 10. 1019/T9PMM.20000
Naveen Kodali, Jacob Abernethy, James Hays, and Zsolt Kira. On convergence and stability of gans. 1rXiv:1705.07215 [es], December 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning Multiple Layers of Features from Tiny Images. Citeseer, 2009.

---

Under review as a conference paper at ICLR 2021
Karol Kurach, Mario Lučié, Xiaohua Zhai, Marcin Michalski, and Sylvain Gelly. A large-scale study on regularization and normalization in gans. In International Conference on Machine Learning, pp. 3581-3590. PMLR, May 2019.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recog- nition. Proceedings of the IEEE, (6(11):2278-2324 November 1998. ISSN 1558-2256. doi: N0.1109/5.72671.
Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabas Poczos. Mmd gan: Towards deeper understanding of moment matching network. In Advances in Neural Information Processing Systems 30, pp. 2203-2213, Curran Associates, Inc., 2017.
Yujia Li, Kevin Swersky, and Rich Zemel. Generative moment matching networks. In International Conference on Machine Learning, pp. 1718-1727, 2015.
Jae Hyun Lim and Jong Chul Ye. Geometric gan. arXiv:1705.0285 /cond-mat, stat], May 2017.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In 2015 IEEE International Conference on Computer Vision (ICCV), pp. 3730-3738, December 2015. doi: 10.1109/ICCV.2011 2015.425.
Xudong Mao, Qing Li, Haoran Xie, Raymond Y. K. Lau, Zhen Wang, and Stephen Paul Smol- ley. Least squares generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, pp. 2794-2802, 2017.
Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. The numerics of gans, Advances in Neural Information Processing Systems, 30: 10:1525-1835 2017.
Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? International Conference on Machine Learning, pp. 3481-3490. PMLR, July 2018
Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. In International Conference on Learning Representations, 2017.
Paul Milgrom and Ilya Segal. Envelope theorems for arbitrary choice sets. Econometrica, 70(2): 583-601, 2002. ISSN 1468-0262. doi: 10.1111/1468-024 020--002220026666
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In International Conference on Learning Representations, February 2018.
Vaishnavh Nagarajan and Zico Kolter. Gradient descent gan optimization is locally stable, Ad- vances in Neural Information Processing Systems, 30:5585-5595, 2017.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. F-gan: Training generative neural sam- plers using variational divergence minimization. In Advances in Neural Information Processing Systems 29, pp. 271-279. Curran Associates, Inc., 2016.
Ali Razavi, Aaron van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with v.- In Advances in Neural Information Processing Systems 32, pp. 14866-148777 Curran Associates, Inc., 2019.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In Interna- lional Conference on Machine Learning, pp. 1530-1538. PMLR, June 2015.
Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, and Thomas Hofmann. Stabilizing training of generative adversarial networks through regularization. In Advances in Neural Information Pro- cessing Systems 30, pp. 2018-2028. Curran Associates, Inc., 2017.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. Improved techniques for training gans. Advances in Neural Information Processing Systems 29, pp. 2234 2242. Curran Associates, Inc., 2016.
10

---

Under review as a conference paper at ICLR 2021
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing in- gredient for fast stylization. arXiv:1607.08022 [cs], November 2017.
Thomas Unterthiner, Bernhard Nessler, Calvin Seward, Günter Klambauer, Martin Heusel, Hubert Ramsauer, and Sepp Hochreiter. Coulomb gans: Provably optimal nash equilibria via potential fields. arXiv preprinl urXivv/7088 07879, 2017.
Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. arXiv:2007.03898 [cs, stat), July 2020.
Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. Self-attention generative adversarial networks. volume 97 of Proceedings of Machine Learning Research, pp. 7354-3363 Long Beach, California, USA, June 2019. PMLR.
Sharon Zhou, Mitchell Gordon, Ranjay Krishna, Austin Narcomey, Fei-Fei, and Michael Bern- stein. Hype: A benchmark for human eye perceptual evaluation of generative models, In Advances in Neural Information Processing Systems, 3449-3461, 2019.


Reviews:
[{"review_id": "vySURmuERV7", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper concerns how to efficiently regularize the generatior for training generative adversarial networks (GANs). A new regularizer for the generator loss is proposed to penalize the norm of the gradient with respect to discriminator’s parameters ($\\phi$). In other words, the generator learns to encourage small norm of the discriminator’s gradient w.r.t $\\phi$. The author(s) also propose a heuristic to remove the introduction of a further hyperparameter. The author(s) applied the proposed regularizer to WGAN-GP, AutoGAN, and StyleGAN2 to validate its effectiveness. Their experiments reveal that the new regularizer is promising.\n\nPros:\n- The idea of penalizing the gradient w.r.t discriminator’s parameters when learning the generator is interesting. It could be great if the authors could explain more about its implications/meaning.\n- Different formulations and architectures for GAN have been taken into account for investigation. \n\nCons:\n- There exist some works that propose regularizers to both generator and discriminator, including [Mescheder et al., 2017; Nie & Patel, 2019]. Those regularizers are based on gradient vector fields over the parameters of both discriminator and generator. However, this paper does not provide any discussion about those closely related works and hence places itself in an unclear context.\n- Some recent studies [Brock et al., 2019; Zhang et al., 2019] found that regularizing the generator, e.g., using spectral normalization (SN), can help improve training and quality of the generator. SN is a very efficient way to do regularization and can make GAN training much more stable. Therefore, the idea of regularization for generator is not new.\n- Since this paper proposes a penalty when training GAN, I expect to see comparisons with some other regularization methods, e.g, SN and gradient penalty. Unfortunately, such a careful comparison is missing. Instead the author(s) used AutoGAN and StyleGAN2, which focus on generator architecture, for investigation. Such experiments do not help much to understand the effectiveness of the proposed regularizer, in comparison with some other approaches. As a result, the significance of this work is unclear.\n\nMinor comments:\n- An investigation about the sensitivity of parameter $\\lambda$ should be done. It could help us see whether or not the proposed heuristic is effective.\n- One motivation of this work is the optimality of the discriminator. However, it is wellknown that the gradient vanishing problem may appear if one train the discriminator to optimality. Therefore the motivation seems not to be supportive.\n- Formular (1) seems to be wrong. \n- “advAs as an attempt fulfill” in page 4 --> “advAs as an attempt to fulfill”\n\nReference:\n- Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2019. \n- Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. The numerics of gans. In Advances in Neural Information Processing Systems, pp. 1825–1835, 2017. \n- Weili Nie and Ankit Patel. Towards a better understanding and regularization of gan training dy- namics. In Conference on Uncertainty in Artificial Intelligence (UAI), 2019. \n- Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. Self-attention generative adversarial networks. In International Conference on Machine Learning, pp. 7354–7363, 2019. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "vUFWPEcVtf9", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary of contributions:**\nThis paper propose a new regularization for training the generator in GANs. They argue that when the discriminator is not optimal this encourages the generator to stay in region where the discriminator is close to optimal. They then propose an heuristic to adaptively control the coefficient of the regularization term. Finally they show empirically that the proposed regularization can indeed improve the generator performance in several setting.\n\n**Major Concern:**\nI have a big concern about the contribution of this work, the proposed regularization eq 8 is exactly the regularization also proposed in [[1] Gradient descent GAN optimization is locally stable (NeurIPS 2017)](http://papers.nips.cc/paper/7142-gradient-descent-gan-optimization-is-locally-stable) see eq 4.\nThis related work is not even mentioned in this paper, also while the perspective is a bit different here, I find the theoretical motivation more rigorous in [1].\n\n**Clarity:**\nThe paper is quite clear and well written.\n\n**Significance:**\nThe experiments gives a good overview of the performance of the proposed method, however the results don't always show a major advantage of the proposed regularization for example on CIFAR there doesn't seem to be any benefit. Also this should be compared to other regularization technique for the generator for example Jacobian Clamping proposed in [Is Generator Conditioning Causally Related to GAN Performance? (ICML 2018)](http://proceedings.mlr.press/v80/odena18a.html).\nIt would also be interesting to have a plot showing the evolution of the adaptative regularization coefficient along training.\n\n**Other comments:**\nThere is actually a cheap and unbiased estimator for the AdvAs loss by using this property: $||\\mathbb{E}[X]||^2 = \\mathbb{E}[X_i^TX_j]$ where $X_i$ and $X_j$ are two independent random variables sampled from the same distribution as X. For an example on such an unbiased estimator in a context related to yours see Appendix C of [Stochastic Hamiltonian Gradient Methods for Smooth Games  (ICML 2020)](https://proceedings.icml.cc/static/paper_files/icml/2020/6356-Paper.pdf).\n\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "l83gld8C01", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new regularizer to improve GAN training. By noticing that the discriminator does not always reach optimum at each iteration, this paper proposes Adversary's Assistant (AdvAs) for helping the discriminator to satisfy this condition. Interestingly, compared to the previous methods for improving GAN training, this work applies the regularizer at the generator (rather than the discriminator) and is theoretical motivated. Experiments on several GAN objectives, datasets and network architectures are provided to support the effectiveness of AdvAs.\n\n\n*Pros\n\n(1) This paper is clearly written. Even I am not an expert in GAN, I do not encounter too many difficulties in understanding the whole paper.\n\n(2) The whole framework is theoretical motivated. Given that the discriminator is not always at an optimal point during training, this paper derives several theorems and corollaries, which leads to the finding that adding a regularization on the generator could satisfy a necessary condition for training GAN optimally.\n\n(3) Empirical results are provided to show the proposed AdvAs can help GAN training under different settings.\n\n\n\n*Cons\n\n(1) This paper uses a minibatch to approximately compute the regulizer $r(\\theta,\\phi)$. I am wondering if the proposed AdvAs is sensitive to the estimation quality of $r(\\theta,\\phi)$? For example, if large batch size is used, will the results be better? If yes, then what is the \"minimal\" batch size to train a good GAN with AdvAs (i.e., outperforms the baseline)?\n\n(2) I appreciate that this paper honestly states that the proposed AdvAs cannot help BEGAN and LSGAN. I encourage the authors to delve deeper into this observed phenomenon and provide a brief discussion on explaining the possible reasons why AdvAs cannot help here.\n\n(3) As the main purpose of AdvAs is to encourage the value of Eq. (7) be close to 0, the authors are encouraged to also plot the value of $D_2h(p_θ, a_φ)$ during the training, as direct evidence for supporting the effectiveness of AdvAs.\n\n\n\n**Overall, I think it is an interesting paper, with good theoretical motivation and strong empirical results, therefore I tend to accept it at this time. Nonetheless, I am not an expert in GAN and cannot accurately access the value/importance of this paper. I am open to increase/decrease my score if other expert reviewers provide any positive/negative comments.\n\n\n\n ", "extra_scores": {}, "extra_text": {}}, {"review_id": "dZdPaNTbNxZ", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThe authors observe that many GAN forumations assume optimality of the discriminator at each generator step and clarify this by reviewing a variant of an envelope theorem.\nObserving that this optimality condition isn't usually satisfied in practice, the authors propose a generator regularizer term which encourages the discriminator to be approximately optimal by penalizing the norm of the discriminator's gradient vector.\nExperiments on MNIST, CIFAR10, and CelebA show that use of the regularizer often yields improvement wrt baselines.\n\nReview:\n\nIn summary, the paper's main strength is clarity of exposition; its main weaknesses are that the algorithm isn't novel (see below) and that the experimental results are limited. Overall I can't recommend acceptance in the current form.\n\nCorrectness:\n\nThe motivation, theory, and algorithms proposed are broadly correct to my knowledge.\nI do want to point out that it's possible to obtain an unbiased estimate of $|| \\partial_\\theta L ||_2^2$ efficiently by computing two independent estimates of the discriminator gradient vector (i.e. using two independent minibatches) and multiplying them. This owes to the fact that $E[X]^2 = E[X]E[X'] = E[X X']$ when $X, X'$ are independent and identically distributed.\n\nNovelty:\n\nThe observation that GAN theory sometimes relies on an envelope theorem, and the specific envelope theorem given, aren't particularly novel (for instance, the original GAN paper by Goodfellow made a version of this argument), but they are (in my opinion) under-appreciated in existing literature and so the extra exposition is welcome.\n\nThe authors point out that while superficially similar to other gradient penalties, it has a very different motivation, structure, and probably works in a different way; I generally agree with their argument.\nHowever, the proposed regularizer does have exactly the same form (and similar motivation) as the one proposed in https://arxiv.org/abs/1706.04156, so the algorithm isn't novel in the end.\n\nThe proposed algorithm is also closely related to https://arxiv.org/abs/1705.10461 .\n\nIf this paper is to be accepted, at a minimum it needs a full discussion of the algorithms given in these two works and how they relate to the present work.\n\nSignificance:\n\nEmpirically the experiments support the usefulness of the proposed algorithm, albeit not to a very great degree.\nThe baselines compared against aren't state-of-the-art.\n\nOne experimental weakness is that in many places the authors claim a reduction in discriminator-steps-per-generator-step as an improvement, but don't separately search over generator/discriminator learning rates in the baseline.\nIf you want to make this kind of claim, the appropriate baseline is a GAN in which the G and D LRs have been tuned (and tuned independently of each other).\nOtherwise, for example, a trivial \"penalty\" term which just equals the original loss (and hence has the effect of multiplying the generator gradient by two, which is equivalent to multiplying the learning rate by two for SGD) might show improvement.\n\nClarity:\n\nThe paper deserves high marks on clarity in my opinion. It reads easily and contextualizes its work well. The claims made are precise and well-scoped. \nClarity of the exposition is important because it means this paper advances our understanding despite the somewhat limited empirical improvement from the algorithm.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper received low ratings and the reviewers pointed out a number of issues. The authors' short response failed to address these concerns. 
Decision: Reject