DO-GAN: A Double Oracle Framework for Generative Adversarial Networks
Authors: Aye Phyu Phyu Aung
Xinrun Wang
Runsheng Yu
Bo An
Senthilnath Jayavelu
Xiaoli Li
Venue:  (2021)
Keywords: GAN
Generative Models
Adversarial Networks
Game Theory

Abstract: In this paper, we propose a new approach to train Generative Adversarial Networks (GAN) where we deploy a double-oracle framework using the generator and discriminator oracles. GAN is essentially a two-player zero-sum game between the generator and the discriminator. Training GANs is challenging as a pure Nash equilibrium may not exist and even finding the mixed Nash equilibrium is difficult as GANs have a large-scale strategy space. In DO-GAN, we extend the double oracle framework to GANs. We first generalize the player strategies as the trained models of generator and discriminator from the best response oracles. We then compute the meta-strategies using a linear program. Next, we prune the weakly-dominated player strategies to keep the oracles from becoming intractable. We apply our framework to established architectures such as vanilla GAN, Deep Convolutional GAN, Spectral Normalization GAN and Stacked GAN. Finally, we conduct evaluations on MNIST, CIFAR-10 and CelebA datasets and show that DO-GAN variants have significant improvements in both subjective qualitative evaluation and quantitative metrics, compared with their respective GAN architectures.

pure Nash Under review as a conference paper at ICLR 2021
DO-GAN: A DOUBLE ORACLE FRAMEWORK FOR GENERATIVE ADVERSARIAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, we propose a new approach to train Generative Adversarial Networks (GANs) where we deploy double-oracle framework using the generator and discriminator oracles. GAN is essentially a two-player zero-sum game between the generator and the discriminator. Training GANs is challenging equilibrium may not exist and even finding the mixed Nash equilibrium is difficult as GANs have a large-scale strategy space. In DO-GAN, we extend the double oracle framework to GANs. We first generalize the player strategies as the trained models of generator and discriminator from the best response oracles. We then compute the mtta-ssrat using a linear program. Next, we prune the weakly- dominated player strategies to keep the oracles from becoming intractable. We apply our framework to established architectures such as vanilla GAN, Deep Convo- lutional GAN, Spectral Normalization GAN and Stacked GAN. Finally, conduct evaluations on MNIST, CIFAR- 10 and CelebA datasets and show that DO-GAN variants have significant improvements in both subjective qualitative evaluation and quantitative metrics, compared with their respective GAN architectures.
INTRODUCTION
Generative Adversarial Networks (GANs) (Goodfellow et al. 2014) have been applied in various domains such as image and video generation, mmage-to-image translation and eext-to-image synthe- sis (Liu et al. 2017 77 Reed et al., 2016). Various architectures are proposed to generate more realistic samples (Radford et al.) 2015 Mirza & Osindero 2014: Pu et al. 2016) as well as regularization techniques (Arjovsky et al. can be viewed as a two-player game where the generator samples the data and the discriminator classifies the data as real or generated. The two networks are alternately trained to maximize their respective utilities until convergence corresponding to pure Nash Equilibrium (NE).
et 20188)))) 8b). From the game-theoretic perspective, GANs
However, pure NE cannot be reliably reached by existing algorithms as pure NE may not exist (Farnia & Ozdaglar 2020 Mescheder et al., 2017). This also leads to unstable training in GANs depending on the data and the hyperparameters. Therefore, mixed NE is more suitable solution concept (Hsieh et al., 2019). Several recent works propose mixture architectures with multiple generators and discriminators that consider mixed NE such as MIX+GAN (Arora et al., 2017) and MGAN (Hoang et al.) 2018). MIX+GAN and MGAN cannot guarantee to converge to mixed NE. Mirror-GAN (Hsich et al. 2019) finds the mixed NE by sampling over the strategy space and proposes provably convergent proximal methods. However, the sampling approach may not be efficient as mixed NE may only have few strategies in the support set.
Double Oracle (DO) algorithm (McMahan et al. 2003) is a powerful framework to compute mixed NE in large-scale games. The algorithm starts with restricted game with a small set of actions and solves it to get the NE strategies of the restricted game. The algorithm then computes players' best-responses using oracles to the NE strategies and add them into the restricted game for the next iteration. DO framework has been applied various disciplines (Jain et al.) 2011 Bošansky et al. 2013). as well as Multi-agent Reinforcement Learning (MARL) settings (Lanctot et 2017).
Inspired by the successful applications of DO framework, we, for the first time, propose a Double Oracle Framework for Generative Adversarial Networks (DO-GAN). This paper presents four key contributions. First, we treat the generator and the discriminator as players and obtain the best responses from their oracles and add the utilities meta-matrix. Second, we propose linear

---

Under review as a conference paper at ICLR 2021
program to obtain the probability distributions of the players' pure strategies (meta-strategies for the respective oracles. The linear program computes an exact mixed NE of the meta-matrix game in polynomial time. Third, we propose pruning method for the support set of best response strategies to prevent the oracles from becoming intractable as there is a risk of the meta-matrix growing very large with each iteration of oracle training. Finally, we provide comprehensive evaluation on the performance of DO-GAN with different GAN architectures using both synthetic and real-world datasets. Experiment results show that DO-GAN variants have significant improvements in terms of both subjective qualitative evaluation and quantitative metrics.
2
RELATED WORKS
In this section, we briefly introduce existing GAN architectures, double oracle algorithm and its applications such as policy-state response oracles that are related to our work.
GAN Architectures. Various GAN architectures have been proposed to improve the performance of GANs. Deep Convolutional GAN (DCGAN) (Radford et al.) 2015) replaces fully-connected layers in the generator and the discriminator with deconvolution layer of Convolutional Neural Networks (CNN). Weight normalization techniques such as Spectral Normalization GAN (SNGAN) (Miyato
cet al. 2018a) stabilize the training of the discriminator and reduce the intensive hyperparameters tuning. There are also multi-model architectures such as Stacked Generative Adversarial Networks (SGAN) (Huang et al. 2017) that consist of a top-down stack of generators and a bottom-up discriminator network. Each generator is trained generate lower-level representations conditioned on higher-level representations that can fool the corresponding representation discriminator. Training GANs is very hard and unstable as pure NE for GANs might not exist and cannot be reliably reached by the existing approaches (Mescheder et al. 2017). Considering mixed NE, MIX+GAN (Arora al., 2017) maintains a mixture of generators and discriminators with the same network architecture but have their own trainable parameters. However, training a mixture of networks without parameter sharing makes the algorithm computationally expensive. Mixture Generative Adversarial Nets (MGAN) (Hoang et al., 2018) propose to capture diverse data modes by formulating GAN as game between a classifier, discriminator and multiple generators with parameter sharing. However, MIX+GAN and MGAN cannot converge to mixed NE. Mirror-GAN (Hsich ct al., 2019) finds the mixed NE by sampling over the mimente-dimensionall strategy space and proposes provably convergent proximal methods. The sampling approach may be inefficient to compute mixed NE as the mixed NE may only have a few strategies with positive probabilities in the infinite strategy space.
Double Oracle Algorithm. Double Oracle (DO) algorithm starts with a small restricted game between two players and solves it to get the player strategies at NE of the restricted game. The algorithm then exploits the respective best response oracles for additional strategies of the players. The DO algorithm terminates when the best response utilities are not higher than the equilibrium utility of the current restricted game, hence, finding the NE of the game without enumerating the entire strategy space. Moreover, in two- -laypeayer zero-sum games, DO converges to min-man max equilibrium (McMahan (et al. 2003). DO framework used to solve large-scale normal-form and extensive-form games such as security games (T'sai et al. 2012: Jain et al. 2011), poker games (Waugh et al., (009) and search games (Bosansky et al.) 2012). DO framework is also used in MARL. settings (Lanctot et al., 2017. Muller et al.) 2020). Policy-Space Response Oracles (PSRO) generalize the double oracle algorithm in a multi-agent reinforcement learning setting (Lanctot et al. 2017). PSRO treats the players' policies the best responses from the agents' oracles, builds the meta-matrix game and computes the mixed NE but it uses Projected Replicator Dynamics that update the changes in the probability of each player's policy each iteration. Since the dynamics need to simulate the update for several iterations, the use of dynamics takes a longer time to compute the stta-sttategies and does not guarantee to compute an exact NE of the meta-matrix game. However, in DO-GAN, can use linear program to compute the players' meta-strategies in polynomial time since GAN is a two-player zero-sum game (Schrijver 1998).
2

---

Under review as a conference paper at ICLR 2021
3 PRELIMINARIES
In this section, we mathematically explain the preliminary works that are needed to explain our DO-GAN approach including generative adversarial networks and game theory concepts such as normal-form game and double oracle algorithm.
3.1 GENERATIVE ADVERSARIAL NETWORKS
Generative Adversarial Networks (GANs) (Goodfellow et al. 2014) have become one of the dominant methods for fitting generative models to complicated real-life data. GANs are deep neural net architectures comprised of two neural networks trained in an adversarial manner to generate data that resembles a distribution. The first neural network, a generator G, is given some random distribution (z(z) on the input noise z and a real data distribution Pdats (x) on training data x. The generator is supposed to generate as close as possible to Paota (x). The second neural network, a discriminator D, to discriminate between two different classes of data (real or fake) from the generator.
Let the generator's differentiable function be denoted G(z, Ty)) and similarly D(x, Ra) for the discriminator, where G' and are two neural networks with parameters Tg and zd. Thus, D(x) represents the probability that x comes from the real data. The generator loss LG and the discriminator loss LD are defined as:
LD Ex 1x)1- log D(x)] + Ex- log(1 - D(G(z))),
(1)
LG Exmpate [og((1 D(G(z))
(2)
GAN is then up as two- zero-sum game between G and D as follows:
ming max E Ex-ppara Dlog D(x)] + (x [ogg(1 D(G(z)))
(3)
During training, the parameters of G and D are updated alternatlly until we reach the global optimal solution D(G(z) 0.5. Next, we let II, and IIa be the set of parameters for G and D. considering the set of probability distributions oy and od, the mixed strategy formulation (Hsieh et al. 2019) is:
min max F (x) [log D(x, ma)] + Eng Ex=px(s) [log(1 [1ogg(1))
D(G mg); ma)]-
09 oa
Similarly to GANS, DCGAN, SNGAN and SGAN can also be viewed as two-player zero-sum games with mixed strategies of the players. DCGAN modifies the vanilla GAN by replacing colle-connected layers with the convolutional layers. SGAN trains multiple generators and discriminators using the loss as linear combination of loss terms: adversarial loss, conditional loss and entropy loss.
3.2 NORMAL FORM GAME AND DOUBLE ORACLE ALGORITHM
normal-form game is a tuple (II, U, where n is the number of players, = (II,...., II,) is the set of strategies for each player i N, where N = [1, (I. and U II + R8 is a payoff table of utilities R for each joint policy played by all players. Each player chooses the strategy to maximize own expected utility from II, or by sampling from distribution over the set of strategies a, (II)) We can use linear programming, fictitious play (Berger (007) or regret minimization (Roughgarden 2010) to compute the probability distribution over players' strategies.
In the Double Oracle (DO) algorithm (McMahan et al. 2003), there are two best response oracles for the row and column player respectively. The algorithm creates restricted games from subset of strategies at the point of each iteration t for row and column players, i.e., II, C II, and II, C II as well as a meta-matrix U at the 4h iteration. We then solve the meta-matrix to get the probability distributions on ITA and IT. Given a probability distribution 0a of the column player strategies, B (R,(a.) gives the row player's best response to ae Similarly, given probability distribution o, of the row player's strategies, (R,( is the column player's best response to a,,. The best responses are added to the restricted game for the next iteration. The algorithm terminates when the best response utilities not higher than the equilibrium utility of current restricted game. Although in the worst-case, the entire strategy space may be added to the restricted game, DO is guaranteed to converge to mixed NE in two-player zero-sum games. DO is also extended to the multi-agent reinforcement learning in PSRO (Lanctot et al. 2017) to approximate the best responses to the mixtures of agents' policies, and compute the meta-strategies for the policy selection.
3

---

Under review as a conference paper at ICLR 2021
Table I: Comparison of Terminologies between Game Theory and GAN
Game Theory terminology
GAN terminology
player
generator/ discriminator
the parameter setting of generator/ discriminator e.g. TT and
strategy
the sequence of parameters (strategies) till epoch t e.g. (T " "*>... .... o
policy
Note: Not used in DO-GAN.
game
the minmax game between generator and discriminator
minmax game between generator & discriminator with their respective set of strategies epoch of DO framework
veta-game// meta- metaimatrix
meta- veta-strategy
the mixed NE strategy of gemerattorddiscriminato at epoch t
DO-GAN: DOUBLE ORACLE FRAMEWORK FOR GAN
As discussed in previous sections, computing mixed NE for GANs is challenging as there is an extremely large number of pure strategies, i.e., possible parameter settings of the generator and discriminator networks. Thus, we propose a double oracle framework for GANs (DO-GAN) to compute the mixed NE efficiently. DO-GAN builds restricted meta-matrix game between the two players and computes the mixed NE of meta-matrix game, then DO-GAN iteratively adds more generators and discriminators into the meta- meta-matrix game until termination.
4.1 GENERAL FRAMEWORK OF DO-GAN
GAN can be translated as a two-player zero-sum game between the generator player and the discriminator player To compute the mixed NE of GANs, at iteration t, DO-GAN creates restricted meta-matrix game U with the trained generators and discriminators as the strategies of the two players, where the generators and discriminators are parameterized by Wg E 9 and aa E We use ( wa) to denote the generator player's 's payoff when playing ". against as I.D- Since GAN is zero-sum, the discriminator player's payoff -U' Ta). We define of and o'j as the mixed strategies generator player and discriminator player, respectively. With slight abuse of notation, define the generator player's 's expected utility of the mixed strategies (a od) as T Ua x = Enyeg ExSED of (T))) (ma)) U"T ma). We use (a ott) to denote the mixed NE of the restricted meta-matrix game U*. We solve U to obtain the mixed NE, compute the best responses and add them into Tt for the next iteration. Figure presents an illustration of DO-GAN and Al Algorithm] [ describes the overview of the framework.
which is defined
repposate
noulte
Expand Meta-matrix Game
Terminate
<e(1)
(a(2)
(T(1)
-2
Metn- matrix Game
Compute Best Responses
Start
(2(2)
0
-3
End
Solve Meta-matrix
Figure 1: An illustration of DO-GAN, Figure adapted from (Lanctot et al. 2017
Our algorithm starts by initializing two empty arrays G and D to store multiple generators and discriminators (line D. We train the first ng and na with the canonical training procedure of GANs (line We store the parameters of the trained models in the two arrays G and D (line compute the adversarial loss Lnp and add it the meta-matrix 1 (line[4). We initialize the strateets
4

---

Under review as a conference paper at ICLR 2021
"y" [1] and of-h = [1] since there is only one pair of generator and discriminator available (line 5 For each epoch, responses Tig and T/a to o' and o via Adam Optimizer, respectively, then add them into and D (lines) 10p. We then augment (+-11 by adding m and T/2 and calculating (' T'd) to obtain U and compute the missing entries (line 11 We compute the missing payoff entries U (
generatorOracle(( and discriminatorOracles to obtain the best
Ta), (a)) YTa E D and U* (T ma), VTg E G by sampling few batches of training data. After that, we compute the mixed NE (a 9 add of U with linear programming (line 12 The algorithm terminates if the criteria described in Algorithm 2 is satisfied (line 13). We also prune the support strategy set of the players
t
as described Algorithm 3 14 to avoid G and becoming intractable.
In generatorOracle(] we train mg to obtain the best response against o' i.e., (") > UH [TT Vng E ng.. Similarly, in disciminattorrraclel]
train IT to obtain the best response against of i.e.. U""a"" Ta)) 2 U""o""" (a)) VTa E Llg. Full details of generator oracle and discriminator oracle can be found in Appendix
Algorithm 1: Double Oracle Framework for GAN (DO-GAN)
Initialize generator and discriminator arrays G = 0 and D 0: Train generator & discriminator get the first Fy and mai
с 9 U { D + DU {Ta}:
Compute the adversarial loss LD and add it to meta meta-matrix U";
Initialize o0 = [1] and of = [A];
for epoch f E 21, do
m f- generatorOracle(oj
G - (
v' + discriminattorrrccleiss
10
DU
11
Augment ( with Ti and T'a to obtain U and compute missing entries;
12
Compute mixed NE (o of) ) for U with linear program;
// Section 4.2
13
rermi nat oncheck (U*, """""""0"" then break;
// Section 4.3
PruneMetaMatrix (U' ,))
// Section 4.4
Algorithm 3: PruneMetaMMtriix(U
Algorithm 2: Ter.iiaationCheeeeeeeee o , ,
// I stores indices to be pruned from G and D // G stores models to be pruned from and D
U is of size m n
I, 0; La 0
/ [G| m. m |
Kg 0: K. 0
if
9
> then 1} do
Compute (" a
for {0, (0.....
Compute (
if o (
min o then
Compute ("([mm)
1 -
U (i): K, + K, (
gentne = ('( at) U (a o
D >
then for j E (O....
disinc Ua D(m))= ((m)) (- (ot' of))
[D] - 1) do
genine < && disine < e then
if if (D(j)) = min o then
return True
Ia Ia
Ka < (D)))
else return False
gK - D Ka;
+- Ji J
4.2 LINEAR PROGRAM OF META-MATRIX GAME
Since the current restricted meta-matris game U a zero-sum game, we can use a linear program to compute the mixed NE in polynomial time (Schrijver 1998). Given the generator player g mixed strategy oi the discriminator player d will play strategies that minimize the expected utility of Thus, the mixed NE strategy for the generator player of is to maximize the worst-case expected utility, which is obtained by solving the following linear program:
ot = arg max,! {v: :0 20 Σ
(1) = 1, 1o ma) 2 u. VTa E D).
(5)
5

---

Under review as a conference paper at ICLR 2021
Similarly, obtain the mixed NE strategy for the discriminator of by solving linear program that maximizes the worst-case expected utility of the discriminator player. Therefore, we obtain the mixed NE (o a of the restricted meta-matrix game U
4.3 TERMINATION CHECK
DO terminates the training by checking whether the best response xt (or TT)) is in the support set 9 (or D) @Jain al.)
| 2011). but we cannot apply this approach to DO-GAN GAN has dimens--immnnional strategy space (Hsieh et al. 2019). Hence, we terminate the training if the best responses cannot bring a higher utility to the two players than the entries of the current support sets, as discussed in (Lanctot et al. 2017 Muller et al. 2020). Specifically, we first compute (a bx od and the expected utilities
for new generator and discriminator (G[mm, U'""" D(nl) (line 3 Then, we calculate the
utility increment (lines
5 and returns True if both (g(m) ( and Una D(nl) cannot bring
higher utility than U (agg of") by (lines
4.4 PRUNING META-MATRES
As the meta- meta-matrix grows with every epoch of DO, there is risk that the support strategy set becomes very large and G and become intractable. from (Cheng & Wellman 2007), as depicted in Algorithm When either [ or [D] is greater than the limit the support set size s, we prune at least one strategy with the least probability, which is the strategy that contributes the least the player's winning. Specifically, define J1,6 where I is the set of row numbers to be removed, b is the total rows of matrix. To remove the 2nd row of
avoid this, we adapt the greedy pruning algorithm
1 If [ > 8, at least one strategy with minimum probability is pruned from 9, similarly for D (lines 3 Finally, prune the using matrix multiplication (line 10
matrix having rows, we define 1 {1}, b = 3 and J(13,3 =
EXPERIMENTS
We conduct our experiments on a machine with Xeon(R) CPU E5-2683 v3 v 2.00GHz and 4x Tesla v100-PCIE-1166 running Ubuntu operating system. We evaluate the double oracle framework for some established GAN architectures such as vanilla GAN (Goodfellow et al. 2014). DCGAN (Rad) Ford et al. 2015), SNGAN (Miyato al. 2018a) and SGAN (Huang et 2017). We adopt the parameter settings and criterion of the GAN architectures as published. We set = mentioned otherwise. We compute the mixed NE of the meta-matrix game with Nashpy The evaluation details are shown in Appendix B
10 unless
Epoch
Epoch 5000
Epoch 10000
Epoch 15000
Epach 20000
Target
Epoch
Fpoch
Epoch 10000
Fpoch 15000
Epoch 20000
Target
DO-GAN
Figure 2: Comparison of GAN and DO-GAN on 2D synthetic dataset
5.1 EVALUATION ON SYNTHETIC 2D GAUSSIAN MIXTURE DATASET
To illustrate the effectiveness of the architecture, we train a double oracle framework with the simple vanilla GAN architecture 2D mixture of 8 Gaussian mixture components with cluster standard
htm1

---

Under review as a conference paper at ICLR 2021
deviation 0.1 which follows the experiment by (Metz et al. 2017). Figure 2 shows the evolution of 512 samples generated by GAN and DO-GAN through 20000 epochs. The goal of GAN and DO-GAN is to conrectly generate samples at 8 modes as shown in the target. The results show that GAN can only identify 6 out of 8 modes of the synthetic Gaussian data distribution, while the DO-GAN obtain all the 8 modes of the distribution. Furthermore, DO-GAN takes shorter time (less than 5000 epochs) to identify all 8 modes of the data distribution. We present a more detailed evolution of data samples through the training process 2D Gaussian Mixtures in Appendix C
Ablations. We also varied the support set size of the training 5, 10, 15 and recorded the computation time as discussed in Appendix D We found that the training cannot converge when 5 and takes long time when 15. Thus, we chose 10 for the training process.
5.2 EVALUATION ON REAL- WORLD DATASETS
We evaluate the performance of the double oracle framework which takes several established GAN architectures as the backbone as discussed in Appendix GG] i.c., GAN (Goodfellow et al. 2014).
DCGAN (Radford et al. 2015) and SGAN (Huang et 2017 with convolutional layers for the deep neural networks of GAN as well as SNGAN (Miyato et al. 2018a) which uses normalization techniques. We rum experiments
MNIST (LeCun & Cortes 2010), CIFAR-10 (Krizhevsky et al., 2009) and CelebA (Liu et al. 2015) datasets. MNIST contains 60,000 samples of handwritten digits with images of 28 x 28. CIFAR- 10 contains 50, 000 training images of 32 32 of 10 classes. CelebA
is a large-scaled face dataset with more than 200K images of size 128 x 128.
5.2. QUALITATIVE EVALUATION
We choose the CelebA dataset for the qualitative evaluation since the training images contain noticeable artifacts (aliasing, compression, blur) that make the generator difficult produce perfect and faithful images. We compare the performance of DO-DCGAN, DO-SNGAN and DO-SGAN with their counterparts SNGAN which trained for 40 epochs with termination e of 10-5 for DO-SNGAN where other architectureas are trained for 25 epochs with termination e of 5 x 10 for the double oracle variants. The generated CelebA images of DCGAN and DO-DCGAN are shown in Figure 3 where we find that DCGAN suffers mode-collapse, while DO-DCGAN does not. We also present the generated images of SNGAN vs DO-SNGAN and SGAN vs DO-SGAN using fixed noise at different training epochs in Figure E and [
10
From the results, we can see that SNGAN, SGAN, DO-SNGAN and DO-SGAN are able to generate various faces, i.e., no mode-collapse. Judging from subjective visual quality, we find that DO-SNGAN and DO-SGAN are able to generate plausible images faster than SNGAN and SGAN during training, i.e., 17 epochs for DO-SGAN and 20 epochs for SGAN, More experiment results on CIFAR-10 can be found in Appendix P
Loech
Spoch 5
Looch 10
Epoch 15
Spech 20
Looch
ts 20
Epoch 13
Fporh
Figure 3: Training images with fixed noise for DCGAN and DO-DCGAN until termination.
We do not evaluate the performance of vanilla GAN and its DO variant CelebA dataset since DCGAN and SGAN outperform vanilla GAN in image generation tasks (Radford al. 2015)
7

---

Under review as a conference paper at ICLR 2021
Figure 4: Training images with fixed noise for SNGAN and DO-SNGAN until termination.
SGAN
Epoch
Epoch
Epoch
Epoch 15
Epoch
DO-SGAN
Epoch
Epoch 5
Epoch 10
Epoch 15
Epoch 17
Figure 5: Training images with fixed noise for SGAN and DO-SGAN until termination.
5.2.2 QUANTITATIVE EVALUATION
In this section, we evaluate the performance of various architectures by quantitative metrics.
Inception Score. We first leverage the Inception Score (IS) (Salimans et al. 2016) by using Inception_v3 (Szegedy et al. 2016) as the inception model. To compute the inception score, first compute the Kullback-Leinn IS exp(E,, (L(D(p(v)x) |I p(Y)))) where p(v) is the conditional label distributions for the images in the split and (xy(x) is that of the image x estimated by the reference inception model. Inception score evaluates the quality and diversity of all generated images rather than the similarity to the real data from the test set.
(KL) divergence for all generated images and use the equation
FID Score. Fréchet Inception Distance (FID) measures the distance between the feature vectors of real and generated images using Inception v3 model (Heusel et al., 2017). Here, we let p and q be the distributions of the representations obtained by projecting real and generated samples to the last hidden layer of Inception model. Assuming that p and g the multivariate Gaussian distributions, FID measures the 2- Wasserstein distance between the two distributions. Hence, FID Score can capture the similarity of generated images real ones better than the Inception score.
8

---

Under review as a conference paper at ICLR 2021
Table Inception scores (higher is better) and FID scores (lower is better). The mean and standard deviation are drawn from running 10 splits on 10000 generated images. The magenta values are the improvements of the DO-GAN variants compared with their counterparts.
Inception Score
FID Score
MNIST
CIFAR-10
CIFAR-10
CelebA
GAN
1.04 0.05
3.84 0.09
71.44
DCGAN
1.26 + 0.05
6.32 + 0.05
37.66
10.92
SNGAN
1.35 + 0.11
7.58 + 0.12
25.5
7.62
SGAN
1.39 0.09
8.62 0.12
24.83
6.98
MIX+DCGAN
7.72 0.09
MGAN
8.33 + 0.10
26.7
DO-GAN
1.39 0.09 (+0.35)
7.20 + 0.16 (+-3336)
31.44 (-40.00)
DO-DCGAN
1.42 0.11 (+0.16)
7.86 + 0.14 (+1.54)
22.25 (-15.41)
7.11
(-381)
DO-SNGAN
1.42 + 0.07 (+0.07)
8.55 0.08 (+0.977)
18.20 (-7.30)
6.92
-0.70)
DO-SGAN
1,42 + 0.09 (+0.03)
8.69 0.10 (+0.07)
16.56
(-8.27)
6.32
(0066)
Note: MIX+DCGAN and MGAN results are directly copied from (Arora et al.
2017
Hoang et al.
2018)
Results. The results are shown in Table 2 In CIFAR-10 dataset, DO-GAN, DO-DCGAN and DO-SNGAN obtain much better results (7 = 0.16, 7.86 14 and 8.55 than GAN, DCGAN do not see a significant
and SNGAN (3.84 0.09, 6.32 + 0.05 and 7.58 + 0.12). However, improvement in DO-SGAN compared to SGAN 8.62 = 0.12 and 8.69 + 0.10 since SGAN already generate diverse images. We did not include IS for CelebA dataset IS cannot reflect the real image quality for the CelebA, as observed in (Heusel et al. 2017). In CIFAR-10 dataset, DO-GAN, DO-DCGAN, DO-SNGAN and DO-SGAN obtain much lower FID scores (31.44, 22.25. 16.56, 18.20) respectively. The trend follows CelebA obtaining 7.11 for DO-DCGAN while 10.92 for DCGAN, 7.62 for SNGAN while 6.92 for DO-SNGAN, 6.98 for SGAN and 6.32 for DO-SGAN respectively. Although we see a significant improvement in the quality of DO-SGAN images, FID score for DO-SGAN is affected by the distortions. According the results, can see that DO framework performs better than each of their original counterpart architectures. More details can be found in Appendix
CONCLUSION
We propose novel double oracle framework GANs, which starts with restricted game and incrementally adds the best responses of the generator and the discriminator to compute the mixed NE, We then compute the players' metatstrateeies by using linear program. We also prune the support strategy set of players. We apply DO-GAN approach to established GAN architectures such as vanilla GAN, DCGAN, SNGAN and SGAN. Extensive experiments with the 2D Gaussian synthetic data set as well as real-works datasets such as MNIST, CIFAR-10 10 and CelebA show that DO-GAN variants have significant improvements comparison their respective GAN architectures both in terms of subjective image quality as well as in terms of quantitative metrics.
REFERENCES
Martin Arjovsky. Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In ICML, pp. 214-223, 2017.
Sanjeev Arora, Rong Ge. Yingyu Liang, Tengyu Ma, and Yi Zhang. Generalization and equilibrium in generative adversarial nets (GANs). In ICML, pp. 224-232, 2017.
Ulrich Berger. Brown's original fictitious play. Journal of Economic Theory, 135(1):572-578, 2007.
B Bosansky, Christopher Kiekintveld, Viliam Lisy, and Michal Pechoucek. Iterative algorithm for solving two-player zero-sum extensive-form games with imperfect information. In ECAI, pp. 193-198, 2012.

---

Under review as a conference paper at ICLR 2021
Branislav Bošanský, Christopher Kiekintveld, Viliam Lisy, Jiri Cermak, and Michal Pechoucek. Double-oracle algorithm for computing an exact Nash equilibrium zero-sum extensive- games. In AAMAS, pp. 335-342, 2013.
Shih-Fen Cheng and Michael P Wellman. Iterated weaker-than-weak dominance. In IJCAI, pp- 1233-1238, 2007.
Farzan Farnia and Asuman Ozdaglar. GANs may have no Nash equilibria. arXiv preprint 1244 2020.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS, pp. 2672-2680, 2014.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge to local Nash equilibrium. In NeurIPS, pp. 6626-6637, 2017.
Quan Hoang, Tu Dinh Nguyen, Trung Le, and Dinh Phung. MGAN: Training generative adversarial nets with multiple generators. In ICLR, 2018.
Ya-Ping Hsieh, Chen Liu, and Volkan Cevher. Finding mixed nash equilibria of generative adversarial networks. In ICML, pp. 2810-2819, 2019.
Xun Huang, Yixuan Li, Omid Poursaeed, John Hoperoft, and Serge Belongie. Stacked generative adversarial networks. In CVPR, pp. 5077-5086, 2017.
Manish Jain, Dmytro Korzhyk, Ondřej Vančk, Vincent Conitzer, Michal Pěchouček, and Milind Tambe. A double oracle algorithm for zero-sum security games on graphs. In AAMAS, pp. 327-334, 2011.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. CIFAR-10 (Canadian Institute for Advanced Research). https//wwww.csttrrott edu/ kriz/cifar.html, 2009.
Mare Lanctot, Vinicius Zambaldi, Audrunas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Pérolat, David Silver, and Thore Graepel. A unified game-theoretic approach to multiagent reinforcement learning. In NeurlPS, pp. 4190-4203, 2017.
Yann LeCun and Corinna Cortes.
MNIST
handwritten digit database.
http://yaanlecmm.co////d/mmmm 2010. URLLtttp://yan. wow/exxb/mmii//eee
Ming-Yu Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In NeurlPS, pp. 700-708, 2017.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV. pp. 3730-33300 3738, 2015.
Brendan McMahan, Geoffrey Gordon, and Avrim Blum. Planning in the presence of cost functions controlled by an adversary. In ICML, pp. 536-543, 2003.
Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. The numerics of GANs. In NeurlPS, pp. 1825-1835, 2017.
Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. In ICLR, 2017.
Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint
arXiv:1411.1784, 2014.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. ICLR, 80.8a.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8):1979-1993 2018b.
10

---

Under review as a conference paper at ICLR 2021
Paul Muller, Shayegan Omidshafiei, Mark Rowland, Karl Tuyls, Julien Perolat, Siqi Liu, Daniel Hennes, Luke Marris, Mare Lanctot, Edward Hughes, et al. A generalized training approach for multiagent learning. In ICLR, 2020.
Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin. Variational autoencoder for deep learning of images, labels and captions. In NeurlPS, pp. 2352-2360, 2016.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1571.06433 2015.
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In ICML, 2016.
Tim Roughgarden. Algorithmic game theory. Communications of the ACM, 53(7):78-86, 86, 2010.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In NeurlPS, pp. 2234-2242, 2016.
Alexander Schrijver. Theory of Linear and Integer Programming. John Wiley & Sons, 1998.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR, pp.
2016.
Jason Tsai, Thanh H Nguyen, and Milind Tambe. Security games for controlling contagion. In AAAI, pp. 1464-1470, 2012.
Zhengwei Wang. Qi She, and Tomas Ward. Generative adversarial networks in computer vision: A survey and taxonomy. arXiv preprint 17Xiv:1906.0152,, 2019.
Kevin Waugh, Nolan Bard, and Michael Bowling. Strategy grafting in extensive games. In NeurIPS, pp. 2026-2034, 2009.

---

Under review as a conference paper at ICLR 2021
A FULL ALGORITHM OF DO-GAN
Algorithm 4: GeneratorOracllllog] (o D)
Initialize generator G with random parameter setting T
for iteration ko kn k,, do Sample noise z;
Ty = Sample a discriminator from D with o/;:
Initialize discriminator D with parameter setting Rdi Update the generator G's parameters Ty via Adam optimizer:
Vag log (1 - D(G(z)))
Algorithm 5: Di piseriminaterrraccceo (0)"
Initialize discriminator with random parameter setting n'Z;
for iteration .u... kn do
Sample minibatch of data x; for a minibatch do
Sample noise z;
"g =
Sample a generator from 9 with o
Initialize a generator G with a parameter setting Tg;
Generate and add to mixture G(z);
Update the discriminator D's parameters v'a via Adam optimizer:
10
V log D(x) + log (1 D(G(z)))
We train the oracles for some iterations which we denote as 1.... For experiments, train each oracle for an epoch for the real-world datasets and 50 iterations for the 2D Synthetic Gaussian Dataset. At each iteration I, sample the generators from the support G with the meta-strategy at to generate the images for evaluation. Similarly, we conduct the performance evaluation with the generators sampled from with the final og at termination. SGAN consists of top-down stack of GANs, c.g, for a stack 2, Generator is the first layer stacked on Generator with each of them connected Discriminator and 0 respectively. Hence, DO-SGAN, we store the meta-strategies for the Generator 0 and 1 in of* and the Discriminator and 0 for of In GeneratorOraclef, we first sample Discriminator 1 and () from discriminator distribution of and train Generator first then followed by calculating loss with Discriminator 1 and train Generator 0 subsequently, and finally calculate final loss with Discriminator 0 and train the whole model end to end. We perform the same process for Disciminatoorracclee.
12

---

Under review as a conference paper at ICLR 2021
B IMPLEMENTATION DETAILS
Table 3: Training Hyperparameters
GAN
DCGAN
SNGAN
SGAN
Generator Learning Rate
0.0002
0.0002
0.0002
0.0001
Discriminator Learning Rate
0.0002
0.0002
0.0002
0.0001
batch size
64
64
100
Adam: beta
0.5
0.5
0.5
0.5
Adam: beta
0.999
0.999
0.999
0.999
We implement our proposed method with Python 3.7, Pytorch=1.4.0 and Torchvision=0.5.00 We set the hyperparameters the original implementations. We present the hyperparamcters set in Table 3 use Nashpy to compute the equilibria the meta-matrix game.
C FULL TRAINING PROCESS OF 2D GAUSSIAN DATASET
Spack
Ejeck eite
baccs
faech vita
Ejech
Ceech
Daick line
Eaech 1i00a
Eaach 3000
Eaech 1000
Epech J400
Eyoc
Epenk 11809
Each 1280
Eash 1000
Epec L8W
(a) GAN
(b) DO-GAN
Figure Full comparison of GAN and DO-GAN on 2D Synthetic Gaussian Dataset
Epoch 0
Epoch 5000
Epoch 10000
Epoch 15000
Epoch 20000
GAN
Epoch
Epoch 5000
Epoch 10000
Epoch 15000
Epoch 20000
DO-GAN
Figure 7: GAN and DO-GAN comparison with Gaussian Mixture 7 modes
Figure shows the full training process of DO-GAN and GAN on 2D Synthetic Gaussian Dataset. From the results, we find that GAN struggles to generate the samples into 8 modes while DO-GAN generate all the 8 modes of the distribution. Furthermore, DO-GAN takes shorter time (less than 5000 iterations) to identify all 8 modes of the data distribution. Moreover, we present the experiment results mode and 9-mode Gaussian Mixtures in Figure 7 and g
13

---

Under review as a conference paper at ICLR 2021
Epoch 0
Epoch 5000
Epoch 10000
Epoch 15000
Epoch 20000
GAN
Epoch
Epoch 5000
Epoch 10000
Epoch 15000
Epoch 20000
DO-GAN
Figure 8: GAN and DO-GAN comparison with Gaussian Mixture 9 modes
D INVESTIGATION OF SUPPORT SET SIZE
We vary the support set size to 5, 10, 15 and record the training evolution and the running time as presented in Table 4 and Figure 9 We find that if the support size is too small, e.g., 8 = 5, the best responses which are optimal yet have better utilities than the models the support are added and pruned from the meta meta-matric matrix repeatedly making the training not able to converge. However, 15 takes significantly longer time as the time for the augmenting of mnta--atrrx becomes exponentially long with the support set size. Hence, we chose 10 as our experiment support set size since we observed that there no significant trade-off and shorter runtime.
Table 4: Runtime of DO-GAN on 2D Gaussian Dataset with s 5, 10,
Support Set Size
Runtime (GPU hours)
5
>
1
10
0.5627
8 = 15
0.9989
g
Epoch
Epoch
y-10
Epoch 0
Epoch 5000
Epoch 10000
Epoch 15000
Epoch 20000
Epoch 25000
converged
w-55
Epech
Epach 15008
Epoch 20000
Figure 9: Training evolution on 2D Gaussian Dataset with 5, 10, 15
14

---

Under review as a conference paper at ICLR 2021
E GENERATED IMAGES OF CELEBA AND CIFAR-10
In this section, we present the training images of CelebA and CIFAR-10 datasets.
Fpoch
Fpoch 25
Epech
Epoch
Epoch
Epoch
Figure 10: Training images with fixed noise for DCGAN and DO-DCGAN until termination.
Epoch
Epoch 10
Epoch 20
Epoch 30
Epoch
Epoch
Epoch 10
Epoch 20
Epoch 30
Epoch
Figure 11: Training images with fixed noise for SNGAN and DO-SNGAN until termination.
Figure 10 shows the training samples of DO-DCGAN and DO-DCGAN through the training process.
Figure 11 also shows those of SNGAN which is trained for 40 epochs with termination of 5 x 10-5 for DO-SNGAN, The results show that DCGAN suffers from mode-collapse, generating similar face while DO-DCGAN can generate more plausible and varying faces. We also present the generated images of DCGAN, DO-DCGAN, SNGAN, DO-SNGAN, SGAN and DO-SGAN of CIFAR-I0 dataset showing that DO-DCGAN, DO-SNGAN and DO-SGAN can generate better and more identifiable images than DCGAN, SNGAN and SGAN respectively. We present more of the generated samples from SNGAN, DO-SNGAN, SGAN and DO-SGAN on CelebA dataset in Figure B3
15

---

Under review as a conference paper at ICLR 2021
@ DCGAN
(b) SNGAN
Cc) SGAN
(d) DO-DCGAN
(e) DO-SNGAN
(i) DO-SGAN
Figure 12: Generated images of CIFAR- 10 dataset
(a) SNGAN
(b) DO-SNGAN
Figure 13: Generated images of CelebA dataset for DO-SNGAN and SNGAN
16

---

Under review as a conference paper at ICLR 2021
(c) SGAN
(d) DO-SGAN
Figure 13: Generated images of CelebA dataset for DO-SGAN and SGAN
F FID SCORE AGAINST ITERATIONS
To compute FID score, we use Inception v3 model with max pool of 192 dimensions and the last layer coding layer mentioned in (Heusel et al. 2017). We resized MNIST, CIFAR-10 generated and test images to 32 x 32 and CelebA images to 64 x 64. The FID score against training epochs for CIFAR-10 dataset follows:
SGAN B DO-SGAN
FID Score
10a
10
101
50
100
150
200
250
300
350
400
450
500
Epochs
Figure 14: FID score vs. Epochs for SGAN and DO-SGAN trained on CIFAR-10
Figure 14 presents the FID score against each epoch of training for SGAN and DO-SGAN CIFAR- 10. While both perform relatively well in generating plausible images, we can see that DO-SGAN terminates early epoch 288 and has better FID score of 16.56 compared 24.83 at 300 epoch until 21.284 at 500 epoch for the training of SGAN.
17

---

Under review as a conference paper at ICLR 2021
CHOICE OF GAN ARCHITECTURES FOR EXPERIMENTS G
DCGAM 2016 (Tron corvilation general
4BRGAN, 2017 (Mutemnic as discris disrieriuttu)
Network architacture
@*8666h 2015 Bherpe ect and loper sieel
1AGAN 2013 telf attrrtinn modklt)
(MC- 2617 (Saellary casollee)
nexutor
2015
lobele)
Architecture
Latent space
SICAN 2016 (Eocddr leaning invere
(yclena 2011. DisceidN, 2013 DuilisN, 2011 (magg tyletiande)
SRGAN, Irage saper revelaioon
SaleGAN 2010 Suule speniff yeveratkes
Application focused
Pace Completion GAN 2017 Face completion
Proposed GANs Taxonomy
More GAN 2013 DVD SAN, 2033 video generation
SAGAA, 2019 Jsruge marios latian loarned one Image
*SAA
(ettic)
IPM based
JUGAN 2017 (Manterrate dissres)
Loss Types
PCGAN 2014 divergente" LLGAAL 2015 Pearson distrpenct) (-GGA) 2865 (hcheerpene))
Nill (Seeeed gratier lock)
take sart ples!
Loss
pauiity
"WCMI 2018 stency WGAM)
WGAM
gisdhits peralty WCAN,
Regularization
-NRCAN 2015 Poonalls ribsinng medea))
SS 6AN, 2004 (Sel-caparr sice avoid discriniioon fargettigg!
Figure 15: Taxonomy of GAN Architectures from (Wang et al. 2019
We carried out experiments with the variants of GANs to evaluate the performance of our DO-GAN framework. We refer to the taxonomy of GANs (Wang et al. 2019) and choose each architecture from the groups of GANs focused on Network Architecture, Latent Space and Loss: DCGAN, SNGAN and SGAN as shown in Figure 15] We have also included comparisons with mixture architectures such MIXGAN and MGAN.
18

---

Under review as a conference paper at ICLR 2021
H
EXAMPLE OF META-MATRIX WITH 5 GENERATORS AND 5 DISCRIMINATORS
Epoch 5
0.38008
1e-05
6e-05
0.00027
0.0004
5.0715
0.0273
2e-05
2e-05
3e-05
Discriminators
6
~
4.2523
5.3273
1.3967
3e-05
3e-05
3.9012
7.4184
5.3091
2.326
4e-05
2
3.7772
10.301
6.1547
4.9584
0.01739
2 Generators
(a) Meta-matrix epoch =
Meta-Strategies: 0g = 0,, = 10,0,000,11
Expected Payoff: (o "=)"""/"""=01 0.017
Epoch 20
7.5
4.2523
5.3273
1.3967
0.00138
0.00265
Discriminators
1.2102
4661
4.0996
1.7503
0.01295
0.20349
4.0336
3.6881
0.032
0.38008
1e-05
6e-05
7.7964
4.0879
1
Generators
(b) Meta-matrix epoch t 20 Meta-Strategies:
o 20 [0.00, 0.27, 0.32, 0.00, 0.41]
20- [0.29, 0.00, 0.32, 0.00, 0.38
0
Expected Payoff: 20 (a
20+
od = 1.69
19


Reviews:
[{"review_id": "j2sJk3TTgk", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to use the well-known Double Oracle methods for solving large scale games for computing the equilibrium in GANs. The main idea of a mixed strategy being a mixture over generators (and mixture over discriminator for toher player( is from Hsieh et al. The double oracle approach is shown to yield superior results on three image datasets.\n\nPros:\nThe idea of using double oracle algorithm for GAN is novel though surprising that no one has tried it earlier.\n\nCons:\n- I feel it is very important to compare (experimentally) to Hsieh et al. ICML19. That work is also computing NE in the exact same model of mixture over generator and discriminator, which this paper borrow. Not sure why this is not done. In fact, the authors have this line about Hsieh et al \"The sampling approach may be inefficient to compute mixed NE as the mixed NE may only have a few strategies with positive probabilities in the infinite strategy space\" - do the authors have any evidence (theory or experiment) to support this?\n- The evaluation, while exploring many GANs, is missing a few recent ones. Have the authors tried LOGAN (achieves same scores almost of CIFAR). Also, wondering why WGAN is not considered - WGAN also is a game and actually better than some of the GANs tested.\n- Clearly the NE is not being computed exactly because of the many approximations (e.g., approximate best response). \n- It is very surprising that only 10 pure strategies are enough in the mixed strategy support when the pure strategy space is infinite - there are results on small support for approximate NE (please cite those also). I am not sure though if it is so small, the lack of rigor here is unconvincing.\n- There is no new technique in the way double oracle is used. That makes it seem like an engineering effort.\n- Can some of the other approaches in PSRO framework work better than double oracle? (why do you the authors call PSRO use as \"beyond games\", PSRO is still being used in a game, just a very large game)\n- I do not think comparing time by comparing epochs is fine - doesn't the double oracle epochs include Nash equilbrium computation. Please state these clearly.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "A-YNwAhdiZB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a  new training framework for GAN, inspired by the double oracle (DO) algorithm in game theory. The authors design many mechanisms to make it possible to employ DO in GAN training. The motivation is clear, and the experimental results support the claim. \n\nFor the proposed training framework, I have the following concerns:\n\n1. It is unclear what tasks are suitable for the proposed DO-framework. In the paper, several experiments are conducted in the laboratory and real-world environment. However, some of the more common GAN scenarios at this stage (2020) have not been proven to be suitable for DO-framework, such as high-resolution image synthesis, e.g., CelebA-HQ and LSUN scene generation [1] and conditional generation [2]. I am curious about the performance of the DO-framework in these tasks.\n2. Will computational complexity be an obstacle to the application of this method? Will the proposed method require more storage space?\n3. In figure.13, the DO-GAN stopped before 300 epochs. However, its FID still seems to be declining. If DO does not terminate the program, will it be better?\n\nReferences:\n[1] Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4401-4410).\n\n[2] Park, T., Liu, M. Y., Wang, T. C., & Zhu, J. Y. (2019). Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2337-2346).\n\nUpdates:\n\nI have read the author's response and the comments of other reviewers. Although as an \"approach as the proof of concept\", limited experiments on large-scale GAN models are still necessary. Just provide evidence to prove it is feasible. This is a little disappointment for me. For my other questions, I agree with the author's response. Since I have given a positive initial rating (6), I will keep this rating. Thanks to the author's reply and AC's efforts.", "extra_scores": {}, "extra_text": {}}, {"review_id": "8a3Qyi2ti38", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper applies Double-Oracle (DO) / PSRO to training a GAN, a 2-player zero-sum game. DO cannot be applied directly \"out-of-the-box\". Instead of an exact oracle, the generator and discriminator are trained using local gradient optimizers for a finite number of steps. Also, in DO, the meta-game matrix can grow very large and maintaining and training against a large population of neural networks is expensive, so the population is pruned throughout training.\n\nThe authors list 4 \"key contributions\": 1) treating D and G as players and substituting gradient optimization (Adam) for oracles to create a meta-population, 2) computing the NE of the resulting meta-game as an LP, 3) pruning the meta-population to keep the computation tractable, and 4) evaluating this approach on several GAN architectures and datasets of interest. I really only view the last one as a key contribution. The rest are contributions of prior work (McMahan, Lanctot, Cheng & Wellman).\n\nOverall, the value of the paper lies in applying DO to GANs. I have been eager to see this tried and I view it as a necessary piece of research. The authors confirm that this approach can indeed improve performance: better Inception & FID scores on MNIST, CIFAR-10, and Celeb-A. The training speedup supported by Figures 2 and 4 is relatively meagre. The improved robustness to mode dropping in Figure 3 is good. These aren't criticisms against the paper. However, in order to draw more general conclusions on these phenomena, I would have liked to see more architectures tried. For example, not all DC-GANs are created equal. One challenge in getting GANs to work is finding the right number of layers, neurons per layer, filter sizes, strides, etc. I'm curious to learn if DO makes GANs more robust to these choices. In my opinion that would be an important contribution.\n\nI would also like to see a discussion of training time for DO versus vanilla training. It's clear that DO is much more expensive than standard training, but by how much? Given the extra computation required, is training in 17 epochs vs 20 worth it?\n\nAs I said, there is not a lot of originality on the algorithmic side. Had the authors proposed a novel pruning scheme with superior performance on GANs, for example, that could have helped strengthen the paper.\n\nAlso, I found it interesting that the Nash distributions found in the appendix were sparse. This means that despite maintaining a population of players, we might be able to get away with a small number at test time.\n\n\nQuality:\nThe quality of the paper is at a high enough level for ICLR.\n\nClarity:\nThe paper is generally clear.\n- Please make it clear that \"generatorOracle()\" and \"discriminatorOracle()\" are approximate oracles obtained by an Adam optimizer, not \"true\" oracles.\n- Figure 3: Should show Epoch 17 in top row as well to support your argument.\n- In \"Results\" under Table 1, the authors state that DO-SGAN obtains much higher FID scores. I think you mean lower.\n\nOriginality:\nIn my opinion, the first 3 \"key\" contributions are really just 1: the idea of applying DO to GANs. This idea is novel, although the most interesting problems that must be solved in order to apply DO have already been tackled in previous work (PSRO - Lanctot 2017, Pruning - Cheng & Wellman 2007). The authors do not propose any significant modifications to the previous approaches. Also, the formulation of Nash of a 2-player zero-sum game as an LP is standard even within the recent DO / meta-game literature (e.g., \"Open-ended Learning in Symmetric Zero-sum Games\", Balduzzi 2019), so not a contribution.\n\nSignificance:\nDO provides a principled training regime for GANS, that although potentially expensive, does show meaningful improvement.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "3LmLoyInbku", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary of paper:\n\nThis paper proposes a GAN training method that involves keeping a table of historical losses for the generator and discriminator across training iterations and calculating the loss using a double-oracle framework inspired by game theory.\n\nStrengths:\n\n-- To my knowledge, this is a novel approach which uses new ideas in a related field to potentially help the unstable training of GANs.\n\n-- The background preliminaries are thoroughly explained, which is important, as much of this might be new to the target audience.\n\n-- The idea of computing losses across iterations to save on the total number of iterations that must be performed is a good one given the high computation costs of GAN training.\n\n\nWeaknesses:\n\n-- I found this paper to be poorly written and hard to understand. The game theory language used here is not standard GAN terminology and thus made it difficult for me to follow. \n\n-- If model weights themselves are compared/combined across iterations, for example, took me several read-throughs to understand because of unnecessary terminology like calling them \"policies\" and \"meta-game\" which are not used in work on GAN training.\n\n-- Not all GAN loss functions are zero-sum (in fact the best performing models e.g. Big-GAN do not use a zero-sum loss), but as I see that is required for this approach, or at least is the only thing  that is considered.\n\n-- Loss values themselves are not necessarily indicative of generative performance. There are stable equilibria that do not yield good generation, and a different equilibrium at a lower value may or may not be better. As such, only so much can be done with historical loss values.\n\n-- I don't understand how this is supposed to help mode collapse, which is claimed as a beneficial result several times in the results section. To combat mode collapse, it will have to change the way gradients are differentiated for different points. This seems prima facie unrelated to the proposed method, and if it really is causally related, that would be valuable but this needs to be investigated with deeper experimental analysis for it to be claimed.\n\n-- The experiments are weak. The toy example is entirely vacuous, as getting a regular GAN to match a handful of modes in 2D is quite possible with any number of small tricks that have been around for years and are not onerous.\n\n-- The natural image results are only on CIFAR and CelebA, and are low quality at that. Qualitatively, the generated images still look bad, and the use of a DCGAN as the main baseline is misleading as the vanilla DCGAN is nobody's standard to beat anymore. Models that have achieved state-of-the-art results in the last couple years like Big-GAN should be included.\n\n\n\nConclusion: The motivation for this particular meta-game strategy is not very clear, and the results are not good enough to be relevant to where the current state of GANs are, and as such I vote to reject.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper uses the double oracle method from game theory and applies it to GANs.

This idea is interesting and Double Oracle actually seems like a good fit to train GANs. This could lead to interesting results in the future.

Reviewers disagree on the clarity of the paper, probably because the game theory vocabulary is not something that is common among the papers published at ICLR, so extra care should be taken to explain these notions.

They also point out that the method only applies to some GANs and not all (in particular the loss needs to be zero-sum).
The experimental section is too weak and the metrics used need to be changed. Results are too far from the state of the art to be convincing. The authors based their experimental setup on DCGAN: this is too old, too many improvements have made since then. Other criticisms of the experimental section include: comparison to newer methods must be made, analysis and discussion of the results must be pushed further.

Generally, the average score of the reviews is too low for acceptance. The reviewers agree that the idea is both interesting and pertinent, but this paper cannot be published as it is now. The theoretical part mostly consists in applying double oracle to GANs, and the experimental section is too weak. At least one of these parts (preferably both) must be strengthened for this paper to be impactful.
Decision: Reject