Introduction to Generative
Adversarial Networks
Ian Goodfellow, OpenAI Research Scientist
NIPS 2016 Workshop on Adversarial Training
Barcelona, 2016-12-9

Adversarial Training•

A phrase whose usage is in flux; a new term that applies to both new andold ideas•

My current usage: “Training a model in a worst-case scenario, with inputs
chosen by an adversary”•Examples:•An agent playing against a copy of itself in a board game (Samuel, 1959)•Robust optimization / robust control (e.g. Rustem and Howe 2002)•Training neural networks on adversarial examples (Szegedy et al 2013,
Goodfellow et al 2014)(Goodfellow 2016)

Generative AdversarialNetworks•

Both players are neural networks•

Worst case input for one network is produced byanother network(Goodfellow 2016)

Generative Modeling•

Density estimation•Sample generation

Training examples

Model samples(Goodfellow 2016)

Adversarial Nets Framework
D(x) tries to benear 1D tries to makeD(G(z)) near 0,
G tries to makeD(G(z)) near 1

Diﬀerentiable
function DD

x sampled fromdata

x sampled frommodel

Diﬀerentiable
function GInput noise z(Goodfellow 2016)

Z=exp ( E(x, z))xzMinimax
x = G(z; ✓ Game)

equation(G)

J(D)

=J(G)

=1
Ex⇠pdata log D(x)2(D)
J

1Ez log (12D (G(z)))-Equilibrium is a saddle point of the discriminator loss
-Resembles Jensen-Shannon divergence-Generator minimizes the log-probability of the discriminatorbeing correct(Goodfellow 2016)

p(x) =p(x | h)p(h)maintains samples from a Markov chain from one learning step to the next in order
hin a Markov chain as part of the inner loop of learning.
The procedure is formally prithm 1.

Discriminator Strategyce, equation 1 may not provide sufficient gradient for G to learn well. Early in l
is poor, D can reject samples with high confidence because they are clearly differe
D(x)log(1for any D(G(z)))
pdata (x) andpmodel (x)Ratheris always
ing data. Optimal
In this case,saturates.
than training G to mD(G(z))) we can train G to maximize log D(G(z)).
This(x)objective function resul
pdata=ed point of the dynamics of G and D(x)D but provides
much stronger gradients early in lpdata (x) + pmodel (x)Discriminator

Estimating this ratiousing supervised learning isthe key approximation
mechanism used by GANs(a)(b)

DataModeldistribution...

x1(c)

z(d)

(Goodfellow 2016)

J(D)

J(G)

J(D)

=

J(G)

=ng

=
=

Ex⇠pdata log D(x)Ez log (12(D)J

2

1Ex⇠pdata log D(x)2
1
Ez log D (G(z))
21Ez log (12D (G(z)))Non-Saturating Game
D (G(z)))-Equilibrium no longer describable with a single loss
-Generator maximizes the log-probability of the discriminatorbeing mistaken
1-Heuristically motivated; generator can still learn even whendiscriminator successfully rejects all generator samples(Goodfellow 2016)

Vector Space Arithmetic

CHAPTER 15. REPRESENTATION LEARNING-Manwith glasses

=+Man

Woman

Woman
withGlasses
Figure 15.9: A generative model has learned a distributed representation that disentangl

he concept of gender from the concept of wearing glasses. If we begin with the repr
entation of the concept of a manwith glasses,
thensubtract the vector representing th(Radfordet al,2015)
oncept of a man without glasses, and finally add the vector representing the conce2016)
f a woman without glasses, we obtain the vector representing the concept of(Goodfellowa woma

Non-convergence•

Optimization algorithms often approach a saddle
point or local minimum rather than a globalminimum•

Game solving algorithms may not approach anequilibrium at all(Goodfellow 2016)

Mode Collapse
min max V (G, D) 6= max min V (G, D)
GDDG• D in inner loop: convergence to correct distribution
Under review as a conference paper at ICLR 2017•

G in inner loop: place all mass on most likely point

AN training on a toy 2D mixture of Gaussians
distribution after increasing numbers of training
n. The top row shows training for a GAN with(Metzet al 2016)
ut and converges to the target
distribution.
The(Goodfellow 2016)

Minibatch Features•

Add minibatch features that classify each example
by comparing it to other members of the minibatch(Salimans et al 2016)•Nearest-neighbor style features detect if a minibatch
contains samples that are too similar to each other(Goodfellow 2016)

Minibatch GAN on CIFAR

Training Data
(Salimans et al 2016)

Samples(Goodfellow 2016)

Minibatch GAN on ImageNet(Salimans et al 2016)(Goodfellow 2016)

Cherry-Picked Results(Goodfellow 2016)

Problems with Counting(Goodfellow 2016)

Problems with Perspective(Goodfellow 2016)

Problems with GlobalStructure(Goodfellow 2016)

Phillip Isola

Jun-Yan Zhu

Tinghui Zhou

Alexei A. Efros

Image to Image Translation
Berkeley AI Research (BAIR) Laboratory
University of California, Berkeley

Figure 13: Example results of our method on day!night, co{isola,junyanz,tinghuiz,efros}@eecs.berkeley.edu
InputLabels to Street Scene

input

Aerial to Map

Ground truth

Labels to Facade

OutputBW to Color

outputinput

outputinput

Day to Night

input

Input

outputinput

outputEdges to Photooutputinput

outputFigure 1: Many problems in image processing, graphics, and vision involve translating an input image into a corresponding output im
These problems are often treated with application-specific algorithms, even though the setting is always the same: map pixels to p
Conditional adversarial nets are a general-purpose solution that appears Figureto workon results
a wideof ontheseproblems.
Hereedges
we14: well
Exampleof variety
our method
automaticallydetected
results of the method on several. In each case we use the same architecture and objective, and simply train on different data.

(Isola et al 2016)(Goodfellow 2016)

Plug and Play GenerativeModels•

New state of the art generative model (Nguyen et al2016) released days before NIPS•

Generates 227x227 realistic images from all
ImageNet classes•

Combines adversarial training, moment matching,denoising autoencoders, and Langevin sampling(Goodfellow 2016)

omingGeometric IntelligenceMontreal Institute for Learning Algorithms

l.com

jason@geometric.ai

yoshua.umontreal@gmail.com

Alexey Dosovitskiy
University of Freiburg

Jeff Clune

PPGN Samples

osovits@cs.uni-freiburg.de

University of Wyoming

jeffclune@uwyo.edu

stracton, photo-realistic images hasn machine learning. Recently,
ne interesting way to synthesize
g gradient ascent in the latentrk to maximize the activationsn a separate classifier network.
method by introducing an addie, improving both sample qualding to a state-of-the-art genhigh quality images at higher
n previous generative models,ageNet categories. In addition,
ilistic interpretation of related
hods and call the general class
Generative Networks.” PPGNs
ator network G that is capable
image types and 2) a replaceC that tells the generator whate generation of images condis an ImageNet or MIT Placesalso conditioned on a caption
ing network). Our method also

(Nguyen et al 2016)

Figure 1: Images synthetically generated by Plug and Play
Generative Networks at high-resolution (227x227) for four

(Goodfellow 2016)

GANs allow many answers
Mean Squared ErrorGANs(Goodfellow 2016)

Next Video Frame Prediction

CHAPTER 15. REPRESENTATION LEARNING

Ground TruthMSE

Adversarial

Figure 15.6: Predictive generative networks provide an example of the importance oflearning which features are salient. In this example, the predictive generative network
has been trained to predict the appearance of a 3-D model of a human head at a specificviewing angle. (Left)Ground truth. This is the correct image, that the network shouldemit. (Center)Image produced by a predictive generative network trained with mean
(Lotteretal2016)squared error alone. Because the ears do not cause an extreme diﬀerence in brightness(Goodfellow 2016)

Adversarial training for people•

Markets•

Cycles due to non-convergence?

•

Auctions, taxation, etc.

•

Deliberate practice (Ericsson et al 1993)(Goodfellow 2016)

Conclusion•

Adversarial training is a term encompassing old andnew work•

GANs are a generative models that use supervised
learning to estimate a density ratio•

GANs allow a model to learn that there are many
correct answers•

Adversarial training can be useful for people as well asmachine learning models(Goodfellow 2016)