Unsupervised methods
Diving deep into autoencoders
Petar Veličković
Artificial Intelligence Group
Department of Computer Science and Technology, University of Cambridge, UK
COMPGI23—Introduction to Deep Learning, University College London

10 October 2017

Introduction

I In this lecture, I will guide you through the essentials of using

deep neural networks for unsupervised learning.
I We will focus primarily on autoencoders, which offer a goodtradeoff between model capacity and ease of training—and arestill widely used both industrially and in research.
I For completeness, we will also briefly survey two historicallyused architectures for unsupervised learning (RBMs andDBNs), and a bleeding-edge GAN architecture (more details on
Advanced Deep Learning & Reinforcement Learning. . . )The three “flavours” of machine learning

I Unsupervised learning

I Supervised learning(more details next week!)

I Reinforcement learning(more details on Advanced DL & RL)

Unsupervised learning

I The environment gives you unlabelled data—and asks you toassign useful features/structure to it.

featuresAgent

x~1 , x~2 , . . . , x~nEnvironment

I Example: study data from patients suffering from a disease, inorder to discover different (previously unknown) types of it.

Example: Clustering

What’s the point if we don’t have labels?

I As in the above example, unsupervised learning can often be a

precursor to supervised learning, if we don’t even know what
the labels should be (e.g. disease subtypes)!
I Often vastly increases the amount of data available! Obtaininglabelled data is not always:
Appropriate (e.g. if, as above, we don’t even know the labels);
Cheap (e.g. segmenting medical images);
I Feasible (e.g. clinical studies for extremely rare diseases).

I

I

I Can aid better dimensionality reduction, simplifying the work ofother algorithms, allow for synthesising new training
data. . . and much more.
I Humans are essentially learning (mostly) unsupervised!

Example: Medical image segmentation

How can unlabelled data help?

How can unlabelled data help?

How can unlabelled data help?

How can unlabelled data help?

Unsupervised learning is the future! (LeCun, 2017)

(Re)introducing neural networks and deep learning
O1

O2

×∞

I1

I2

I3

Neural networks

I To make life simpler (esp. notationally!), let’s start with a slightly

more thorough introduction to simple neural networks.
I This might restate some of the material you’ve already seen,

with the aim of making notation more consistent!
I Neural networks are structures of interconnected processing

units (neurons).
I Each neuron computes a linear combination of its inputs,

afterwards potentially applying an activation function, to
produce its output.
I Occasionally, I will illustrate how to specify neural networks ofinterest using Keras (keras.io). (highly recommended!)A single neuron
Within this context sometimes also called a perceptron (. . . )

x2
x3
..
.
xn

w2



n
P
~)=σ b+wi xi
h(~x ; w
Σσ

i=1

w3
n

x1

b
w1

w

+1

Popular choices for the activation function σ:
I Identity: σ(x) = x;
I Rectified linear unit (ReLU): σ(x) = max(0, x);
1
I Sigmoid functions: σ(x) =1+exp(−x) (logistic); σ(x) = tanh x.

Activation functions

2

Common activation functions

1.5

1

σ(z)

0.5

0

−0.5
−1

Identity
Logistic
Tanh
ReLU

−1.5

−2
−2 −1.8−1.6−1.4−1.2 −1 −0.8−0.6−0.4−0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2zNeural networks and deep learning

I It is easy to extend a single neuron to a neural network—simply

connect outputs of neurons to inputs of other neurons.
I We may do this in two ways:
I
I

Feedforward: the computation graph does not have cycles;
Recurrent: the computation graph has cycles.

I Typically we organise neural networks in a sequence of layers,

such that a single layer only processes output from theprevious layer. Everything with > 1 hidden layer is “deep”!

A few details on training

I Neural networks are trained from known (input, output)

samples. The training algorithm adapts the neurons’ weights tomaximise predictive power on the training examples.
I This is done, for a single training example (~x , y), by:~ );Computing the output of the network y 0 = h(~x ; w
Determining the loss of this output L(y , y 0 );I Computing partial derivatives of the loss with respect to eachweight, ∂∂L~ , and using these to update weights.
w
I Key words: backpropagation, stochastic gradient descent.
I
I

I More details next week!

A simple classifier
Let’s ignore the activation functions and “deep learning” fornow. . . here is a simple, shallow, 4-class classifier.
+1
x1
x2
x3
..
.
xn

Σ
Σ
Σ
Σ

Choose the class which has the maximal output:P
C = argmaxj bj + ni=1 wij xi

Block notation
Note that this layer is essentially doing a matrix multiplication. . .

~x

W×

+~b



C = argmaxj W~x + ~b
N.B. W of size 4 × n, ~b of size 4.

jSoftmax

I Problem: what should the targets be?
I Outputs are unbounded! For an example of the second class,
the targets should be ~y = −∞ +∞ −∞ −∞ . . .
I Solution: transform the outputs monotonically to the [0, 1]

range, using the softmax function:exp(zi )softmax(~z )i = P
j exp(zj )Probabilistic classification

I This conveniently also makes the outputs add up to 1, so we
can interpret yi0 = softmax(h(~x ))i = P(~x in class i).
I Now the target for an example of the second class should be~y = 0 1 0 0 (∼ one-hot encoding).
I Typically express the loss function as the cross-entropy:

L(~y , ~y 0 ) = −

K
X
i=1

where K is the number of classes.

yi log yi0

Back in business

~x

W×

+~b

softmax

Integrating into our simple classifier:




~
~
C = argmaxj softmax Wx + bjGoing deeper with LEGOTM

~x

W1 ×

+~b1


C = argmaxj

W2 ×

softmax

Making things deep is now easy. . .

+~b2



~
~
~
softmax W2 ReLU W1 x + b1 + b2
j

N.B. the ReLU is important! A composition of linear functions isitself a linear function. . . (at least in theory—thank you, OpenAI :))Fully connected layers
The “matrix-multiply–bias–activation” (sometimes also called fully
connected or Dense) layer is a common building block of neuralnetworks.

Dense(7)Dense(4)~xsoftmax

Keras code:x = Input(shape=(7,))
h = Dense(7, activation=’relu’)(x)y = Dense(4, activation=’softmax’)(h)Working with images

I Simple fully-connected neural networks (as described already)typically fail on high-dimensional datasets (e.g. images).
Treating each pixel as an independent input. . .
. . . results in h × w × d new parameters per neuron in the first
hidden layer. . .
I . . . quickly deteriorating as images become larger—requiringexponentially more data to properly fit those parameters!

I

I

I Key idea: downsample the image until it is small enough to be

tackled by such a network!
I

I

Would ideally want to extract some useful features first. . .

=⇒ exploit spatial structure!

The convolution operator

Enter the convolution operator

I Define a small (e.g. 3 × 3) matrix (the kernel, K).
I Overlay it in all possible ways over the input image, I.
I Record sums of elementwise products in a new image.

(I ∗ K)xy =h XwX
i=1 j=1

Kij · Ix+i−1,y+j−1

I This operator exploits structure—neighbouring pixels influence

one another stronger than ones on opposite corners!
I Start with random kernels—and let the network find the optimal

ones on its own!

Convolution example

0×1 1×0 1×1 1
0×0 0×1 1×0 1
0×1 0×0 0×1 1
0 0 0 10 0 1 10 1 1 01 1 0 0
I

0
1
1
1
0
0
0

0
0
1
0
0
0
0

0
0
0
0
0
0
0

∗

1 0 1
0 1 0
1 0 1

K

=

1
1
1
1
3

4
2
2
3
3

3
4
3
3
1
I∗K

4
3
4
1
1

1
3
1
1
0

Convolution example

0
0
0
0
0
0
1

1×1 1×0 1×1 0
0×0 1×1 1×0 1
0×1 0×0 1×1 1
0 0 1 10 1 1 01 1 0 01 0 0 0
I

0
0
1
0
0
0
0

0
0
0
0
0
0
0

∗

1 0 1
0 1 0
1 0 1

K

=

1
1
1
1
3

4
2
2
3
3

3
4
3
3
1
I∗K

4
3
4
1
1

1
3
1
1
0

Convolution example

0
0
0
0
0
0
1

1
0
0
0
0
1
1

1×1 1×0 0×1 0
1×0 1×1 1×0 0
0×1 1×0 1×1 1
0 1 1 01 1 0 01 0 0 00 0 0 0
I

0
0
0
0
0
0
0

∗

1 0 1
0 1 0
1 0 1

K

=

1
1
1
1
3

4
2
2
3
3

3
4
3
3
1
I∗K

4
3
4
1
1

1
3
1
1
0

Convolution example

0
0
0
0
0
0
1

1
0
0
0
0
1
1

1
1
0
0
1
1
0

1×1 0×0 0×1 0
1×0 1×1 0×0 0
1×1 1×0 1×1 0
1 1 0 01 0 0 00 0 0 00 0 0 0
I

∗

1 0 1
0 1 0
1 0 1

K

=

1
1
1
1
3

4
2
2
3
3

3
4
3
3
1
I∗K

4
3
4
1
1

1
3
1
1
0

Stacking convolutions

Downsampling (∼ max-pooling)
Convolutions light up when they detect a particular feature in aregion of the image. Therefore, when downsampling, it is a goodidea to preserve maximally activated parts. This is the inspiration
behind the max-pooling operation.

12

20

30

0

8

12

2

0

34

70

37

4

112 100 25

12

2 × 2 Max-Pool

2030

112 37

Stacking convolutions and poolings

Stacking convolutions and poolings

Rough rule of thumb: increase the depth (number of convolutions)as the height and width decrease.
FC

Softmax

Conv.

Pool

Conv.

Pool

FC

CNN representations

Three ways to examine the CNN’s internal representations:
1. Observe the learnt kernels;
2. Pass an input through the network, observe the activations;
3. Coming later in this lecture. . .

Observing kernels
I Typically, as the kernels are small, gaining useful informationfrom them becomes difficult already past the first layer.
I However, the first layer of kernels reveals something

magical. . . In almost all cases, these kernels will learn tobecome edge detectors!

Passing data through the network: Input
O1

I1

O2

I2

I3

Passing data through the network: Shallow layer
O1

I1

O2

I2

I3

Passing data through the network: Deep layer
O1

I1

O2

I2

I3

Passing data through the network: Output
O1

I1

O2

I2

I3

Issues with learning through backpropagation

I We will start by analysing a direct way in which unsupervised

techniques can aid the kind of supervised learning morecommon for deep neural networks.
I “Deep learning” was around for decades, but took a long time

to become practically usable.
I We’d start with some randomly initialised weights, present our

training examples to the network, and. . . the network wouldn’t
really learn that well.

Loss function surfaces

Beware of overfitting!
Learning the sine function
4.5
4
3.5
3
2.5
2
1.5
1

y

0.5
0
−0.5
−1
−1.5
−2
−2.5

Training set
TargetLinear fit
Degree-3
Degree-14

−3
−3.5
−4
−4.5
−2 −1.8 −1.6 −1.4 −1.2 −1 −0.8 −0.6 −0.4 −0.2

0

x

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

(In)appropriate initialisers

I It has since been determined that initialisation plays a criticalrole in neural network training stability.
I Extremely, what happens to error gradients if you initialise the

network with zero weights? Or randomly sample them with ahuge variance?
I Be good to your signals. . .
I Using an appropriate initialiser can mean the difference

between getting great results and not converging at all!

What can we do?

I Reuse a network that performs great on a much larger dataset,and fine-tune (some of) its weights.
This is the concept of transfer learning (week 7), and isfundamental to the successes of many deep learning startups. :)I Not always possible to do!

I

I I will now show how we can employ unsupervised techniques

to determine weights that are “good” for working with our input
data (regardless of what the outputs are!).
I

One of the first “success stories” of deep learning!

I In 2010, Glorot and Bengio discovered very appropriate

parameters for randomly initialising weights (week 5).
Unsupervised pre-training is very scarcely used nowadays.

Reconstruction objective

I Assume that we want to initialise a single-layer fully-connectedneural network, to be trained by backpropagation.
I We will further assume, for simplicity, that the inputs in our

training dataset are binarised (0/1).
I Without a clear target, we can assume that a good choice of

weights will cause the output to retain most of the information
about the input.
I Therefore, the weights should be chosen such that we canalso use them to reconstruct the input given the output!

Restricted Boltzmann Machine (RBM)

I A simple unsupervised stochastic extractor of binary features,~h, from binary data, ~x .
I Parametrised by a weight matrix W and bias vectors ~a and ~b totransform the data to the feature space, but also to go back!

1
I P(hj = 1|~x ) = σ (W~x + ~b)j =1+exp(−(W~x +~b)j )1
I P(xi = 1|~h) = σ (WT ~h + ~a)i =1+exp(−(WT ~h+~a)i )I Trained efficiently using contrastive divergence (Hinton, 2010).

Once trained, can use W and ~b as initial values for a neural net!

Restricted Boltzmann Machine (RBM)
~h

W

~x


P(hj = 1|~x ) = σ (W~x + ~b)j =1

1 + exp −(W~x + ~b)j

Restricted Boltzmann Machine (RBM)
~h

W

~x



P(hj = 1|~x ) = σ (W~x + ~b)j =1

1 + exp −(W~x + ~b)j

Restricted Boltzmann Machine (RBM)
~h

W

~x



P(xi = 1|~h) = σ (WT ~h + ~a)i =1

1 + exp −(WT ~h + ~a)i

RBM feature extraction (MNIST ‘2’ digits)

RBM reconstruction

Deep Belief Network (DBN)

I If we want to go deeper, we can stack additional RBMs.

I Use the layers trained so far to generate outputs (by samplingor averaging), and use those as inputs for a new RBM. This
construction is known as a deep belief network (DBN).

I Can iteratively stack as many layers as we like!

Deep Belief Network (DBN)

~h2
W2
~h1
W1
~x

Pre-training can help! (Erhan et al., 2010)

Dimensionality reduction

I We now focus on the general unsupervised problem ofdimensionality reduction—finding a way to appropriately
compress our input into a useful “bottleneck” vector of smaller
dimensionality (we often call this algorithm an encoder).
I Obvious application to supervised learning: feeding the outputof the bottleneck into a simple classifier (e.g. k-NN, SVM,
logistic regression. . . ), perhaps fine-tuning the encoder as well.
I Fundamentally, dimensionality reduction (along withappropriate interpretability) is the essence of unsupervised
learning—to compress data well, one must first understand it!

Reconstruction strikes again

I Once again—in absence of any other information (that would

be contained in labels), the best notion of “usefulness” for thebottleneck is our ability to reconstruct the input from it.
I Broadly speaking, we aim to specify two transformations:I
I

The encoder – enc : X → Z
The decoder – dec : Z → X

where X and Z are the input and code spaces, respectively(these are often simply Rn and Rm with n > m).
I Then we seek to find parameters of the encoder/decoder thatminimise the reconstruction loss:
L(~x ) = kdec(enc(~x )) − ~x k2

Principal Component Analysis

I Perhaps the simplest instance of this framework is theprincipal component analysis (PCA) algorithm.
I Encode by projecting the n-dimensional data onto a set of morthogonal axes (n ≥ m).
I To preserve the most information, always choose one of theaxes to be the direction in which the dataset has the highest
variance! Then constrain subsequent ones to be orthogonal. . .
I Preserve m axes with highest variance.

PCA in action

PCA details

I Since projection onto orthogonal axes is a linear operation, the

PCA encoder can be seen as simple matrix multiplication:enc(~x ) = W~x
where W is of size m × n.

I As this is an orthogonal transformation, its inverse (along

retained axes only) is its matrix’s transpose:dec(~z ) = WT ~z
I We therefore seek to choose W to minimise k~x − WT W~x k2 .

Can solve this explicitly (using eigenvalue analysis)!

PCA reconstruction

Limitations of PCA

Limitations of PCA

Limitations of PCA

Linear model =⇒ Incapable of capturing nonlinear manifolds!

Alternative perspective
I It should be simple to relate the operations of PCA to those of a

two-layer fully-connected neural network without activations!
I This would allow us to work in exactly the same scenario, but
train using backpropagation!
~z = enc(~x ) = W1~x + ~b1~x 0 = dec(~z ) = W2~z + ~b2
I Once again, we optimise the reconstruction loss

L(~x 0 ) = k~x 0 − ~x k2
I We have just built our first autoencoder!

Autoencoder

~z

~x 0

~x

encoder

decoder

Autoencoder

~z

~x 0

~x

encoder

decoder

Autoencoder in Keras

Dense(m)~x

Keras code (for MNIST):x = Input(shape=(784,))z = Dense(32)(x)y = Dense(784)(z)Dense(n)Autoencoder performance on MNIST

Autoencoder performance on MNIST

Going deeper. . .

I Thus far, our autoencoder was only capable of the same kindof expressivity as PCA.
I

More so, it was doing so inefficiently compared to PCA!

I However, enabling training by backpropagation means that we

can now introduce depth and nonlinearity into the model!
I This should allow us to capture complex nonlinear manifolds

more accurately. . .

Deep autoencoders
I We will introduce an additional layer of depth, ReLU

activations, and use logistic sigmoid units to reconstruct theimage (closer to 1 ∼ whiter).
~z = enc(~x ) = ReLU W2 ReLU W1~x + ~b1 + ~b2
~x 0 = dec(~z ) = σ W4 ReLU W3~z + ~b3 + ~b4
I Now we can interpret the output as the probability of each pixel

being white—can use cross-entropy as the reconstruction loss!
L(~x 0 ) = −

n
X
i=1

xi log xi0 + (1 − xi ) log(1 − xi0 )I We now have a deep autoencoder!

From autoencoders. . .

~z

~x 0

~x

encoder

decoder

. . . to deep autoencoders

~z

~x 0

~x

encoder

decoderDeep autoencoder in Keras

Dense(h 1)

Dense(m)Dense(h 2)~xKeras code (for MNIST):x = Input(shape=(784,))h 1 = Dense(128, activation=’relu’)(x)z = Dense(32, activation=’relu’)(h 1)h 2 = Dense(128, activation=’relu’)(z)y = Dense(784, activation=’sigmoid’)(h 2)

Dense(n)Deep autoencoder performance on MNISTDeep autoencoder performance on MNISTDeep(er) autoencoder performance on MNIST

Reintroducing convolutional layers

I But. . . we’ve been working with images all this time! Didn’t I

say that fully-connected layers are bad for images?
I They are indeed—MNIST is small (28 × 28 × 1). We can’t gomuch further with only fully-connected layers.

I Luckily, inserting convolutional and pooling layers into

autoencoders is not a major issue.
I How do we decode (especially from a downsampled image)?

Towards the deconvolutional layer

I Essentially, we want to transform the output of a convolution

into something of the shape of its input, while maintaining thedesired connectivity patterns.
I Luckily (and perhaps surprisingly), the backpropagation updateof a convolutional layer is itself a convolution!
I

Details omitted, to follow in week 3 (but illustrations incoming)!

I This means that we can carefully craft a (potentially strided)

convolutional layer that will behave like a “target” convolutional
layer in the backwards direction!
I This forms the basis of the deconvolutional layer, which isextremely useful across a variety of applications of deeplearning to computer vision!

Deconvolution

simple convolution∗=

deconvolution

Deconvolution

simple convolution∗=

deconvolution

Deconvolution

simple convolution∗=

deconvolution

Convolutional autoencoder

I Now we stack convolutional and pooling layers in the encoder,and deconvolutional layers in the decoder. All other details
remain unchanged.

z = enc(X)= Pool (ReLU (Conv (Pool (ReLU (Conv (X, K1 ))) , K2 )))X0 = dec(z) = σ (Deconv (ReLU (Deconv (z, K3 )) , K4 ))Convolutional autoencoder (CAE)

X

Conv.

Pool

Conv.
ReLU

ReLU

encoderPoolz

Deconv.

Deconv.

ReLU

logistic

decoderX′

Convolutional autoencoder (CAE)

X

Conv.

Pool

Conv.
ReLU

ReLU

encoderPoolz

Deconv.

Deconv.

ReLU

logistic

decoderX′

Convolutional autoencoder in Keras: Encoderx = Input(shape=(28, 28, 1))x up = ZeroPadding2D((2, 2))(x)h 1 = Conv2D(16, (3, 3), padding=’same’,
activation=’relu’)(x up)
p 1 = MaxPooling2D((2, 2))(h 1)h 2 = Conv2D(8, (3, 3), padding=’same’,
activation=’relu’)(p 1)
p 2 = MaxPooling2D((2, 2))(h 2)h 3 = Conv2D(8, (3, 3), padding=’same’,
activation=’relu’)(p 2)z = MaxPooling2D((2, 2))(h 3)Convolutional autoencoder in Keras: Decoder

h 4 = Conv2DTranspose(8, (3, 3), padding=’same’,strides=(2, 2), activation=’relu’)(z)h 5 = Conv2DTranspose(16, (3, 3), padding=’same’,strides=(2, 2), activation=’relu’)(h 4)y up = Conv2DTranspose(1, (3, 3), padding=’same’,strides=(2, 2), activation=’sigmoid’)(h 5)y = Cropping2D((2, 2))(y up)

Convolutional autoencoder performance on MNISTConvolutional autoencoder performance on MNISTwith significantly fewer parameters (∼3K vs. ∼220K)!

Intermediate layers (Turchenko et al., 2017)

The code is discriminative (Turchenko et al., 2017)

Applying autoencoders to eliminating noise

I One popular application of autoencoders is denoising.

I Namely, we may wish to be able to reconstruct our input while

simultaneously eliminating any noise present within it!
I This comes from several motivations:

Real-world data is often noisy;
Combatting overfitting the training data;
I Learning more robust representations!
I

I

Denoising autoencoder

I Constructing a denoising autoencoder (of any kind!) requires

modifying two aspects of the framework.
I Firstly, the input data ~x is corrupted by a corruption process (forimages, this could be e.g. injecting Gaussian noise):~x̃ = corrupt(~x )I Then, the network is judged on how well it reconstructs theoriginal input from the corrupted input:
L(~x ) = kdec(enc(corrupt(~x ))) − ~x k2(similarly if cross-entropy is used)Denoising autoencoder (DAE)

~z

~x 0

~x

encoder

decoderDenoising autoencoder (DAE)

∼
~z

∼
∼
~x

∼

~x 0

∼
∼
∼
corrupt

encoder

decoderDenoising convolutional autoencoder in Keras

x = Input(shape=(28, 28, 1))
x c = GaussianNoise(0.5)(x)h 1 = Conv2D(32, (3, 3), padding=’same’,
activation=’relu’)(x c)p 1 = MaxPooling2D((2, 2))(h 1)h 2 = Conv2D(32, (3, 3), padding=’same’,
activation=’relu’)(p 1)z = MaxPooling2D((2, 2))(h 2)h 3 = Conv2DTranspose(32, (3, 3), padding=’same’,strides=(2, 2), activation=’relu’)(z)y = Conv2DTranspose(1, (3, 3), padding=’same’,strides=(2, 2), activation=’sigmoid’)(h 3)Denoising autoencoder performance on MNIST

Denoising autoencoder performance on MNIST

Robust representations! (Goodfellow et al., 2016)

Robust representations! (Alain and Bengio, 2014)

Aside: encoder-decoder architectures

I Strictly speaking, a denoising autoencoder is not performing

autoencoding, i.e. it’s not trying to reconstruct its exact input.
I We can generalise this idea further, to make our network havea similar structure to an autoencoder, but actually perform
some useful transformation on the input!
I This gives rise to encoder-decoder architectures, which are

now used across the board for tasks such as segmentation.

An encoder-decoder architecture for segmentation

SegNet (Kendall et al., 2015)

Generative modelling returns

I To conclude our exploration of autoencoders, we will focus ourattention on generative models.
I The autoencoder architectures covered so far will do a good

job at learning an appropriate dimensionality reduction of
inputs similar to our training set’s.
I But what does this tell us about the underlying properties of thedata? Can we plug in arbitrary codes ~z into the decoder andexpect useful results?

Generative modelling“What I cannot create, I do
not understand.”—Richard Feynman

A probabilistic generative framework

I We will think of the code, ~z , as an object that captures theessential properties of our input data, ~x .
I We can make it easy to generate new, useful, codes—namely,

we can engineer the prior distribution of codes P(~z ) to be a
“tractable” probability distribution (such as a N (~0, I), the unit
Gaussian).
I Once we have the code, generating data implies sampling theprobability distribution P(~x |~z ). This can also be designed to be
tractable (e.g. a learnt decoder neural network).
I This allows us to generate new data points!

A probabilistic generative framework

I Let’s assume that the code prior P(~z ) and the generating

distribution P(~x |~z ) are both parametrised by θ (these could be,say, the weights of a neural network).
I We will denote the prior by pθ (~z ) and the generating

distribution by pθ (~x |~z ) to make this explicit.
I What do we have so far?

pθ (~z )~zpθ (~x |~z )decoder~xLearning the generative framework

I Note that, having specified pθ (~z ) and pθ (~x |~z ), we can also

derive an expression for P(~x ) = pθ (~x )!

I This roughly corresponds to: “how likely is it that our model will

generate ~x ?” and sounds like a great objective to maximise!
I We could train our model to maximise pθ (~x ) over all thesamples ~x in our training set.

Maximising the evidence

I Unfortunately, computing this quantity (sometimes called theevidence) requires integrating over all possible codes:ZZpθ (~x ) = P(~x ) =P(~x , ~z ) d~z =pθ (~x |~z )pθ (~z ) d~z
~z ∈Z

~z ∈Z

and this is intractable in all except the simplest of cases!
I We could try approximating it using Monte Carlo methods, but

this does not scale for large datasets and large networks.
I It is possible to make this objective work—and we will see how!

For now, we need to address an even more serious issue. . .

Performing inference

I For inference purposes, we also would very much like to beable to attach codes to known inputs ~x .
I This would allow us to use the code for other purposes, suchas dimensionality reduction, obtaining new inputs similar to ~x(e.g. for data augmentation) by modifying the code, etc.
I This involves sampling codes from the posterior distribution

P(~z |~x ), and is almost always insanely hard!

Why is the posterior so hard?

I At a glance, it might seem like we could use Bayes’ theorem tohelp us evaluate the posterior:pθ (~z |~x ) = P(~z |~x ) =P(~x |~z )P(~z )pθ (~x |~z )pθ (~z )=P(~x )pθ (~x )I However, the pesky evidence reappears in the denominator :(and this time we need to sample from something involving it,
not just evaluate it. . .

Variational inference

I This issue is circumvented by employing variational inference.
I We accept that the posterior is hard, and rather than explicitly

sampling it, we choose to sample from something simpler.
I Generally, we choose to approximate the posterior with arecognition model, qφ (~z |~x ), parametrised by φ. Typically, thiswill also be a neural network, with φ being its weights!
I The parameters φ should be selected to make qφ (~z |~x ) as close

to the true posterior pθ (~z |~x ) as possible!

Variational autoencoder (Kingma & Welling, 2015)

I Our model of the world now looks like this:~xqφ (~z |~x )encoder~zpθ (~x |~z )decoder~x 0

I Looks like an autoencoder, doesn’t it?
I Indeed—when pθ and qφ are specified by neural networks, thisis the general setup of a variational autoencoder (VAE).
I The loss function is now a bit more complicated. . .

Towards a VAE loss
I As discussed, a VAE has two objectives that need to be

simultaneously satisfied. For a training example, ~x :I

We want to make our decoder highly likely to generate thisexample—this implies maximisinglog pθ (~x )I

Simultaneously, we would like our encoder to not stray too far
from the true posterior pθ (~z |~x ). This implies minimising
DKL (qφ (~z |~x )kpθ (~z |~x ))where DKL is the Kullback-Leibler (KL) divergence:Z
q(x)DKL (q(x)kp(x)) =q(x) log
dx
p(x)
x∈X

The evidence lower bound
I Combined, these objectives give us the evidence lower

bound (ELBO), which our network needs to maximise:
ELBO(θ, φ) = log pθ (~x ) − DKL (qφ (~z |~x )kpθ (~z |~x ))I The troublesome posterior pθ (~z |~x ) is still here. . .

I Luckily, we can rewrite ELBO in a form that completely

eliminates the posterior!
ELBO(θ, φ) = E~z ∼qφ (~z |~x ) [log pθ (~x |~z )] − DKL (qφ (~z |~x )kpθ (~z ))
{z
}
|
{z
} |
Reconstruction accuracy

Regularisation

Derivation to follow in the next two slides—likely omitted.

ELBO derivation
Recall: ELBO(θ, φ) = log pθ (~x ) − DKL (qφ (~z |~x )kpθ (~z |~x ))Z
log pθ (~x ) =qφ (~z |~x ) log pθ (~x ) d~z~z ∈ZZpθ (~x , ~z )d~z=qφ (~z |~x ) logpθ (~z |~x )~z ∈ZZpθ (~x , ~z ) qφ (~z |~x )=qφ (~z |~x ) log
d~zqφ (~z |~x ) pθ (~z |~x )
~z ∈Z
Z
Z
qφ (~z |~x )pθ (~x , ~z )=qφ (~z |~x ) logd~z +qφ (~z |~x ) log
d~zqφ (~z |~x )pθ (~z |~x )
~z ∈Z
~z ∈Z
Z
pθ (~x , ~z )=qφ (~z |~x ) log
d~z + DKL (qφ (~z |~x )kpθ (~z |~x ))qφ (~z |~x )~z ∈ZLooks like we can cancel out the term containing the posterior! :)ELBO derivation
Recall: ELBO(θ, φ) = log pθ (~x ) − DKL (qφ (~z |~x )kpθ (~z |~x ))Zpθ (~x , ~z )log pθ (~x ) =d~z + DKL (qφ (~z |~x )kpθ (~z |~x ))qφ (~z |~x ) log
qφ (~z |~x )~z ∈ZFinally,pθ (~x , ~z )d~zqφ (~z |~x )~z ∈ZZpθ (~x |~z )pθ (~z )=qφ (~z |~x ) log
d~zqφ (~z |~x )
~z ∈Z
Z
Z
~
~
~
~
~
=
qφ (z |x ) log pθ (x |z ) dz −Z

ELBO(θ, φ) =~z ∈Zqφ (~z |~x ) log~z ∈Zqφ (~z |~x ) log= E~z ∼qφ (~z |~x ) [log pθ (~x |~z )] − DKL (qφ (~z k~x )kpθ (~z ))qφ (~z |~x )d~zpθ (~z )The ELBO function—analysed

ELBO(θ, φ) = E~z ∼qφ (~z |~x ) [log pθ (~x |~z )] − DKL (qφ (~z |~x )kpθ (~z ))
{z
}
|
{z
} |
Reconstruction accuracy

Regularisation

I The first term can be interpreted as the usual autoencoder

reconstruction loss—given input ~x , use the encoder to sample
a code ~z from qφ (~z |~x ), then use the decoder to computepθ (~x |~z ). We want to maximise this value!
I Minimising the second term forces the distribution of generated

codes to remain close to the prior—prevents the network from“cheating” (by, e.g., assigning distant codes to every example)!

Practical choice of pθ (~x |~z )I Now I will present a specific example of a VAE implementation(which is also most common).
I Our decoder will generate samples ~x 0 coming from a Gaussian

distribution, where each component is sampled independently
with standard deviation one.
I Essentially, pθ (~x |~z ) = N (~µ, I).
I For computing the mean sample µ~ , we will utilise a neuralnetwork (with weights θ) of exactly the same kind as
before—with logistic output units.
I We can then use the cross-entropy of the encoder input ~x andµ~ as the reconstruction loss (just as before).

Practical choice of qφ (~z |~x )I For the encoder, we will use m Gaussian distributions togenerate each of the m elements of the code.
I The parameters of these distributions (~µ, ~σ ) are computed by aneural network (with weights φ and linear output units). Thisimplies qφ (~z |~x ) = N (~µ, diag(~σ 2 )).
I If we use the prior pθ (~z ) = N (~0, I), then the regularisation term

has an expression we can easily work with!
−DKL (N (~µ, diag(~σ 2 ))kN (~0, I)) =

m
X
j=1

1 + log σj2 − µ2j − σj2Variational autoencoder (VAE)

~z

~x 0

~x

encoder

decoderVariational autoencoder (VAE)
µ

µ

µ
µ
µ

~z

µ
µ
µ

~x
σ

µ

σ
σ
σ
encoderµµdecoder~x 0

Reparametrisation trick

I Naı̈vely plugging in the sampling operation into the encoder will

not work, because the sampling operation has no gradient!
I To make it work, we can instead sample upfront ~ε ∼ N (~0, I),

feed it as an external input, and transform appropriately:~z = µ~ + ~ε~σ

def sampling(args):
z mean, z log var = args
epsilon = K.random normal(shape=(batch size, m),mean=0., stddev=1.)
return z mean + K.exp(z log var) * epsilonz = Lambda(sampling)([z mean, z log var])Generative power of VAEs on MNIST (m = 2)

Using VAEs to generate fake faces

https://youtu.be/XNZIN7Jh3Sg

Combatting entanglement
Can we make sense of what the code represents?
What happens if we shift one of the code elements slightly?

Tackling the black box problem

I Neural networks have achieved state-of-the-art results across avariety of domains, but interpreting their exact mode ofoperation remains difficult.
I For some domains (e.g. autonomous vehicles or medicine),interpretability is key!
I Autoencoders typically suffer from the entanglement

problem—each code element encodes “a little bit of everything”
about the output! Shifting one code element ends up blurring
the entire result. . .
I Making sense of entangled codes is hard!

Disentangled VAE (Higgins et al., 2016)

I Disentanglement parameter β ≥ 0 – controls the relative

importance of the reconstruction accuracy and theregularisation error.
I

Scales the pressure put on qφ (~z |~x ) to approximate pθ (~z ):L(θ, φ) = E~z ∼qφ (~z |~x ) [log pθ (~x |~z )] − β DKL (qφ (~z |~x )kpθ (~z ))I Under some continuity assumptions in the training data, thismodel is capable of learning to disentangle its representations(for large enough β)!
I (Recall: in our case, pθ (~z ) = N (~0, I)—so the prior has nocovariance between code elements!)

Example Toy Problem (Peychev et al., unpublished)

267,021 synthetically generated binary images in total (size: 64 x64) after removing duplicates. Input generative factors:
I Shape (ellipse, square, triangle)
I Position X and Y (16 values each)
I Scale (6 values)
I Rotation (60 values over the [0; π] range)

Results (β = 4)

Learnt means of each code element zi as a function of all 16x16
locations, averaged across objects, rotations and scales.
position y

z0

z1

z2

z3

z4

z5

z6

z7

z8

z9

15

15

15

15

15

15

15

15

15

15

10

10

10

10

10

10

10

10

10

10

5

5
5

10 15

5
5

10 15

5
5

10 15

5
5

10 15

5
5

5

5

10 15

5

10 15

5

0

0.5

1

1.5

10 15

position x

-2

-1.5

-1

-0.5

2

5
5

10 15

5
5

10 15

5

10 15

Results (β = 4)

(scaled, averaged)
learnt mean

Learnt means of each code element zi as a function of the scale
factor, averaged across rotations and positions.
z0

1

z1

1

0
-1
5

5

5

5

5

5

5

0

-1
0

5

z9

1

0

-1
0

z8

1

0

-1
0

z7

1

0

-1
0

z6

1

0

-1
0

z5

1

0

-1
0

z4

1

0

-1
0

z3

1

0

-1
0

z2

1

0

-1
0

5

0

5

scaling level(scaled, averaged)
learnt mean

Learnt means of each code element zi as a function of the rotation
factor, averaged across scales and positions.
z0

1

z1

1

0
-1
50

50

50

50

50

50

50

0

-1
0

50

z9

1

0

-1
0

z8

1

0

-1
0

z7

1

0

-1
0

z6

1

0

-1
0

z5

1

0

-1
0

z4

1

0

-1
0

z3

1

0

-1
0

z2

1

0

-1
0

50

0

rotation shift

The colours correspond to shapes (ellipse, square, triangle).

50

Results (β = 0)
I This autoencoder learns an entangled representation.
position y

z0

z1

z2

z3

z4

z5

z6

z7

z8

z9

15

15

15

15

15

15

15

15

15

15

10

10

10

10

10

10

10

10

10

10

5

5

5

5

5

5

5

5

5

5

10 15

5

10 15

5

10 15

5

10 15

5

10 15

5

10 15

5

10 15

5

5

10 15

5

10 15

5

10 15

position x(scaled, averaged)
learnt mean(scaled, averaged)
learnt mean

-10

z0

1

z1

1

z2

1

-5

z3

1

0

z4

1

5

z5

1

10

z6

1

z7

1

z8

1

0

0

0

0

0

0

0

0

0

0

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

0

5

0

5

0

5

0

5

0

5

0

5

0

5

0

5

z9

1

0

5

0

5

scaling level
z0

1

z1

1

0

1

0

-1
50

rotation shift

50

50

50

50

50

50

0

-1
0

50

z9

1

0

-1
0

z8

1

0

-1
0

z7

1

0

-1
0

z6

1

0

-1
0

z5

1

0

-1
0

z4

1

0

-1
0

z3

1

0

-1
0

z2

-1
0

50

0

50

Shifting the code (β = 4)

Shifting the code (β = 0)

Disentanglement as a function of β
Fully connected, simple autoencoder

0.8

0.8

0.7
0.6
0.5
0.4
0.3
0.2
0.1

0.7
0.6
0.5
0.4
0.3
0.2
0.1

0

0
0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

Convolutional, simple autoencoder

1

0.5

1

0.9

0.9

0.8

0.8

0.7
0.6
0.5
0.4
0.3
0.2
0.1

1.5

2

2.5

3

3.5

4

4.5

5

4.5

5

Convolutional, denoising autoencoder

1

classifier accuracy / disentanglement levelclassifier accuracy / disentanglement level

Fully connected, denoising autoencoder

1
0.9

classifier accuracy / disentanglement levelclassifier accuracy / disentanglement level

1
0.9

0.7
0.6
0.5
0.4
0.3
0.2
0.1

0

0
0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

0

0.5

1

1.5

2

2.5

3

3.5

4

Classification with disentangled VAEs
I Using the VAE’s code to classify on MNIST – does not explicitly

satisfy the required assumptions made for the disentangled
autoencoder to be successful, but tradeoffs evident.
Using disentangled fully connected
autoencoder for MNIST classification

Using disentangled convolutional
autoencoder for MNIST classification

1

MNIST classification accuracy

MNIST classification accuracy

1

0.95

0.9

0.85

0.95

0.9

0.85
0

0.5

1

1.5

2

2.5

3

3.5

4

0

0.5

1

1.5

2

2.5

3

3.5

4

Disentangling Atari (Higgins et al., 2016)

http://tinyurl.com/jgbyzke

Generative Adversarial Networks

I We will conclude with a brief overview of a bleeding-edge

generative modelling approach—one of the most popular ideas
to hit deep learning in the past decade.
I “The most important one, in my opinion, is adversarial training(also called GAN for Generative Adversarial Networks). This,and the variations that are now being proposed is the mostinteresting idea in the last 10 years in ML, in my opinion.”—Yann LeCun

GANs are everywhereA probabilistic generative framework, revisited

I Recall the basic generative framework from before. . .

pθ (~z )G(~z )~zgenerator~x(N.B. pθ (~x |~z ) is replaced by G(~z ), and is deterministic!)
I What we really want to do is make the distribution of thisgenerator approach the true data distribution, pdata (~x ).
I With VAEs, we used the KL-divergence as an objective for

enforcing one distribution to approach another.
I
I

Tractable only if we know our target distribution!
But we only have empirical samples from pdata (~x ). . .

Simplifying the distributions

I Looking over full data distributions (e.g. individual image pixels)

is hard.
I We can utilise a neural network to extract underlying featuresfrom inputs of these two distributions, and examine those.
I If these features are “good enough”, the network should becapable of telling the two distributions apart!
I Call this network the discriminator, D(~x ).
I Essentially, a binary classifier, telling whether ~x came frompdata (~x ) or G(~z ). N.B. The discriminator effectively specifies
the loss function we’re optimising!

The GAN framework

pdata (~x )~xrealD(~x )~xpθ (~z )G(~z )~zgenerator

discriminatorreal?

~xfakeTwo neural networks playing a game. . .
Alternate updating their weights; hopefully they improve together!

The GAN framework—update step 1

pdata (~x )~xrealD(~x )~xpθ (~z )G(~z )~zgenerator

discriminatorreal?

~xfakeTrain discriminator to maximise probability of ‘real’ on real data.

The GAN framework—update step 2

pdata (~x )~xrealD(~x )~xpθ (~z )G(~z )~zgenerator

discriminatorreal?

~xfakeTrain discriminator to maximise probability of ‘fake’ on fake data.

The GAN framework—update step 3

pdata (~x )~xrealD(~x )~xpθ (~z )G(~z )~zgenerator

discriminatorreal?

~xfakeTrain generator to maximise probability of ‘real’ on fake data.

The desired final outcome

min max V (D, G) = E~x ∼pdata (~x ) [log D(~x )] + E~z ∼pθ (~z ) [log(1 − D(G(~z )))]GD

I Assuming everything goes well, concluding the training

process we obtain two very useful networks!
The generator, G(~z ), becomes capable of generating extremely
useful examples, which can then be used for data augmentation.
I The discriminator, D(~x ), becomes a high-quality feature
extractor from data, which can then be used for the usual
supervised learning tasks.

I

I The exact update rule we use in the above scenarios will

depend on the distance metric between pdata (~x ) and G(~z )that we seek to optimise.

Two potential distance metrics

I The Jensen-Shannon (JS) divergence, DJS :DJS (q(x)kp(x)) =1
1
DKL (q(x)km(x)) + DKL (p(x)km(x))
22where m(x) = 12 (p(x) + q(x))
Optimised in the original GAN paper (Goodfellow et al., 2014),
assuming optimal discriminator.
I Therefore, want the discriminator to be very good.
I However, suffers from vanishing gradients when discriminator is
too good (making the generator stop improving). Tradeoffs?!

ITwo potential distance metrics

I The Wasserstein/Earth Mover (EM) distance, W :W (p(x), q(x)) =infγ∈Π(p,q)E(x,y)∼γ [kx − yk]

Takes into account the underlying geometry of the distributions
Indicates the cost of transforming p into q under an “optimaltransport plan”.
I Intractable in this form (but various interesting developments)!

I
I

Aside: Why Wasserstein is desirable

KL-divergence is +∞, JS-divergence is a constant (log 2),
Wasserstein distance is equal to the distance between the lines!

A deep convolutional GAN (Radford et al., 2015)

Architectures like this allow us to create some really interesting
applications with image distributions. . .

Generating fake celebrities (Hjelm et al., 2017)

Domain transfer: CycleGAN (Zhu et al., 2017)https://junyanz.github.io/CycleGAN/

An overview of historical deep learning ideas

I Initially, we needed to extract hand-crafted features before

applying a machine learning model to them.
I

Deep neural networks can perform feature extraction bythemselves.

I Then, we needed to select a hand-crafted loss function to

optimise.
I

GANs use a neural network (the discriminator) to compute acustomised loss!

I We need to figure out a correct way to perform the optimisationof the loss function.
I

Learn how to learn?

Thank you!

Questions?
petar.velickovic@cst.cam.ac.uk
Special thanks:
Momchil Peychev (University of Cambridge)
Devon Hjelm (Montréal Institute for Learning Algorithms)
Reading material:‘Deep Learning’, Chapter 13 (PCA)‘Deep Learning’, Chapter 14 (AEs, DAEs)‘Deep Learning’, Chapter 20 (RBMs, DBNs, VAEs, GANs)