Paper:
We focus on generative autoencoders, such as variational or adversarial autoencoders, which jointly learn a generative model alongside an inference model. Generative autoencoders are those which are trained to softly enforce a prior on the latent distribution learned by the inference model. We call the distribution to which the inference model maps observed samples, the learned latent distribution, which may not be consistent with the prior. We formulate a Markov chain Monte Carlo (MCMC) sampling process, equivalent to iteratively decoding and encoding, which allows us to sample from the learned latent distribution. Since, the generative model learns to map from the learned latent distribution, rather than the prior, we may use MCMC to improve the quality of samples drawn from the generative model, especially when the learned latent distribution is far from the prior. Using MCMC sampling, we are able to reveal previously unseen differences between generative autoencoders trained either with or without a denoising criterion.
1 INTRODUCTION
Unsupervised learning has benefited greatly from the introduction of deep generative models. In particular, the introduction of generative adversarial networks (GANs) (Goodfellow et al., 2014) and variational autoencoders (VAEs) (Kingma & Welling, 2014; Rezende et al., 2014) has led to a plethora of research into learning latent variable models that are capable of generating data from complex distributions, including the space of natural images (Radford et al., 2015). Both of these models, and their extensions, operate by placing a prior distribution, P (Z), over a latent space Z ⊆ Rb, and learn mappings from the latent space, Z, to the space of the observed data, X ⊆ Ra. We are interested in autoencoding generative models, models which learn not just the generative mapping Z 7→ X , but also the inferential mapping X 7→ Z. Specifically, we define generative autoencoders as autoencoders which softly constrain their latent distribution, to match a specified prior distribution, P (Z). This is achieved by minimising a loss, Lprior, between the latent distribution and the prior. This includes VAEs (Kingma & Welling, 2014; Rezende et al., 2014), extensions of VAEs (Kingma et al., 2016), and also adversarial autoencoders (AAEs) (Makhzani et al., 2015). Whilst other autoencoders also learn an encoding function, e : Ra → Z, together with a decoding function, d : Rb → X , the latent space is not necessarily constrained to conform to a specified probability distribution. This is the key distinction for generative autoencoders; both e and d can still be deterministic functions (Makhzani et al., 2015).
The functions e and d are defined for any input from Ra and Rb respectively, however the outputs of the functions may be constrained practically by the type of functions that e and d are, such that e maps to Z ⊆ Rb and d maps to X ⊆ Ra. During training however, the encoder, e is only fed with training data samples, x ∈ X and the decoder, d is only fed with samples from the encoder, z ∈ Z, and so the encoder and decoder learn mappings between X and Z.
The process of encoding and decoding may be interpreted as sampling the conditional probabilities Qφ(Z|X) and Pθ(X|Z) respectively. The conditional distributions may be sampled using the encoding and decoding functions e(X;φ) and d(Z; θ), where φ and θ are learned parameters of the
encoding and decoding functions respectively. The decoder of a generative autoencoder may be used to generate new samples that are consistent with the data. There are two traditional approaches for sampling generative autoencoders:
Approach 1 (Bengio et al., 2014):
x0 ∼ P (X), z0 ∼ Qφ(Z|X = x0), x1 ∼ Pθ(X|Z = z0)
where P (X) is the data generating distribution. However, this approach is likely to generate samples similar to those in the training data, rather than generating novel samples that are consistent with the training data.
Approach 2 (Kingma & Welling, 2014; Makhzani et al., 2015; Rezende et al., 2014): z0 ∼ P (Z), x0 ∼ Pθ(X|Z = z0)
where P (Z) is the prior distribution enforced during training and Pθ(X|Z) is the decoder trained to map samples drawn from Qφ(Z|X) to samples consistent with P (X). This approach assumes that∫ Qφ(Z|X)P (X)dX = P (Z), suggesting that the encoder maps all data samples from P (X)
to a distribution that matches the prior distribution, P (Z). However, it is not always true that∫ Qφ(Z|X)P (X)dX = P (Z). Rather Qφ(Z|X) maps data samples to a distribution which we
call, P̂ (Z): ∫ Qφ(Z|X)P (X)dX = P̂ (Z)
where it is not necessarily true that P̂ (Z) = P (Z) because the prior is only softly enforced. The decoder, on the other hand, is trained to map encoded data samples (i.e. samples from∫ Qφ(Z|X)P (X)dX) to samples from X which have the distribution P (X). If the encoder maps
observed samples to latent samples with the distribution P̂ (Z), rather than the desired prior distribution, P (Z), then: ∫
Pθ(X|Z)P (Z)dZ 6= P (X)
This suggests that samples drawn from the decoder, Pθ(X|Z), conditioned on samples drawn from the prior, P (Z), may not be consistent with the data generating distribution, P (X). However, by conditioning on P̂ (Z): ∫
Pθ(X|Z)P̂ (Z)dZ = P (X)
This suggests that to obtain more realistic generations, latent samples should be drawn via z ∼ P̂ (Z) rather than z ∼ P (Z), followed by x ∼ Pθ(X|Z). A limited number of latent samples may be drawn from P̂ (Z) using the first two steps in Approach 1 - however this has the drawbacks discussed in Approach 1. We introduce an alternative method for sampling from P̂ (Z) which does not have the same drawbacks.
Our main contribution is the formulation of a Markov chain Monte Carlo (MCMC) sampling process for generative autoencoders, which allows us to sample from P̂ (Z). By iteratively sampling the chain, starting from an arbitrary zt=0 ∈ Rb, the chain converges to zt→∞ ∼ P̂ (Z), allowing us to draw latent samples from P̂ (Z) after several steps of MCMC sampling. From a practical perspective, this is achieved by iteratively decoding and encoding, which may be easily applied to existing generative autoencoders. Because P̂ (Z) is optimised to be close to P (Z), the initial sample, zt=0 can be drawn from P (Z), improving the quality of the samples within a few iterations.
When interpolating between latent encodings, there is no guarantee that z stays within high density regions of P̂ (Z). Previously, this has been addressed by using spherical, rather than linear interpolation of the high dimensional Z space (White, 2016). However, this approach attempts to keep z
within P (Z), rather than trying to sample from P̂ (Z). By instead applying several steps of MCMC sampling to the interpolated z samples before sampling Pθ(X|Z), unrealistic artifacts can be reduced (see Figure 2). Whilst most methods that aim to generate realistic samples from X rely on adjusting encodings of the observed data (White, 2016), our use of MCMC allows us to walk any latent sample to more probable regions of the learned latent distribution, resulting in more convincing generations. We demonstrate that the use of MCMC sampling improves generations from both VAEs and AAEs with high-dimensional Z; this is important as previous studies have shown that the dimensionality of Z should be scaled with the intrinsic latent dimensionality of the observed data.
Our second contribution is the modification of the proposed transition operator for the MCMC sampling process to denoising generative autoencoders. These are generative autoencoders trained using a denoising criterion, (Seung, 1997; Vincent et al., 2008). We reformulate our original MCMC sampling process to incorporate the noising and denoising processes, allowing us to use MCMC sampling on denoising generative autoencoders. We apply this sampling technique to two models. The first is the denoising VAE (DVAE) introduced by Im et al. (2015). We found that MCMC sampling revealed benefits of the denoising criterion. The second model is a denoising AAE (DAAE), constructed by applying the denoising criterion to the AAE. There were no modifications to the cost function. For both the DVAE and the DAAE, the effects of the denoising crtierion were not immediately obvious from the initial samples. Training generative autoencoders with a denoising criterion reduced visual artefacts found both in generations and in interpolations. The effect of the denoising criterion was revealed when sampling the denoising models using MCMC sampling.
2 BACKGROUND
One of the main tasks in machine learning is to learn explanatory factors for observed data, commonly known as inference. That is, given a data sample x ∈ X ⊆ Ra, we would like to find a corresponding latent encoding z ∈ Z ⊆ Rb. Another task is to learn the inverse, generative mapping from a given z to a corresponding x. In general, coming up with a suitable criterion for learning these mappings is difficult. Autoencoders solve both tasks efficiently by jointly learning an inferential mapping e(X;φ) and generative mapping d(Z; θ), using unlabelled data from X in a self-supervised fashion (Kingma & Welling, 2014). The basic objective of all autoencoders is to minimise a reconstruction cost, Lreconstruct, between the original data, X , and its reconstruction, d(e(X;φ); θ). Examples of Lreconstruct include the squared error loss, 12 ∑N n=1 ‖d(e(xn;φ); θ) − xn‖2, and the cross-entropy loss, H[P (X)‖P (d(e(X;φ); θ))] =
− ∑N n=1 xn log(d(e(xn;φ); θ)) + (1− xn) log(1− d(e(xn;φ); θ)).
Autoencoders may be cast into a probablistic framework, by considering samples x ∼ P (X) and z ∼ P (Z), and attempting to learn the conditional distributions Qφ(Z|X) and Pθ(X|Z) as e(X;φ) and d(Z; θ) respectively, with Lreconstruct representing the negative log-likelihood of the reconstruction given the encoding (Bengio, 2009). With any autoencoder, it is possible to create novel x ∈ X by passing a z ∈ Z through d(Z; θ), but we have no knowledge of appropriate choices of z beyond those obtained via e(X;φ). One solution is to constrain the latent space to which the encoding model maps observed samples. This can be achieved by an additional loss, Lprior, that penalises encodings far away from a specified prior distribution, P (Z). We now review two types of generative autoencoders, VAEs (Kingma & Welling, 2014; Rezende et al., 2014) and AAEs (Makhzani et al., 2015), which each take different approaches to formulating Lprior.
2.1 GENERATIVE AUTOENCODERS
Consider the case where e is constructed with stochastic neurons that can produce outputs from a specified probability distribution, and Lprior is used to constrain the distribution of outputs to P (Z). This leaves the problem of estimating the gradient of the autoencoder over the expectation EQφ(Z|X), which would typically be addressed with a Monte Carlo method. VAEs sidestep this by constructing latent samples using a deterministic function and a source of noise, moving the source of stochasticity to an input, and leaving the network itself deterministic for standard gradient calculations—a technique commonly known as the reparameterisation trick (Kingma & Welling, 2014). e(X;φ) then consists of a deterministic function, erep(X;φ), that outputs parameters for a probability distribution, plus a source of noise. In the case where P (Z) is a diagonal covariance Gaussian, erep(X;φ)
maps x to a vector of means, µ ∈ Rb, and a vector of standard deviations, σ ∈ Rb+, with the noise ∼ N (0, I). Put together, the encoder outputs samples z = µ+ σ, where is the Hadamard product. VAEs attempt to make these samples from the encoder match up with P (Z) by using the KL divergence between the parameters for a probability distribution outputted by erep(X;φ), and the parameters for the prior distribution, giving Lprior = DKL[Qφ(Z|X)‖P (Z)]. A multivariate Gaussian has an analytical KL divergence that can be further simplified when considering the unit Gaussian, resulting in Lprior = 12 ∑N n=1 µ
2 + σ2 − log(σ2)− 1. Another approach is to deterministically output the encodings z. Rather than minimising a metric between probability distributions using their parameters, we can turn this into a density ratio estimation problem where the goal is to learn a conditional distribution, Qφ(Z|X), such that the distribution of the encoded data samples, P̂ (Z) = ∫ Qφ(Z|X)P (X)dX , matches the prior distribution, P (Z). The GAN framework solves this density ratio estimation problem by transforming it into a class estimation problem using two networks (Goodfellow et al., 2014). The first network in GAN training is the discriminator network, Dψ , which is trained to maximise the log probability of samples from the “real” distribution, z ∼ P (Z), and minimise the log probability of samples from the “fake” distribution, z ∼ Qφ(Z|X). In our case e(X;φ) plays the role of the second network, the generator network, Gφ, which generates the “fake” samples.1 The two networks compete in a minimax game, where Gφ receives gradients from Dψ such that it learns to better fool Dψ . The training objective for both networks is given by Lprior = argminφ argmaxψ EP (Z)[log(Dψ(Z))]+ EP (X)[log(1−Dψ(Gφ(X)))] = argminφ argmaxψ EP (Z)[log(Dψ(Z))] + EQφ(Z|X)P (X) log[1− Dψ(Z)]. This formulation can create problems during training, so instead Gφ is trained to minimise − log(Dψ(Gφ(X))), which provides the same fixed point of the dynamics of Gφ and Dψ . The result of applying the GAN framework to the encoder of an autoencoder is the deterministic AAE (Makhzani et al., 2015).
2.2 DENOISING AUTOENCODERS
In a more general viewpoint, generative autoencoders fulfill the purpose of learning useful representations of the observed data. Another widely used class of autoencoders that achieve this are denoising autoencoders (DAEs), which are motivated by the idea that learned features should be robust to “partial destruction of the input” (Vincent et al., 2008). Not only does this require encoding the inputs, but capturing the statistical dependencies between the inputs so that corrupted data can be recovered (see Figure 3). DAEs are presented with a corrupted version of the input, x̃ ∈ X̃ , but must still reconstruct the original input, x ∈ X , where the noisy inputs are created through sampling x̃ ∼ C(X̃|X), a corruption process. The denoising criterion, Ldenoise, can be applied to any type of autoencoder by replacing the straightforward reconstruction criterion, Lreconstruct(X, d(e(X;φ); θ)), with the reconstruction criterion applied to noisy inputs: Lreconstruct(X, d(e(X̃;φ); θ)). The encoder is now used to model samples drawn from Qφ(Z|X̃). As such, we can construct denoising generative autoencoders by training autoencoders to minimise Ldenoise + Lprior. One might expect to see differences in samples drawn from denoising generative autoencoders and their non-denoising counterparts. However, Figures 4 and 6 show that this is not the case. Im et al.
1We adapt the variables to better fit the conventions used in the context of autoencoders.
(2015) address the case of DVAEs, claiming that the noise mapping requires adjusting the original VAE objective function. Our work is orthogonal to theirs, and others which adjust the training or model (Kingma et al., 2016), as we focus purely on sampling from generative autoencoders after training. We claim that the existing practice of drawing samples from generative autoencoders conditioned on z ∼ P (Z) is suboptimal, and the quality of samples can be improved by instead conditioning on z ∼ P̂ (Z) via MCMC sampling.
3 MARKOV SAMPLING
We now consider the case of sampling from generative autoencoders, where d(Z; θ) is used to draw samples from Pθ(X|Z). In Section 1, we showed that it was important, when sampling Pθ(X|Z), to condition on z’s drawn from P̂ (Z), rather than P (Z) as is often done in practice. However, we now show that for any initial z0 ∈ Z0 = Rb, Markov sampling can be used to produce a chain of samples zt, such that as t → ∞, produces samples zt that are from the distribution P̂ (Z), which may be used to draw meaningful samples from Pθ(X|Z), conditioned on z ∼ P̂ (Z). To speed up convergence we can initialise z0 from a distribution close to P̂ (Z), by drawing z0 ∼ P (Z).
3.1 MARKOV SAMPLING PROCESS
A generative autoencoder can be sampled by the following process:
z0 ∈ Z0 = Rb, xt+1 ∼ Pθ(X|Zt), zt+1 ∼ Qφ(Z|Xt+1)
This allows us to define a Markov chain with the transition operator
T (Zt+1|Zt) = ∫ Qφ(Zt+1|X)Pθ(X|Zt)dX (1)
for t ≥ 0. Drawing samples according to the transition operator T (Zt+1|Zt) produces a Markov chain. For the transition operator to be homogeneous, the parameters of the encoding and decoding functions are fixed during sampling.
3.2 CONVERGENCE PROPERTIES
We now show that the stationary distribution of sampling from the Markov chain is P̂ (Z).
Theorem 1. If T (Zt+1|Zt) defines an ergodic Markov chain, {Z1, Z2...Zt}, then the chain will converge to a stationary distribution, Π(Z), from any arbitrary initial distribution. The stationary distribution Π(Z) = P̂ (Z).
The proof of Theorem 1 can be found in (Rosenthal, 2001).
Lemma 1. T (Zt+1|Zt) defines an ergodic Markov chain.
Proof. For a Markov chain to be ergodic it must be both irreducible (it is possible to get from any state to any other state in a finite number of steps) and aperiodic (it is possible to get from any state to any other state without having to pass through a cycle). To satisfy these requirements, it is more than sufficient to show that T (Zt+1|Zt) > 0, since every z ∈ Z would be reachable from every other z ∈ Z. We show that Pθ(X|Z) > 0 and Qφ(Z|X) > 0, giving T (Zt+1|Zt) > 0, providing the proof of this in Section A of the supplementary material.
Lemma 2. The stationary distribution of the chain defined by T (Zt+1|Zt) is Π(Z) = P̂ (Z).
Proof. For the transition operator defined in Equation (1), the asymptotic distribution to which T (Zt+1|Zt) converges to is P̂ (Z), because P̂ (Z) is, by definition, the marginal of the joint distributionQφ(Z|X)P (X), over which theLprior used to learn the conditional distributionQφ(Z|X).
Using Lemmas 1 and 2 with Theorem 1, we can say that the Markov chain defined by the transition operator in Equation (1) will produce a Markov chain that converges to the stationary distribution Π(Z) = P̂ (Z).
3.3 EXTENSION TO DENOISING GENERATIVE AUTOENCODERS
A denoising generative autoencoder can be sampled by the following process:
z0 ∈ Z0 = Rb, xt+1 ∼ Pθ(X|Zt), x̃t+1 ∼ C(X̃|Xt+1), zt+1 ∼ Qφ(Z|X̃t+1).
This allows us to define a Markov chain with the transition operator
T (Zt+1|Zt) = ∫ Qφ(Zt+1|X̃)C(X̃|X)Pθ(X|Zt)dXdX̃ (2)
for t ≥ 0. The same arguments for the proof of convergence of Equation (1) can be applied to Equation (2).
3.4 RELATED WORK
Our work is inspired by that of Bengio et al. (2013); denoising autoencoders are cast into a probabilistic framework, where Pθ(X|X̃) is the denoising (decoder) distribution and C(X̃|X) is the corruption (encoding) distribution. X̃ represents the space of corrupted samples. Bengio et al. (2013) define a transition operator of a Markov chain – using these conditional distributions – whose stationary distribution is P (X) under the assumption that Pθ(X|X̃) perfectly denoises samples. The chain is initialised with samples from the training data, and used to generate a chain of samples from P (X). This work was generalised to include a corruption process that mapped data samples to latent variables (Bengio et al., 2014), to create a new type of network called Generative Stochastic Networks (GSNs). However in GSNs (Bengio et al., 2014) the latent space is not regularised with a prior.
Our work is similar to several approaches proposed by Bengio et al. (2013; 2014) and Rezende et al. (Rezende et al., 2014). Both Bengio et al. and Rezende et al. define a transition operator in terms of Xt and Xt−1. Bengio et al. generate samples with an initial X0 drawn from the observed data, while Rezende et al. reconstruct samples from an X0 which is a corrupted version of a data sample. In contrasts to Bengio et al. and Rezende et al., in this work we define the transition operator in terms of Zt+1 and Zt, initialise samples with a Z0 that is drawn from a prior distribution we can directly sample from, and then sample X1 conditioned on Z0. Although the initial samples may be poor, we are likely to generate a novel X1 on the first step of MCMC sampling, which would not be achieved using Bengio et al.’s or Rezende et al.’s approach. We are able draw initial Z0 from a prior because we constrain P̂ (Z) to be close to a prior distribution P (Z); in Bengio et al. a latent space is either not explicitly modeled (Bengio et al., 2013) or it is not constrained (Bengio et al., 2014).
Further, Rezende et al. (2014) explicitly assume that the distribution of latent samples drawn from Qφ(Z|X) matches the prior, P (Z). Instead, we assume that samples drawn from Qφ(Z|X) have a distribution P̂ (Z) that does not necessarily match the prior, P (Z). We propose an alternative method for sampling P̂ (Z) in order to improve the quality of generated image samples. Our motivation is also different to Rezende et al. (2014) since we use sampling to generate improved, novel data samples, while they use sampling to denoise corrupted samples.
3.5 EFFECT OF REGULARISATION METHOD
The choice of Lprior may effect how much improvement can be gained when using MCMC sampling, assuming that the optimisation process converges to a reasonable solution. We first consider the case of VAEs, which minimise DKL[Qφ(Z|X)‖P (Z)]. Minimising this KL divergence penalises the model P̂ (Z) if it contains samples that are outside the support of the true distribution P (Z), which might mean that P̂ (Z) captures only a part of P (Z). This means that when sampling
P (Z), we may draw from a region that is not captured by P̂ (Z). This suggests that MCMC sampling can improve samples from trained VAEs by walking them towards denser regions in P̂ (Z).
Generally speaking, using the reverse KL divergence during training, DKL[P (Z)‖Qφ(Z|X)], penalises the model Qφ(Z|X) if P (Z) produces samples that are outside of the support of P̂ (Z). By minimising this KL divergence, most samples in P (Z) will likely be in P̂ (Z) as well. AAEs, on the other hand are regularised using the JS entropy, given by 12DKL[P (Z)‖ 1 2 (P (Z) + Qφ(Z|X))] + 1 2DKL[Qφ(Z|X)‖ 1 2 (P (Z) + Qφ(Z|X))]. Minimising this cost function attempts to find a compromise between the aforementioned extremes. However, this still suggests that some samples from P (Z) may lie outside P̂ (Z), and so we expect AAEs to also benefit from MCMC sampling.
4 EXPERIMENTS
4.1 MODELS
We utilise the deep convolutional GAN (DCGAN) (Radford et al., 2015) as a basis for our autoencoder models. Although the recommendations from Radford et al. (2015) are for standard GAN architectures, we adopt them as sensible defaults for an autoencoder, with our encoder mimicking the DCGAN’s discriminator, and our decoder mimicking the generator. The encoder uses strided convolutions rather than max-pooling, and the decoder uses fractionally-strided convolutions rather than a fixed upsampling. Each convolutional layer is succeeded by spatial batch normalisation (Ioffe & Szegedy, 2015) and ReLU nonlinearities, except for the top of the decoder which utilises a sigmoid function to constrain the output values between 0 and 1. We minimise the cross-entropy between the original and reconstructed images. Although this results in blurry images in regions which are ambiguous, such as hair detail, we opt not to use extra loss functions that improve the visual quality of generations (Larsen et al., 2015; Dosovitskiy & Brox, 2016; Lamb et al., 2016) to avoid confounding our results.
Although the AAE is capable of approximating complex probabilistic posteriors (Makhzani et al., 2015), we construct ours to output a deterministic Qφ(Z|X). As such, the final layer of the encoder part of our AAEs is a convolutional layer that deterministically outputs a latent sample, z. The adversary is a fully-connected network with dropout and leaky ReLU nonlinearities. erep(X;φ) of our VAEs have an output of twice the size, which corresponds to the means, µ, and standard deviations, σ, of a diagonal covariance Gaussian distribution. For all models our prior, P (Z), is a 200D isotropic Gaussian with zero mean and unit variance: N (0, I).
4.2 DATASETS
Our primary dataset is the (aligned and cropped) CelebA dataset, which consists of 200,000 images of celebrities (Liu et al., 2015). The DCGAN (Radford et al., 2015) was the first generative neural network model to show convincing novel samples from this dataset, and it has been used ever since as a qualitative benchmark due to the amount and quality of samples. In Figures 7 and 8 of the supplementary material, we also include results on the SVHN dataset, which consists of 100,000 images of house numbers extracted from Google Street view images (Netzer et al., 2011).
4.3 TRAINING & EVALUATION
For all datasets we perform the same preprocessing: cropping the centre to create a square image, then resizing to 64 × 64px. We train our generative autoencoders for 20 epochs on the training split of the datasets, using Adam (Kingma & Ba, 2014) with α = 0.0002, β1 = 0.5 and β2 = 0.999. The denoising generative autoencoders use the additive Gaussian noise mapping C(X̃|X) = N (X, 0.25I). All of our experiments were run using the Torch library (Collobert et al., 2011).2
For evaluation, we generate novel samples from the decoder using z initially sampled from P (Z); we also show spherical interpolations (White, 2016) between four images of the testing split, as depicted in Figure 2. We then perform several steps of MCMC sampling on the novel samples and interpolations. During this process, we use the training mode of batch normalisation (Ioffe &
2Example code is available at https://github.com/Kaixhin/Autoencoders.
Szegedy, 2015), i.e., we normalise the inputs using minibatch rather than population statistics, as the normalisation can partially compensate for poor initial inputs (see Figure 4) that are far from the training distribution. We compare novel samples between all models below, and leave further interpolation results to Figures 5 and 6 of the supplementary material.
4.4 SAMPLES
5 CONCLUSION
Autoencoders consist of a decoder, d(Z; θ) and an encoder, e(X;φ) function, where φ and θ are learned parameters. Functions e(X;φ) and d(Z; θ) may be used to draw samples from the conditional distributions Pθ(X|Z) andQφ(Z|X) (Bengio et al., 2014; 2013; Rezende et al., 2014), where X refers to the space of observed samples and Z refers to the space of latent samples. The encoder distribution, Qφ(Z|X), maps data samples from the data generating distribution, P (X), to a latent distribution, P̂ (Z). The decoder distribution, Pθ(X|Z), maps samples from P̂ (Z) to P (X). We are concerned with generative autoencoders, which we define to be a family of autoencoders where regularisation is used during training to encourage P̂ (Z) to be close to a known prior P (Z). Commonly it is assumed that P̂ (Z) and P (Z) are similar, such that samples from P (Z) may be used to sample a decoder Pθ(X|Z); we do not make the assumption that P̂ (Z) and P (Z) are “sufficiently close” (Rezende et al., 2014). Instead, we derive an MCMC process, whose stationary distribution is P̂ (Z), allowing us to directly draw samples from P̂ (Z). By conditioning on samples from P̂ (Z), samples drawn from x ∼ Pθ(X|Z) are more consistent with the training data. In our experiments, we compare samples x ∼ Pθ(X|Z = z0), z0 ∼ P (Z) to x ∼ Pθ(X|Z = zi) for i = {1, 5, 10}, where zi’s are obtained through MCMC sampling, to show that MCMC sampling improves initially poor samples (see Figure 4). We also show that artifacts in x samples induced by interpolations across the latent space can also be corrected by MCMC sampling see (Figure 2). We further validate our work by showing that the denoising properties of denoising generative autoencoders are best revealed by the use of MCMC sampling.
Our MCMC sampling process is straightforward, and can be applied easily to existing generative autoencoders. This technique is orthogonal to the use of more powerful posteriors in AAEs (Makhzani et al., 2015) and VAEs (Kingma et al., 2016), and the combination of both could result in further improvements in generative modeling. Finally, our basic MCMC process opens the doors to apply a large existing body of research on sampling methods to generative autoencoders.
ACKNOWLEDGEMENTS
We would like to acknowledge the EPSRC for funding through a Doctoral Training studentship and the support of the EPSRC CDT in Neurotechnology.
B CELEBA
B.1 INTERPOLATIONS
C STREET VIEW HOUSE NUMBERS
C.1 SAMPLES
C.2 INTERPOLATIONS

Review:
The authors argues that the standard ancestral sampling from stochastic autoencoders (such as the Variational Autoencoder and the Adversarial
Autoencoder) imposes the overly-restrictive constraint that the encoder distribution must marginally match the latent variable prior. They propose, as an alternative, a Markov Chain Monte Carlo approach that avoids the need to specify a simple parametric form for the prior.
The paper is not clearly written. Most critically, the notation the authors use is either deeply flawed, or there are simple misunderstanding with respect to the manipulations of probability distributions. For example, the authors seem to suggest that both distributions Q(Z|X) and Q(X|Z) are parametrized. For this to be true the model must either be trivially simple, or an energy-based model. There is no indication that they are speaking of an energy-based model. Another example of possible confusion is the statement that the ratio of distributions Q(Z|X)/P(Z) = 1. I believe this is supposed to be a ratio of marginals: Q(Z)/P(X) = 1. Overall, it seems like there is a confusion of what Q and P represent. The standard notation used in VAEs is to use P to represent the decoder distribution and
Q to represent the encoder distribution. This seems not to be how the authors are using these terms. Nor does it seem like there is a single consistent interpretation. 
The empirical results consist entirely of qualitative results (samples and reconstructions) from a single dataset (CelebA). The samples are also not at all up to the quality of the SOTA models. The interpolations shown in Figures 1 and 3 both seems to look like interpolation in pixel space for both the VAE model and the proposed DVAE.