Title: DOUBLY STOCHASTIC ADVERSARIAL AUTOENCODER

Abstract: Any autoencoder network can be turned into a generative model by imposing an arbitrary prior distribution on its hidden code vector. Variational Autoencoder uses a KL divergence penalty to impose the prior, whereas Adversarial Autoencoder uses generative adversarial networks. A straightforward modification of Adversarial Autoencoder is to replace the adversary by maximum mean discrepancy (MMD) test. This replacement leads to a new type of probabilistic autoencoder, which is also discussed in our paper.
However, an essential challenge remains in both of these probabilistic autoencoders, namely that the only source of randomness at the output of encoder, is the training data itself. Lack of enough stochasticity can make the optimization problem nontrivial. As a result, they can lead to degenerate solutions where the generator collapses into sampling only a few modes.
Our proposal is to replace the adversary of adversarial autoencoder by a space of stochastic functions. This replacement introduces a a new source of randomness, which can be considered as a continuous control for encouraging explorations. This prevents the adversary from fitting too closely to the generator and therefore leads to more diverse set of generated samples. Consequently, the decoder serves as a better generative network, which unlike MMD nets scales linearly with the amount of data. We provide mathematical and empirical evidence on how this replacement outperforms the pre-existing architectures.

1. INTRODUCTION:

Any autoencoder network can be turned into a generative model by imposing an arbitrary prior distribution on its hidden code vector. Variational Autoencoder (VAE) [2] uses a KL divergence penalty to impose the prior on the hidden code vector of the autoencoder, whereas Adversarial Autoencoder (AAE) [1] uses generative adversarial networks (GAN) [2] . GAN trades the complexities of sampling algorithms with the complexities of searching Nash equilibrium in minimax games. Such minimax architectures get updated directly with the help of data examples and gradients flowing through a generative and an adversary. A straightforward modification of AAE is to replace the adversarial network by maximum mean discrepancy (MMD) net [4] [5] . This replacement leads to a new type of probabilistic autoencoder, which is also discussed in our paper.

However, an essential challenge remains in both of these probabilistic autoencoders, namely that the only source of randomness at the output of encoder, is the training data itself. Lack of enough stochasticity can make the minimax search non-trivial. As a result, it can lead to degenerate solutions where the generator collapses into sampling only a few modes.

In order to mitigate the mode collapse issue, we introduce randomness to the loss functions of such minimax architectures, which can be considered as a continuous control for encouraging explorations. This prevents the adversary from fitting too closely to the generator and therefore leads to more diverse set of generated samples. Introducing randomness at the high dimensional feature space can make the minimax search problem very challenging. Fortunately, this is a much easier search problem in our case, thanks to the lower dimensionality of latent space (encoder output).

2. PROBABILISTIC AUTOENCODERS:

We consistently refer to any random variable and their realization with calligraphic fonts and small letters respectively. Fix an autoencoder (AE) with encoder parameters of θ ∈ Θ that gets i.i.d data samples X , X 1 , ..., X N as input and outputs the corresponding latent vectors Z, Z 1 , ..., Z N . Assume we don't know the distribution of X but we want to impose an arbitrary distribution P on Z. This is because we like to think of the decoder as a generative network with prior P. The encoding function Q θ (Z|X ) induces aggregated posterior distribution Q θ (Z) formulated in eq. 1:

EQUATION

The decoder and encoder networks of a standard AE can be trained together by minimizing the difference between the original data and its reconstruction. We can impose P on the latent code vector through regularization of a standard AE by minimizing the discrepancies between aggregated posterior Q θ (Z) and an arbitrary prior P. This can be formalized as the following optimization problem:

EQUATION

where δ is a suitable discrepancy measure. In other words, parameters of the encoder θ should be optimized s.t the transported randomness from X match up to that of prior P. Achievement of a solution for eq. 2 via deterministic functions is guaranteed given enough stochasticity in X , thanks to the Lemma 2.22 in [11] . However, lack of enough stochasticity in X leads to not smooth enough aggregated posterior Q θ (Z) which makes the minimization in Eq. 2 non-trivial. This is specially important considering the deterministic functionality of encoder. It is because the only source of randomness originates from the training data itself, which may not be enough for smoothing out the aggregated posterior.

2.1. INTRODUCING RANDOMNESS:

Some ways of introducing randomness for smoothing out the aggregated posterior Q θ (Z) include:

• Introducing the stochasticity N at the input or output of the encoder function:

EQUATION

EQUATION

This may ask for re-parametrization tricks [2] .

• Dropouts [9] Different ways of introducing and exploiting randomness lead to different training dynamics. The choice of discrepancy measure δ itself leads to various types of training dynamics. Our proposal is to introduce randomness at the heart of the discrepancy measure itself. First, we discuss two known discrepancy measures in the next section.

2.2. DISCREPANCY MEASURE:

Adversarial autoencoder can be recovered with the following Jenson-Shannon divergence (JSD) choice for the discrepancy measure:

EQUATION

where Y ∼ P and Ỹ ∼ Q θ (Z) and D α is the adversary with parameter α. Moreover E is w.r.t to the associated random variable of each term. A straightforward modification is to replace the adversarial network by maximum mean discrepancy (MMD) net [4] [5] by assuming the following discrepancy measure:

EQUATION

where f is a function living in a reproducing kernel Hilbert space RKHS H and Y ∼ P and Ỹ ∼ Q θ (Z) as above.

Due to the reproducing property of H, the expectation of any function f in RKHS H with respect to random variable Y can be computed as an inner product with its so called kernel mean embedding

EQUATION

k being the kernel associated with RKHS H.

Definition 1 A kernel k(x, x ) : X × X → R is positive definite (PD)

when for all n > 1 and x 1 , x 2 , .., x n ∈ X and c 1 , ..., c n ∈ R, we have i,j c i c j k(x i , x j ) ≥ 0.

Gretton [12] demonstrates that a closed form solution exists for Eq. 6 with the following unbiased estimator :

EQUATION

We refer to a probabilistic AE that uses Eq. 8 as discrepancy measure MMD-AE.

3. DOUBLY STOCHASTIC KERNEL MACHINES AS ADVERSARY:

The existence of closed form expression in Eq. 8 makes the implementation of MMD-AE straightforward, however, it hides the minimax nature of the problem. If we replace eq. 6 in eq. 2, we are back to a minimax problem similar to that of adversarial networks as the following:

EQUATION

Here is where the discussed problem with the lack of enough stochasticity of X and non-smooth aggregated posterior gets more clear. Using Eq. 7 and Eq. 9, we note that adversary's best response training dynamic is determined by the stochastic gradient terms ∂δ ∂f = ξ(.)

:= E[k(Y, .) -k( Ỹ, .)]. Since prior P is smooth enough, term E Ỹ∼Q θ (Z)

[k( Ỹ, .)] determines the smoothness of the stochastic gradients. Not enough stochasticity in X implies non-smooth gradients and therefore more difficult optimization problem. We propose to massage the stochastic gradients ξ(.) with extra source of stochasticity W. This leads to doubly stochastic gradient terms ζ(.) := E W ξ(.). Introducing new source of randomness to ξ(.) becomes feasible thanks to the existing duality between the Kernel and Random processes according the following Theorem:

Theorem 1 [15] Duality between Kernels and Random Processes: If k(x, x ) is a PD kernel, then there exits a set Ω, a measure P on Ω, and random function φ W (x) : X → R from L 2 (Ω, P), such that k(x, x ) = Ω φ W (x)φ W (x )dP(W).

Input H1 H2:

Encoder

Q θ RKHS H E(f α (Y)) E(f α ( Ỹ)) P Q θ (Z) ξ(.) ζ(.)

Figure 1 : Scheme of DS-AAE Each distribution is mapped into a reproducing kernel Hilbert space via an expectation operation. The generator strategy is to adjust the parameter of encoder θ to decrease the distance between the blue and red dot while adversary's is to adjust α to increase the distance. Not enough stochasticity in X limits the adversary power as is reflected in its stochastic gradient terms ξ = E

Y∼Q θ (Z) [k(Y, .)].

With the introduction of new source of randomness, the adversary gains extra power, as is reflected in its doubly stochastic gradient terms ζ = E W ξ(.). This extra boost to adversary, however, can lead to the search outside of H where the gradients are no longer valid closed form expressions. Fortunately, [6] shows that with small learning rates, the search returns back to H and convergence is guaranteed.

Example For Gaussian RBF kernel, k(x -x ) = exp( x -x 2 /2σ 2 ), φ W (x) = exp(-iW x)

and

P(w) = exp(-w 2 2 /2) (2π) d/2

As the result of Theorem 1, we can rewrite the massaged gradients as

ζ = [φ W (Y) -φ W ( Ỹ)]φ W (.)

with W ∼ P(W).

3.1. APPROXIMATING ADVERSARY f BY ITS DOUBLY STOCHASTIC GRADIENT TERMS:

Using a similar approach to Doubly Stochastic Kernel Machines [6] , we can now approximate a new adversary with parameter α by the linear combination of doubly stochastic gradient terms ζ i.e. f α (.) = αζ(.). However, we modify the adversary's objective function to -[φ W ( Ỹ)φ W (.)] Ỹ∼Q θ (Z) . This modification does not affect the fixed points of the best response dynamics but provides much stronger gradients early in learning. GAN [3] applies the same modification to the objective function of the adversary. The adversary affects the dynamic of training though the adjustment in α. We refer this probabilistic autoencoder as Doubly Stochastic AAE (DS-AAE). Beside the residual error of approximating f with its first order gradient terms, there is another concern that needs to be addressed. The key difference between ζ(.) and ξ(.) is that ζ(.) could fall outside of the RKHS but ξ(.) is always in RKHS. This is due to the term φ w (.) not being in RKHS. Fortunately proof of the convergence with small enough learning rate is discussed in [6] .

The scheme of DS-AAE is visualized in Fig. 1 . To sum up, we proposed replacing the adversary with a space H of stochastic functions, which may not reside in RKHS H but their convergence to the optimal functions in H is guaranteed. This replacement introduces a new source of randomness, MNIST GAN [2] 225 ± 2 GMMN + AE [5] 282 ± 2 Adversarial Autoencoder [1] 340 which can be considered as a continuous control for encouraging explorations. It prevents the adversary from fitting too closely to the generator and therefore leads to more diverse set of generated samples. Consequently, the decoder serves as a better generative network which unlike MMD-AE scales linearly with the amount of data. We provide empirical evidence on how this replacement outperforms the pre-existing architectures in the next section.

4. EXPERIMENTS:

We generate W according to Example 1 with a fixed seed. The encoder and decoder both have 3 layers of 1024, 512, 216 hidden units with ReLU activation for every layer except the last layer of decoder that is a sigmoid activation function. Cross entropy is used for reconstruction loss. The mini-batch size is 1000. The prior P is Gaussian and the dimensionality of the hidden code is 6 for the DS-AAE and 4 for MMD-AE. The only used dropout is at the first input layer of the encoder with the rate of 20%. Initial learning rate for the reconstruction loss is adjusted at 0.001 and 0.001 for the adversarial architectures, followed by Adam stochastic optimization [13] .

Comparison of deep generative models is hard especially for the log-likelihood-free models [7] .

Parzen window estimation of the log-likelihood is obtained by drawing 10K samples from the trained model on MNIST. The results are shown in Table 1 . From the qualitative perspective, we can see from Fig. 2b , Fig. 2d and reported results in [1] that the drawn samples for both MMD-AE and AAE are more homogenous than DS-AAE. In the case of DS-AAE, it is almost as if different persons were writing the digits in each panel. This quality test is also used in [8] . This is because DS-AAE enjoys from extra randomness in the minimax optimization framework, which helps the generative model to explore multiple modes and mitigate the risk of collapse.

The learned coding space of DS-AAE exhibits sharp transitions and has no "holes", as is shown in Fig. 2 (a). This is important to ensure that generating from any part of prior space results in meaningful samples. From this perspective, DS-AAE is similar to AAE and unlike VAE. However it recovers more of a mixture of 2D-Gaussians rather than a 2D-Gaussian distribution. We leave further investigation of this interesting observation to future version of this paper.

SVHN experiment is carried out by the the same architecture and batch size. However, we have used batch-normalization [14] in all the autoencoder layers including the Softmax layer and number of latent codes is 10 in this case. We observe that the generated samples from DS-AA are more diverse, yet they are blurrier than those of MMD-AE, AAE and the rest of GAN architectures. Results are shown in Fig. 3 .

5. CONCLUSION:

The recent proposal of generative adversarial models, trades the complexities of sampling algorithms with the complexities of searching Nash equilibrium in minimax games. From this vantage point, role of a machine learning researcher is similar to that of a mechanism designer, who intervenes with the dynamic of game through design of suitable loss function. A recent development of probabilistic autoencoder enjoys this framework by placing such minimax games at the output of an encoder to force a desirable prior. We propose a different minimax game and loss function. As a mechanism designer, we intervene with the dynamic of training through introducing suitable randomness to the loss functions of such minimax games. This makes the game end up in better regions and as a result, better probabilistic autoencoders can be achieved.

:

(a) DS-AAE: The hidden code Z of the hold-out images with 2 latent variables. Each color represents the associated label on MNIST data set.

Figures and Tables:

(b) DS-AAE: Drawn sample after 1000 epochs (c) MMD-AE: The hidden code Z of the hold-out images with 2 latent variables. Each color represents the associated label on MNIST data set. (d) MD-AE: Drawn samples after 1000 epochs

Figure 2: Comparison between MMD-AE and DS-AAE on MNIST.

Figure 3: SVHN drawn samples after 1000 epochs

Parzen window estimate of the log-likelihood obtained by drawing 10K samples from the trained model.

Table content: <table><tr><td>± 2</td></tr></table>

Peer review (iclr2018):
 This manuscript explores the idea of adding noise to the adversary*s play in GAN dynamics over an RKHS. This is equivalent to adding noise to the gradient update, using the duality of reproducing kernels. Unfortunately, the evaluation here is wholly unsatisfactory to justify the manuscript*s claims. No concrete practical algorithm specification is given (only a couple of ideas to inject noise listed), only a qualitative one on a 2-dimensional latent space in MNIST, and an inconclusive one using the much-doubted Parzen window KDE method. The idea as stated in the abstract and introduction may well be worth pursuing, but not on the evidence provided by the rest of the manuscript.