Preventing posterior collapse in variational autoencoders for text generation via decoder regularization
Abstract
Variational autoencoders trained to minimize the reconstruction error are sensitive to the posterior collapse problem, that is the proposal posterior distribution is always equal to the prior.
We propose a novel regularization method based on fraternal dropout to prevent posterior collapse.
We evaluate our approach using several metrics and observe improvements in all the tested conﬁgurations
Introduction
Deep generative models like Variational Autoencoders (VAE) [12] and Generative Adversarial networks (GAN) [8], among others, enjoy great popularity in many applications of machine learning, including natural language processing (NLP).
Unlike GANs, VAEs can manipulate discrete observed variables which makes them suitable for applications in text generation.
In these models, a sentence is generated according to the following process:
[Equation]
where z ∈ R k is a latent sentence representation (or sentence embedding) and x 1 , x 2 , · · · ∈ X are observed words (or the generated sentence), with X the vocabulary.
The generation stops when a special end of the sentence word is generated.
Without loss of generality, we assume that the prior distribution p ( z ) is ﬁxed.
The subscript θ indicates the parameters of the conditional distribution and, in our case, θ corresponds to the parameters of a neural network.
It is important to note that we do not make any independence assumption in the conditional distribution p ( x t | z , x <t ) .
Training aims to search for parameters θ that maximize the likelihood of observed data, also called the evidence:
[Equation]
where ˜ p is the empirical training data distribution.
In general [Equation] the objective function L is intractable because of the marginalization over latent variables z .
Variational methods propose to introduce a proposal distribution q φ ( z | x ) to create a surrogate objective called the evidence lower bound (ELBO) deﬁned as follows:
[Equation]
where the E deﬁnes a family of lower bounds on L parameterized by φ , i..
e ∀ φ : E ( x , θ, φ ) ≤ L ( x , θ ) .
During training, it is important to search for proposal parameters φ that gives the best bounds (i.e. maximizes the bounds), hence the name variational.
The new training problem is then:
[Equation]
The Expectation-Maximization (EM) algorithm
[4] solves this problem via block coordinate ascent, i.e. maximizing successively with respect to φ (E step) and θ (M step).
Unlike EM, the VAE approach consists in optimizing this problem by joint stochastic gradient ascent over φ and θ .
Moreover, unlike in the standard applications of EM, neither the distribution nor the family of the posterior p θ ( z | x ) are known.
As a result, an independence assumption is made over the coordinates of z in the distribution q φ (also called mean ﬁeld distribution).
Finally, the distribution q φ is amortized over the data and parameterized by a neural network.
It is worth noting that during training, the reconstruction term in equation 2 is estimated via Monte-Carlo method using a single sample.
It is usual to call the distribution q φ ( z | x ) that generates a latent representation from a sentence the encoder and the distribution p θ ( x | z ) that reconstructs the original sentence the decoder .
Unfortunately in practice, VAEs for automatic text generation are sensitive to posterior collapse [2].
Informally, this means that the proposal distribution is not optimized correctly and remains close to the prior distribution for all data points: ∀ x : q φ ( z | x ) (cid:39) p ( z ) .
This leads to a poor approximation of the objective 1 by the ELBO.
In the end, the decoder ignore that latent variable and no sentence representation z is learned.
Previous work on the posterior collapse problem can be classiﬁed in two categories: 1.
On the one hand, modiﬁcations to the objective function were suggested.
Bowman et al. [2] reweight the divergence with the prior term to temper its importance during training.
Kingma et al.
[13] and Pelsmaeker and Aziz [19] proposed to introduce constraints to force the divergence term to be greater than a hyperparameter.
Finally, alternative surrogate objectives to train VAEs in the context of text generation have been proposed [16, 9].
On the other hand, several authors proposed to modify the architecture of the decoder so that they are forced to rely on latent variable information.
Yang et al.
[22] replace the recurrent neural network in the decoder by a convolutional neural network.
Dieng et al.
[5] proposed to use skip connections between the latent representation and the various hidden layers of the decoder.
In this work, we propose a different approach to prevent the posterior collapse problem based on parameter regularization, that is without changing the generative model objective function or the architecture of the decoder.
Our contributions can be summarized as follows:
• We propose to regularize the decoder parameters to prevent the posterior collapse.
In particular, we use fraternal dropout [23] to force the decoder to use the latent representation.
• We experiment our approach in various settings and report improved results with respect to several evaluation metrics.
We hope to encourage future research to explore this direction to improve VAEs for text generation.
Decoder regularization: fraternal dropout
We ﬁrst give an intuitive motivation for our approach.
LSTMs have been widely used as neural language models and achieve competitive results [17].
1 Language models are trained via the same reconstruction term as the one used in VAEs, showing that these neural architectures can efﬁciently maximize this term in the absence of latent variables, i.e. it can be maximized while ignoring the latent variables values.
To bypass this issue, we propose to introduce a regularization term in the objective that forces the hidden representations computed by the LSTM to be similar even if different words in the input are masked.
The decoder is then forced to rely on latent variable information.
To this end, we propose to rely on fraternal dropout [23].
The reconstruction term of the ELBO maximizes the log-likelihood using the usual teacher forcing technique for language models: during training, the auto-regressive model is trained to predict the next word based on gold previous words.
Let x ∈ X n a sentence of length n .
Each word x i is represented by a vector taken from an embedding table.
We denote all of these vectors by a matrix
E ∈ R w × n where w is the word embedding dimension.
A contextual representation is computed for each position in the sentence using a LSTM:
[Equation]
where H ∈ R d × n is a matrix containing hidden representations of dimension d for each sentence position.
The latent variable z is projected and then given as an initialization for both the memory and the hidden state and also concatenated to each input.
We refer the reader to [15] for more details on the architecture and the various parameters.
Word dropout [6] consists in randomly replacing embeddings by a vector of zeros during training to prevent overﬁtting:
[Equation]
where d ∈ { 0 , 1 } n is a vector of booleans where each element is independently drawn from a
Bernoulli distribution of parameter b ∈ [0 , 1] .
The matrix E (cid:48) corresponds to the matrix E where a number of columns are ﬁlled with zeros.
Fraternal dropout consists in creating two matrices E (cid:48) and
E (cid:48)(cid:48) as follows:
[Equation]
Matrices E (cid:48) and E (cid:48)(cid:48) are both used to compute the log-likelihood of the sentence and the mean loglikelihood then replaces the original reconstruction term.
Finally, a regularization term is introduced in the objective function:
[Equation]
where α > 0 is a hyperparameter.
Note that the regularization term forces the hidden representations computed by the LSTM to be similar with different masked inputs E (cid:48) and E (cid:48)(cid:48) but similar latent variables z .
Hence, the regularization term forces the decode to rely on latent variables z to compute the LSTM’s hidden representations.
Experiments
We evaluate our approach using the code distributed by by Li et al.
We kept the same hyperparameters as the ones used by the authors to avoid skewing the results in our favor.
We experiment with two datasets: Yelp [20] and the Stanford Natural Language Inference (SNLI) [1] data.
These two datasets were subsampled to contain 100,000 training sentences and 10, evaluation and testing sentences.
SNLI and Yelp have a respective vocabulary of 9,990 and 8, words and both have an average of 10 words per sentence.
3.1 Evaluation metrics
We use several metrics to evaluate the quality of the learned generative models. (Log-likelihood and perplexity per word)
The negative log-likelihood (NLL) − E q φ ( z | x ) p θ ( x | z ) indicates how well the model reconstructs the input sentence.
The perplexity per word (PPL) is the geometric mean of the reciprocal of the probability assigned to the correct word by the model.
Since we observe the opposite in the ﬁrst case and the reciprocal in the second, we want to minimize both of these metrics.
They are approximated by sampling 100 times in the distribution q for each sentence. (BLEU score) For each sentence in the test set, a latent representation is sampled in the distribution q and a sentence is generated without using teacher forcing from the representation. The BLEU score [18] will represent the proportion of n-grams (for n going from 1 to 4) in the generated sentence that
[Table] [Table]
can be found in the original one.
If the generated sentence is shorter than the original, a penalty is applied since it is easier to avoid mistakes when less words are produced.
(Active units)
The number of active units (AU) represent the number of dimensions of the latent variable that co-vary with the observations.
According to Burda et al.
[3], a greater number of active units is usually representative of a richer latent variable.
We follow their article and use a threshold value of 0 .
The dimension of the latent variable is 32, therefore we will have at most 32 active units. (Mutual Information)
We also report the mutual information between the latent variable and the output probability distribution.
A higher mutual information indicates that the latent variable is better used by the model.
We follow the methodology of He et al.
3.2 Baseline
We compare our approach to previous work including a "standard" VAE, the free bits technique [13] and the pre-training approach [15]:
• (Free bits)
The free bits technique consists in introducing a constraint on the divergence term with the prior distribution so that it doesn’t fall below a pre-determined threshold λ .
We follow previous work and ﬁx λ = 8 in all of our experiments.
• (Pre-training)
This baseline consists in training the model as a classic autoencoder ﬁrst.
Then the decoder is reset and the model is trained as a VAE.
All of our experiments reweight the divergence term during the training of the model.
This technique, proposed by Bowman et al.
[2], consists in steadily increasing the reweighting factor from 0 toduring the ﬁrst epochs of training.
The idea is to force the model to ignore the divergence with prior term in the beginning of training.
3.3 Results and analysis
We ﬁrst evaluate the impact of the fraternal dropout hyperparameter α on the Yelp dataset.
Results are reported in Table 1.
We can observe that a compromise needs to be found between all the metrics, i.e. we need to ﬁnd a point of equilibrium between minimizing the NLL and the PPL and maximizing
AU, MI and the BLEU score.
In the following experiments, we ﬁxed α = 0 .
We report results in different conﬁgurations in Table 2.
Our approach yields improvements for all the metrics in all conﬁgurations for both datasets.
The posterior collapse problem is signiﬁcant in
[Table]
conﬁgurations not using free bits or fraternal dropout, the mutual information being around 1 and the number of active units being 2.
Adding fraternal dropout results in a gain between 4 and 7 points for the mutual information in the two conﬁgurations.
In conﬁgurations where the model is pre-trained, we observe that this pre-training alone does not prevent the posterior collapse since there are onlyand 3 active units respectively while the same model with fraternal dropout retains 22 active units.
Interestingly, our approach has a bigger impact than free bits on Yelp and a similar one on SNLI while using less active units in both cases.
This is an indication that the free bits technique forces the latent variables to be decorrelated artiﬁcially, i
[Table]
e. the decoder still ignores their values.
As explained previously [Table] the BLEU score is computed over sentences generated without teacher forcing and therefore can also be used to estimate the quality of the latent representations.
Once again, fraternal dropout improves results on this metric.
Interpolation We show some examples of sentences generated via interpolation between two latent representations sampled from the prior in Table 3.
We see that our method seems to produce coherent sentences with a gradual change in length and meaning between the successive sentences.
Conclusion
In this work [Table] we propose to rely on parameter regularization to prevent the posterior collapse problem in VAEs.
This approach is different from previous work in the literature.
We observe that our approach has two beneﬁts: it improves the quality of generated text and increases the use of the latent variable.
Future works could explore other methods for parameter regularization [11, 14, 7].
Acknowledgments and Disclosure of Funding
We thank François Yvon and Matthieu Labeau for proofreading the article.
This work beneﬁted from computations done on the Saclay-IA platform and an access to the computational resources of IDRIS through the resource allocation 20XX-AD11011600 attributed by GENCI.