F low P rior: Learning Expressive Priors for Latent Variable Sentence Models (2021)

Abstract: Variational autoencoders (VAEs) are widely used for latent variable modeling of text. We focus on variations that learn expressive prior distributions over the latent variable. We find that existing training strategies are not effective for learning rich priors, so we propose adding the importance-sampled log marginal likelihood as a second term to the standard VAE objective to help when learning the prior. Doing so improves results for all priors evaluated, including a novel choice for sentence VAEs based on normalizing flows (NF). Priors parameterized with NF are no longer constrained to a specific distribution family, allowing a more flexible way to encode the data distribution. Our model, which we call FlowPrior, shows a substantial improvement in language modeling tasks compared to strong baselines. We demonstrate that FlowPrior learns an expressive prior with analysis and several forms of evaluation involving generation.

## introduction variational autoencoders (vaes; @xcite have been widely applied to many natural language processing tasks @xcite @xcite @xcite . vaes provide statistical transparency in describing observations in a latent space and flexibility when used in applications that require directly manipulating the learned representation @xcite . recent work @xcite has combined vaes with bert/gpt in representation learning and guided generation. however, the representation capacity of vaes is still limited for modeling sentences due to two main reasons. one is known as the posterior collapse problem, in which the posterior "collapses" to the prior and the generator learns to ignore the latent variable @xcite . many methods have been developed to address it: annealing @xcite , weakening the capacity of the generator @xcite , manipulating training objectives @xcite @xcite , including the use of free bits (fb) @xcite , and changing training @xcite . the other reason is the restrictive assumption of the parametric forms for the prior and approximate posterior. while these forms are computationally efficient, they limit the expressivity of the model. the main existing solutions @xcite @xcite focus on enriching the variational posterior, while other work focuses on learning an expressive prior @xcite @xcite . in this paper, we follow the latter line of research and draw upon methods in building and learning expressive priors. we first show empirically that the original vae objective, the evidence lower bound (elbo), is not effective when learning priors. the issue is not solely due to posterior collapse since it is not resolved by using modifications based on free bits. to address this issue, we propose using a combined objective, adding to the elbo a second objective (denoted m is ) which is a different lower bound on the log marginal likelihood obtained using importance sampling @xcite . using the combination of the elbo and m is , we compare multiple choices for the prior, including a mixture of gaussians, a prior based on a variational mixture of posteriors (vampprior; @xcite , and a prior based on normalizing flows (nf), specifically real nvp transformations @xcite . using a real nvp prior entails creating an invertible mapping from a simple base distribution to the prior distribution of the latent variable in a vae. this choice allows a flexible prior distribution that is not constrained to a specific parametric family. the hope is that it would be better at modeling the data distribution. we perform an empirical evaluation of priors and objective functions for training vae sentence models on four standard datasets. we find the best performance overall when using the flow-based prior and the combined objective in the training objective. we refer to this setting as flowprior. the generation of prior samples with flowprior comports to the training distribution while maintaining a higher diversity than competing models in our quantitative and qualitative evaluation. to summarize, this paper contributes: (1) a strategy for improved training of sentence vaes based on combining multiple lower bounds on the log marginal likelihood; (2) the first results applying real nvp to model the prior in sentence vaes; and ## background variational autoencoders (vaes; kingma and welling, 2014) are a popular framework for learning latent variable models with continuous latent variables. let x be the observed variable and z the latent variable. the model factorizes the joint distribution over x and z into a prior p ψ (z) and a generator p θ (x | z). maximizing the log marginal likelihood log p(x) is intractable in general, so vaes introduce an approximate posterior q φ (z | x) parameterized using a neural network (i.e., an "inference network"), and replace the log marginal likelihood with the evidence lower bound (elbo): maximizing the right-hand side of the equation above can be viewed as a regularized autoencoder in which the first term is the negative reconstruction error and the second is the negative kl divergence between the approximate posterior q φ (z|x) and the latent variable prior p ψ (z). it is common in practice to fix the prior p ψ (z) to be a standard gaussian distribution and only learn θ and φ @xcite @xcite . while constraining the prior to be a fixed standard gaussian is common, it is not necessary for tractability. researchers have found benefit from using richer priors and posteriors @xcite @xcite @xcite . in this paper, we consider investigating alternative priors while still using the standard gaussian form for the approximate posterior. ## choices for prior families we now describe the three kinds of priors we will compare in our experiments. the first two are based on gaussian mixtures (sec. 3.1) and the third is based on normalizing flows (sec. 3.2). we take these three prior families into consideration because they represent the three main categories of work in learning priors: simple gaussian mixtures (usually as baselines), defining the prior as a function of the approximate posterior @xcite , and flow-based priors @xcite @xcite . note that we do not make any changes to the approximate posterior distribution. that is, the approximate posterior follows a gaussian distribution with a diagonal covariance matrix as in standard vaes. ## related work when considering the parameterized family of vae models, expressive latent components (i.e., posterior and prior) have been widely studied in computer vision @xcite @xcite . however, multimodal priors have been seldom applied to language, with some exceptions @xcite @xcite @xcite . @xcite use autoregressive flow for the prior and posterior and experiment with images. @xcite propose several autoregressive nf architectures and characterize performance on character-level language modeling. @xcite design priors using the glow architecture to improve the performance of nonautoregressive neural machine translation. @xcite empirically characterize the performance of nf and simple gaussian priors in token-level latent variable models, and observe that flexible priors yield higher log-likelihoods but not better bleu scores on machine translation tasks. our work differs from that of ziegler and rush (2019) and @xcite as we are using a non-autoregressive flow-based architecture for the prior, while they are using autoregressive nf. also, we focus on models with a single latent variable for an entire sentence, while similar prior work has focused on token-level latent variables @xcite @xcite . several others have employed nf for flexible modeling in nlp. @xcite present a variational translation model that uses nf in the approximate posterior while keeping the prior as gaussian. @xcite apply nf to a variational wasserstein autoencoder in order to make the posterior more flexible. @xcite use transformed distributions via nf to model the emission density, which improves parsing performance as compared to gaussian baselines. ## conclusion we proposed a method, flowprior, that uses normalizing flow to define the prior in a sentence vae and adds the importance-sampled marginal likelihood (m is ) as a second term to the standard vae objective. our empirical results show flowprior yields a substantial improvement in language modeling and generation tasks as compared to prior work. adding m is improves performance for other models as well, especially in settings when the prior parameters are being learned.