Generative Text Modeling through Short Run Inference (2021)

Abstract: Latent variable models for text, when trained successfully, accurately model the data distribution and capture global semantic and syntactic features of sentences. The prominent approach to train such models is variational autoencoders (VAE). It is nevertheless challenging to train and often results in a trivial local optimum where the latent variable is ignored and its posterior collapses into the prior, an issue known as posterior collapse. Various techniques have been proposed to mitigate this issue. Most of them focus on improving the inference model to yield latent codes of higher quality. The present work proposes a short run dynamics for inference. It is initialized from the prior distribution of the latent variable and then runs a small number (e.g., 20) of Langevin dynamics steps guided by its posterior distribution. The major advantage of our method is that it does not require a separate inference model or assume simple geometry of the posterior distribution, thus rendering an automatic, natural and flexible inference engine. We show that the models trained with short run dynamics more accurately model the data, compared to strong language model and VAE baselines, and exhibit no sign of posterior collapse. Analyses of the latent space show that interpolation in the latent space is able to generate coherent sentences with smooth transition and demonstrate improved classification over strong baselines with latent features from unsupervised pretraining. These results together expose a well-structured latent space of our generative model.

## introduction the state-of-the-art language models (lm) are often modeled with recurrent neural networks (rnn) @xcite or attention-based models * equal contributions. @xcite . they are optimized by making a series of next-step predictions, encouraging the models to capture local dependency rather than global semantic features or high-level syntactic properties. a seminal work by @xcite extends the standard lm to incorporate a continuous latent space which is aimed to explicitly capture global features. they formulate and train the model as a varational autoencoder (vae) @xcite . indeed, the model is able to generate coherent and diverse sentences through continuous sampling, and provide smooth interpolation between sentences, uncovering a well-formed latent space. however, training vae for text is challenging and often leads to a trivial local optimum, posterior collapse. specifically, the training objective of vae can be decomposed into a reconstruction term and a kl term that regularizes the distance between the posterior and prior of the latent variable. due to the autoregressive nature of the decoder, it is able to reconstruct the data well by simply relying on the one-step-ahead groud-truth and evolving model state while completely ignoring the latent codes. the posterior hence collapses into the prior, carrying no information. this is an important open question in this field. as pointed out in @xcite , two paths work together to generate sentences in vae. one path (path a) is through the latent codes, while the other (path b) is conditioned on the prior ground-truth or previously generated tokens. the posterior collapse describes an easy solution, that is, relying on path b and ignoring path a. prior efforts made to address this issue by and large are along the two paths. one can control the information available from path b to force the decoder to employ more path a information. @xcite dropout the input words to the decoder and @xcite utilize a dilated cnn to con-trol the size of context from previously generated words. along path a, various techniques have been developed to improve the latent code quality. @xcite anneal the weight of the kl term from a small number to reduce the regularization in the beginning of the training (anneal-vae), while @xcite further propose to adopt a cyclical annealing schedule (cyclical-vae). @xcite update the encoder multiple times before one decoder update (lagging-vae). @xcite initialize the vae with an autoencoder (ae) and adopt a hinge loss for the kl term such that kl is not driven down below a target rate (fbp-vae and fb-vae). these techniques fall under the framework of amortized variational inference. despite its fast inference, @xcite observes that an amortization gap, the gap between the loglikelihood and the elbo, can be large. thus @xcite proposes semi-amortized variational autoencoders (sa-vae) in which initial variational parameters are obtained from an encoder as in vae, and the elbo is then optimized with respect to the variational parameters to refine them. an alternative to variational inference is markov chain monte carlo (mcmc) sampling. mcmc posterior sampling may be in the form of langevin dynamics @xcite or hamiltonian monte carlo (hmc) @xcite . traditional mcmc can be time-consuming as the markov chains require a long running time, each iteration involving a gradient computation through the decoder. in this article, we propose to apply a short run inference (sri) dynamics, such as finite step langevin dynamics, guided by the posterior distribution of the latent variable as an approximate inference engine. for each training example, we initialize such a short run dynamics from the prior distribution such as gaussian noise distribution, and run a finite number (e.g., 20) of steps of updates. this amounts to a residual network which transforms the initial noise distribution to an approximate posterior distribution. one major advantage of the sri is that it is natural and automatic. designing and tuning a separate inference model is not a trivial task. in prior work, the inference model requires careful tuning to avoid posterior collapse in vaes for text modeling. for instance, the inference model needs to be aggressively trained @xcite , pre-trained with an autoencoder @xcite , or refined with gradient descent guided by the elbo @xcite . in contrast, the short run dynamics guided by the log-posterior of the latent variable can be automatically obtained on modern deep learning platforms. in addition, our method does not assume a closed-form density for the posterior, like a gaussian with diagonal covariance matrix, and hence are possible to have a good approximate posterior and provide good latent code. lastly, we optimize the hyper-parameter of the short run dynamics by minimizing the kl divergence between the short-run-dynamics-induced posterior and the true posterior, to further improve the approximate posterior. empirically, we show that the model trained with the sri is able to outperform a standard lstm language model by employing an lstm generative model, while exhibiting active utilization of the latent space, improving over models trained with vae-based approaches. moreover, we find the learned latent space is smooth, allowing for coherent and smooth interpolation and reconstruction from noisy samples, and captures sufficient global information, enabling enhanced classification accuracy over state-of-the-art baselines. in summary, the following are contributions of our paper. (1) we propose to use short run inference dynamics to train generative models for sentences without the need for an auxiliary inference network. (2) we demonstrate that the generative model trained with the sri is able to accurately model the data distribution and make active use of the latent space, exhibiting no sign of posterior collapse. (3) we show that the learned latent space is smooth and captures rich global representations of sentences. ## related work variational inference. vae (kingma and @xcite is a prominent method for learning generative models. due to the autoregressive nature of the decoder, a naive application of vae to text data results in posterior collapse. following work makes extensive efforts to alleviate this issue @xcite @xcite @xcite @xcite . among them sa-vae developed by @xcite is mostly related to our work. they propose sa-vae where initial variational parameters obtained from the inference model are further refined by running a small number of gradient updates (e.g., 20) guided by the elbo. in our work, instead of relying on a parameteric varational distribution, we run a few gradient updates on the logposterior of the latent variable with initialization from the prior distribution to draw samples directly. thus, there is no need to design and tune an extra inference model, which is highly non-trivial considering that posterior collapse occurs easily in vae training. alternating back-propagation. @xcite propose to learn generative models for images by maximum likelihood, where the learning algorithm iterates over two steps: (i) inferring the latent variable by sampling from its posterior distribution with langevin dynamics; (ii) updating the model parameters based on the inferred latent codes. in the training stage, in step (i), the langevin dynamics is initialized from the latent codes inferred in the last epoch, which is called persistent chain in the literature @xcite . in contrast, the short run dynamics always initializes the gradient descent updates from the prior noise distribution. data-independent initialization renders the dynamics in training and testing consistent. short run mcmc. @xcite introduces short run mcmc as a learned sampling dynamics guided by an energy-based model. it shares the same theoretical underpinning as early work of using stochastic gradient langevin dynamics to learn mixture of gaussians and logistic regression for large-scale data @xcite . our short run inference method for learning latent variable models for text is inspired by these works. ## experiments we apply our method to train latent variable models on text datasets. the dimension of the latent variable is 32 in all experiments. the generator is implemented with a one-layer uni-directional lstm @xcite . the number of hidden units and word embedding size of the lstm vary among datasets to closely follow the experimental setup in recent work @xcite 2019; @xcite . the number steps of the short run dynamics is 20 for all experiments foot_1 . the sample from the short run dynamics is used to predict the initial hidden state of the lstm. it is also concatenated with the word embeddings and then fed to the lstm as input at each time step. the short run inference is more computationally costly than the vanilla vae and has comparable training cost as some improved versions of vae. the number of inner steps of sri (20 steps) is about the same as that of sa-vae and lagging-vae. in training, sri has faster convergence than sa-vae and comparable convergence as lagging-vae in our experiments. in inference, our sampling-based approach is slower than amortized inference. our fb-vae a man with a cane is walking down the street . a man with a cane is walking down the street . a man in a blue shirt is eating food . people are eating food . people walk in a city . people are outside in a city . ours there is a boy skating down a small street . there is a child walking in the snow . the man is riding a horse through the snow . the man is riding a boat . the biker is looking at the lake . the person is looking at a country . ## conclusion this work proposes to use short run inference dynamics to infer latent variables in text generative models. sri dynamics is always initialized from the prior distribution of the latent variable and then performs a small number (e.g., 20) of langevin dynamics updates guided by the posterior distribution. this simple and automatic inference method induces a good approximate posterior and provides good latent code. the model trained with sri accurately models the text data compared to strong language model and generative model baselines and shows no sign of posterior collapse, which is non-trivial to avoid and several remedies have been proposed for in prior art. moreover, the learned space is smooth and captures rich representations of the sentences.