Narrative Text Generation with a Latent Discrete Plan (2020)

Abstract: Past work on story generation has demonstrated the usefulness of conditioning on a generation plan to generate coherent stories. However, these approaches have used heuristics or off-the-shelf models to first tag training stories with the desired type of plan, and then train generation models in a supervised fashion. In this paper, we propose a deep latent variable model that first samples a sequence of anchor words, one per sentence in the story, as part of its generative process. During training, our model treats the sequence of anchor words as a latent variable and attempts to induce anchoring sequences that help guide generation in an unsupervised fashion. We conduct experiments with several types of sentence decoder distributions – left-to-right and non-monotonic, with different degrees of restriction. Further, since we use amortized variational inference to train our model, we introduce two corresponding types of inference network for predicting the posterior on anchor words. We conduct human evaluations which demonstrate that the stories produced by our model are rated better in comparison with baselines which do not consider story plans, and are similar or better in quality relative to baselines which use external supervision for plans. Additionally, the proposed model gets favorable scores when evaluated on perplexity, diversity, and control of story via discrete plan

## introduction maintaining long-term narrative flow and consistency are important concerns when aiming to generate a plausible story @xcite . prior work on narrative text generation has focused on generating consistent stories via story outlines using keywords or key phrases @xcite , event-based representations @xcite @xcite , plot graphs @xcite figure 1: our aim is to generate a story given a title. we propose models which first generate a high level story plan realized via a sequence of anchor words. however, stories do not naturally occur with a tagged set of such anchor words or keywords. many prior works use off the shelf tools to first label stories with plan outlines, thus using external supervision for learning plot structures. for example, @xcite use the rake heuristic @xcite to first identify the most important keyword in each sentence, and then use this to train a model in a supervised fashion. this approach leads to improved coherency and control, but creates a reliance on such heuristics and does not jointly learn anchor words along with the generator. inspired by prior work indicating that anchor words can effectively capture and control high-level generation structure, we investigate to what extent high-level control can be learned in a fully unsupervised fashion, directly from natural story data. we design a hierarchical latent variable model (figure 2 ) that induces sequences of anchor words that explain observed stories, while at the same time learning to generate entire stories by first generating anchor sequences. for training, we use amortized variational learning @xcite , where an inference network is used to approximate the posterior on anchor sequences. at test time, given a title, we first sample a sequence of anchor words using the prior model conditioned on only the title, and then generate the actual story using the decoder conditioning only on the title and the sampled anchor words. to induce a useful latent generation plan and to effectively condition on a sampled plan, we propose a constrained story decoder and constrained inference network. specifically, our constrained decoder begins a story sentence by deterministic copying the corresponding anchor word, and then generates words to the left and then to the right (figure 3 ). for this decoder, the corresponding true posterior on anchor words is sparse: the anchor word must be chosen from the observed sentence. thus, we constrain the output vocabulary of the corresponding inference network to the words of the input sentence. we observe that the proposed constrained inference network does not suffer from mode collapse, leading to models which can effectively learn useful anchor words. further, we also contrast this approach with a model whose decoder is not constrained to use each anchor word in each sentence. the true posterior in this case is over the full vocabulary. we conduct experiments with both constrained and unconstrained decoders and inference networks, and find that the best results are achieved through the combination of an unconstrained decoder with a constrained inference network -indicating, perhaps, that while it is more effective to use flexible models, using a constrained inference network can add a useful inductive bias, leading the model to mimic the constraint of the inference network. we experiment with two english story datasets, and observe that our best models achieve favorable scores relative to several baselines when evaluated on perplexity, diversity, coherency, and controllable story generation as per various automatic and human evaluations. finally, we note that our modelling approach for story generation has an interesting connection with work that treats text as a latent variable in deep generative models @xcite . we treat a latent sequence of anchor words as a form of hierarchical control over generated outputs, while related work treats the latent sequence itself as sequential text that is the output of the model. ## model our goal is to generate a story x, consisting of multiple sentences x 1 , x 2 , ..x k , given a title t. our model's generative process is depicted in figure 2 and operates as follows: first, a sequence of anchor words representing a generation plan is sampled from an auto-regressive prior conditioned on the title. next, for each anchor word, a sentence is generated conditioned on the anchor words and previously generated sentences using a decoder. during training, the sequence of anchor words is unobserved and treated as a latent variable. as described in more detail later, we will explore several choices of decoder -those that treat anchor words as an explicit token in the sentence to be generated, figure 3 : simplified demonstration of generation of a sentence conditioned on anchor words and preceding sentences for the two types of decoders: (1) unconstrained decoder is based on the story generation model of @xcite , which may or may not use the corresponding anchor word. (2) constrained decoder is forced to use anchoring words in corresponding sentences, generating words to the left and then to the right of an anchor word. ## learning and inference our goal is to maximize the log likelihood of the stories conditioned on the corresponding titles. since z is unobserved at training, we must marginalize over all possible values of z. we shall refer to the first term as the reconstruction term and the second term as the kl-term. we make a mean-field assumption in the posterior approximation on z as follows: note that p(z|t) is autoregressive, and thus it is intractable to exactly compute the kl term. we resort to monte carlo sampling to approximate the elbo by drawing samples from inference network; though we will perform this differently for the kl term and the reconstruction term (more details in section 3.2). ## experiments we evaluate and report generation quality of various models using automatic metrics for fluency and diversity, as well as human evaluations for coherence of story and relevance to title. we also analyze the ability of anchor words to control the generated story, and highlight comparisons between various choices of inference networks and decoders. ## related work prior work on story generation has largely focused on plot outline via keywords or key phrases @xcite , event-based representations @xcite , or a sentence theme @xcite . @xcite propose a method to generate a story conditioned on a character description. prior work on narrative text generation with plans has mostly relied on external resources or tools to extract outlines @xcite , and then training in a supervised manner. for example, using vader @xcite to tag sentiment polarity @xcite . much prior work has used manually defined objectives to encourage coherence in generated text. in this context, reinforcement learning has been used to encourage stories to follow certain manually defined goals such as being locally coherent @xcite . prior work on visual story generation aim to learn topically coherent visual story generation @xcite . compared to topics, keywords provide more fine-grained plan, and thus are more likely to provide fine-grained control over generated outputs. in this work we have proposed a constrained inference network and a constrained decoder for story generation. pointer networks @xcite have been used for amortized inference in prior work on summarization @xcite , though in a semi-supervised context. non-monotonic sequence generation has been explored in past for tasks such as machine translation @xcite . in the proposed model, the generation plan can be used to control the story via the anchor words. hard and soft constraints for incorporating keywords into generation have been explored in @xcite ; @xcite . controllable text generation has been explored in other tasks as well, such as summarization @xcite , paraphrasing @xcite , style transfer @xcite , and data-to-text generation @xcite . ## conclusion in this work we have proposed a deep latent variable model which induces a discrete sequence of anchor words as a high-level plan to guide story generation. 2 we train the models though variational learning using a constrained inference network, and compare constrained and unconstrained versions of the decoder. the proposed model performs similarly or better than baselines on various automated and human evaluations. related approaches might be used more broadly for a variety of language generation tasks, or even for related domains like music generation. other modeling extensions might explore richer structure in latent plans -for example, generalizing beyond isolated words. finally, in this work we trained decoders from scratch, though it would be interesting to explore the incorporation of large pretrained models such as gpt2 @xcite to increase fluency. by copying the corresponding anchor word, generating words to the left and then to the right of it. thus lap-cinf-cdec model commits to using the corresponding anchor words. in contrast lap-cinf-udec has more flexibility in using the anchor words, and sometimes anchor words get dropped or get used in a paraphrased manner. e.g. for sample s2, the model did not use the word 'tired' in the last story sentence, though the second last sentence evokes similar understanding. however, this also means the control of the story via the plan is somewhat weaker. e.g. for s4, the story does not use the anchor word 'boiled'. finally, compared to outputs from lap-cinf-udec and lap-cinf-cdec, we observe that supervplan outputs demonstrate much weaker control of the plan on the story, and generate plans less relevant to the title. lap-cinf-udec title: n/a anchor words: reunion together way together members story: we got together to have a reunion. we went to a small town. we had a lot of fun. we took many pictures. then we all got together to talk to some families. lap-cinf-udec title: n/a anchor words: wedding wedding people food beautiful story: i went to the wedding last weekend. there were a ton of people at the wedding. there were a lot of people there. the food was delicious. the cake was beautiful . noplan-lm title: n/a anchor words: n/a story: this is the first day of my vacation. i like to have a trip to location. i have never been to location. here is my apartment. i've been hoping to be there. i'm so excited.