Exploring Story Generation with Multi-task Objectives in Variational Autoencoders (2021)

Abstract: GPT-2 has been frequently adapted in story generation models as it provides powerful generative capability. However, it still fails to generate consistent stories and lacks diversity. Current story generation models leverage additional information such as plots or commonsense into GPT-2 to guide the generation process. These approaches focus on improving generation quality of stories while our work look at both quality and diversity. We explore combining BERT and GPT-2 to build a variational autoencoder (VAE), and extend it by adding additional objectives to learn global features such as story topic and discourse relations. Our evaluations show our enhanced VAE can provide better quality and diversity trade off, generate less repetitive story content and learn a more informative latent variable.

## introduction autoregressive pretrained models such as gpt-2 @xcite have been frequently applied to story generation. while gpt-2 can generate coherent single sentences, it suffers from inconsistencies in the storylines and lacks generation diversity, i.e. the storylines tend to use "bland" language and multiple generation produces similar plot lines @xcite . current story generation models add more controllability into language models for story generation, such as story plan @xcite or commonsense @xcite . these approaches focus on improving generation quality but does not address the diversity issue. variational autoencoder (vae) is an extension of autoencoder (ae) @xcite . it defines a prior distribution and the encoder learns an approximate posterior distribution that is optimised close to the prior distribution. in doing so, the vae is able to learn a more tractable latent space than ae and it is easier to sample meaning-ful latent variables to guide the generation process to generate diverse meaningful sequences. in order to leverage pretrained models for vae, @xcite propose optimus, a large-scale vae that combines bert @xcite and gpt-2 @xcite and further pretrain it on large corpus to create an off-the-shelf pretrained vae. we follow a similar approach to build our vae in this paper, but our aim is to develop a vae for domain-specific story generation (rather than creating a domain-general large-scale pretrained vae) and as such our evaluation focuses on assessing generation capability. our core innovation in this paper is the introduction of multi-task learning objectives to the vae to enhance the latent variables, as @xcite found that they tend to learn local features such as the first few words or the length of input sequences. our first auxiliary objective uses the latent variable to learn story topics, and our second objective seeks to distinguish between original stories and "negative samples", created by altering the stories to simulate common machine generation errors. we conduct experiments on several datasets to show our proposed vae has better quality-diversity trade off than gpt-2 and learn better latent representations than vanilla vae. to summarise: (1) we combine bert and gpt-2 to build domain-specific vae for story generation; (2) we propose an alternative approach to incorporate the latent variable into the vae's @xcite (3) we introduce two auxiliary objectives to encourage the latent variable to capture topic information and discourse relations; and (4) we experiment with several story datasets and show that our enhanced vae produces higher quality latent variables and generates stories with better qualitydiversity trade off compared to gpt-2. conventional approaches of automatic story generation typically contain two parts: (1) learn a language model from the training dataset with the objective of minimising kl divergence between probability distribution of training dataset and language model; and (2) find the most suitable way to decode the story from a given starting point (usually a title or the leading context) with the trained language model. autoregressive transformers such as gpt-2 @xcite and its scaled-up gpt-3 (brown et al., 2020) mask the attention heads after the current word during training so that they can serve as language models to predict the next token. however, even large pretrained language models suffer from issues such as self-repetition, conflicting logic and incoherence @xcite . therefore, recent approaches resort to two main strategies to alleviate above issues, by adding more controllability into the story generation model and incorporating commonsense knowledge. one of the most influential strategies of controllability is "plan and write" @xcite where they first use a rake algorithm to extract the most important word from each sentence and train a storyline planner based on such dataset. the language model is trained conditional on both the previous context and the keywords. during generation, the keywords are generated from the given title and can be used to guide generation of each sentence. commonsense contains shared knowledge about the world @xcite . @xcite fine-tune a pretrained gpt-2 with knowledge triples from commonsense datasets. they first use pre-defined rules to turn triples into sentences (e.g. (eiffel tower, atlocation, paris) → "eiffel tower is at paris") and train on the knowledge sentences with conventional maximum likelihood estimation objective. @xcite combine these two approaches by first training a keyword planner with gpt-2 and use the keywords to search a knowledgebase to retrieve the top ranked sentences to guide the generation process. the aforementioned approaches add complementary information in training the language model, but does not address the diversity issue in language generation. vae can generate content with more diversity @xcite , and has been variously explored in story generation. for example, jhamtani and berg-kirkpatrick (2020) treat the latent variables as story plots to guide story generation and @xcite build a hiererchical conditional vae draft and edit stories. to incorporate pretrained models for building vaes, @xcite propose optimus, a vae that uses bert @xcite as the encoder and gpt-2 @xcite as the decoder. they further pretrain optimus on english wikipedia using standard vae objectives to create an off-the-shelf pretrained vae, and demonstrate its benefits as a pretrained model for downstream tasks. we follow their approach of using bert and gpt-2 for building a vae, although with a different goal: here we are interested in developing domain-specific story generators, and as such our evaluation metrics focus on assessing story generation capabilities. story evaluation is a challenging problem, bleu @xcite and rouge @xcite are commonly used to assess the quality of generated stories. diversity of generated stories is another important evaluation aspect @xcite propose temperature sweep to evaluate the trade off between quality and diversity for story generation models. ## framework denoting the text sequence as x and the latent variable as z, a vae uses the inference model (i.e. the stochastic encoder) q ϕ (z|x) to approximate the posterior distribution, p θ (z|x), since the true posterior density p θ (z|x) = p θ (x|z)p θ (z)/p θ (x) is intractable @xcite . the prior over z is set as a multivariate gaussian p θ (z) = n (z; 0, i). vae is trained with the evidence lower bound (elbo) loss: the left part of equation can be interpreted as the reconstruction loss (l r ) and the right part as the kl loss (l kl ) that pushes the latent space close to the pre-defined prior so as to obtain a regular latent space. we use bert as the encoder and gpt-2 as the decoder to build a vae language model. bert naturally handles multiple sentences (delimited by @xcite ) and we use the @xcite token to represent the whole story and add two linear layers on top to compute the mean (µ) and standard deviation ## dataset we use four datasets in our experiments: rocstories, apnews, reuters and writingprompts. ap-news is a collection of associated press news @xcite from 2009 to 2016. reuters foot_0 is the reuters-21578 "aptemod" corpus for text categorization from the reuters financial newswire service. rocstories (roc) contains commonsense stories of five sentences @xcite . to obtain more generalization as all sentences are rather short in the dataset, we follow the delexicalization approach from prior studies @xcite where male/female/unknown names are replaced by tokens [male]/[female]/[neutral]. the writingprompts (wp) dataset consists of 303,358 human generated long stories from reddit's writing prompts forum foot_1 . @xcite collect them by scraping three years of prompts and their associated stories. we use 10% of the stories in our experiments. in terms of preprocessing, we add [sep] token at the end of each sentence and use wordpiece tokenizer for bert and byte-pair-encoding (bpe) for gpt-2. we set the maximum length of a story as 100 subwords for short story datasets (roc and reuters) and 200 for long story datasets (apnews and writingprompts). ## experiments we use implementations of bert and gpt-2 from huggingface @xcite . we set learning rate at 10 -4 and use adam @xcite as optimiser. the dimension of latent variable is set as 256. all models are trained using 20 epochs on single nvidia v100 gpu node per model. ## conclusion we explore using pretrained models such as bert and gpt-2 to build a vae for story generation. we additionally propose enhancing the vae by introducing two auxiliary objectives to encourage it to learn topical and discourse information in the stories. our experiments show that the latent variable of our enhanced vae is more informative, in that it captures the story topics and good vs. poor quality stories. in terms of story generation, we also demonstrate that our enhanced vae produce generally a better quality-diversity trade off compared to vanilla vae and gpt-2.