Content Planning for Neural Story Generation with Aristotelian Rescoring (2020)

Abstract: Long-form narrative text generated from large language models manages a fluent impersonation of human writing, but only at the local sentence level, and lacks structure or global cohesion. We posit that many of the problems of story generation can be addressed via high-quality content planning, and present a system that focuses on how to learn good plot structures to guide story generation. We utilize a plot-generation language model along with an ensemble of rescoring models that each implement an aspect of good story-writing as detailed in Aristotle’s Poetics. We find that stories written with our more principled plot-structure are both more relevant to a given prompt and higher quality than baselines that do not content plan, or that plan in an unprincipled way.

## introduction despite many recent advances in natural language generation, successful creative narrative composition remains elusive. current neural approaches are plagued by difficulty in mastering structure, will veer between topics, and lack long-range cohesion. they successfully imitate the fluency and style of human writing, but on closer inspection sentences do not fit together to form a whole, and the reader is left with the impression that the generation has no content (see et al., 2019). this lack of structure also degrades the relevance of generations conditioned on a prompt or other source text -a strong language model will repeat key phrases from a given prompt but will not remain on topic. these issues are illustrated in the naive generated story in we hypothesise that this problem can be addressed with a focus on deeper latent narrative structures. in aristotle's poetics, one of the most enduring treatises on the craft of writing good stories, the philosopher lays out the elements of story in order of importance. they are: this philosophical framework fits remarkably well into the traditional natural language generation pipeline approach that emphasizes content planning (reiter and dale, 1997). the pipeline divides generation into three steps: content planning, microplanning and surface realization, where at each step input is modified and refined, getting closer to the final textual output. incorporating a plot in order to generate stories can then be viewed as a proxy for content planning/microplanning before a language model makes use of it to convert it to a readable and grammatically correct natural language output (surface realization). inspired by both the aristotelian and content planning frameworks, we develop a novel system for story generation. we focus on developing a system that can learn to expertly select events, characters, and relevant content, and write good plot structures. after the work on the plot is complete, a large language model can then do what it does best and fill in the descriptions, details, and local specifics of each story. prompt: write the most beautiful end to the universe that you can imagine naive plot with semantic role labelling tags: <@xmath0: <@xmath1: "you're dead. i'll be dead, and there'll always be no more of you."* *"no, my dear. it's not true!" -the last of the world is dead, * * "it'll never be over. the universe will n't end. not yet. but i'll see it, one way or another. there is no other way."* *my last thought, the dying man left me. he told that i was the last human to live. and he told me the universe would end, and the end will come soon. that your death would be the last one to die, and the first one to be the most beautiful. * "there is a universe of life left, my dear, and that's all there is. generated story (from revision): it was the end of the universe. the light filled the sky. the galaxy. all the light in all the worlds, and all the stars. a bright flash. the first light. and then a bright flash of light... then a flash of darkness. then a bright light. a flash of blackness and then the stars began to grow. it was a bright, beautiful bright, light. and then, slowly the stars started to grow in number. there were millions and millions of stars.. but, slowly but surely, all the galaxies began to fade. slowly, but surely. finally, it looked at me with its eyes. one dying star. gold story: it slept, and dreamt of twirling lights. it cradled countless twirling lights, all dancing in their flashing ways, dancing to the sound of a faraway flute. the light was vibrant and warm and spun all around it. the tiny lights grew with the tune, and more lights came to dance within the luminescence. it was surrounded by light, all waltzing in their ways to the distant music. it, too, swirled with the swelling symphony, watching the lights. there were lights of all different sizes and shades and brightness, but each was its own, unique light. each one was brilliant. each one was beautiful. all dancers to the song... for plot generation, we employ a few eventchoice and event-arrangement rescoring models which assist in building the arc and cohesion of the plot, a character rescoring model that helps select which characters appear where, and a relevance model that is responsible for keeping the plot structure and the story on topic. as both improving plot-generation via rescoring and using an aristotelian framework for neural generation are novel concepts, there is no previous work on how to implement them in practice. our contributions are: 1) we propose to leverage the principled aristotelian framework for content planning, 2) we propose an implementation of the framework using a revision-based approach via several rescoring models 3) we show strong experimental results against 4 baselines. ## background existing work in neural story generation has established the strength of adding a content planning stage to structure the generated content (yao et al., 2019; fan et al., 2019) (discussed in more detail in section 7). specifically, this line of work trains a pipeline with one model that generates from @xmath4. it modifies the standard conditional generation task with a source @xmath5...x n (in this case, a prompt) and target @xmath6...y n (in this case, a story) to condition also on an abstract intermediate representation z. note that the approach is not truly modelling p(y|x) since that would involve summing over all z. instead, it models p(y,z|x) = p(z|x)p(y|z, x), but only shows the generated story y at inference time. this is a more controllable task than opendomain generation conditioned on only a prompt x, provided that a good interim structure z can be learnt. we follow this line and explore ways to improve plot planning to close the gap between stories generated from gold plots and those from model-generated plots. plot representation as there are no large datasets with parallel gold-standard plots and stories, all work on plot generation depends on silverstandard plots extracted from stories. we follow fan et al. (2019) to represent plots in semantic role labelling (srl) format. we run coreference resolution to identify entities, and use a compression algorithm to discard less salient information. @xcite figure 1 : the full story generation pipeline. base generation models in gray. aristotelian rescorer input in color, with example positive & negative training pairs. plot structure shown with srl tags for verbs and their arguments. ## approach we focus on learning the best interim plot structure between the input prompt and the story surface realisation. as such, we learn the plot model p(z|x) and the story model p(y|z, x) by fine-tuning a pretrained conditional language model bart (lewis et al., 2019) on 1) pairs of prompts and extracted silver standard plot-structures, and 2) pairs of prompts + plots and stories, respectively. full implementation details can be found in appendix a.3. we propose to modify the decoding objective to incorporate input from each aristotelian rescoring model @xmath7, detailed further in section 3.1) and re-rank the original, or "naive" plot model hypotheses, bringing the plot representation closer to each rescoring model's specialty and desireable story attribute. a diagram of our final system in figure 1 shows each step of the generation process. the modified decoding objective becomes: ## experimental setup dataset. we use the writing prompts dataset (fan et al., 2018), which is a large collection of user-generated stories along with their associated prompts from reddit, to benchmark our models. it ## analysis we analysed the patterns in reported user confidence and found that 8% of prompts are low user confidence (<3) and 8% are high confidence (>4.5), so we look further into these as examples where the top three systems are minimally and maximally distinguishable. we include examples of outputs for these prompts for all three models in appendix c. for overall quality, there are no low confidence titles, and the aristotelian plot system is preferred @xcite examples from each baseline system for the prompt in to assess the cases where the aristotelian plot did not improve over the bart baselines, we measure both word and verb incorporation rates, in while the high incorporation rates show that the story model does utilize the plot, there is a gap between the current utilisation and the possible upper bound. the focus of our work is on improving plot generation, but we hypothesise that modifications to the story model to improve incorporation rates would further widen that performance gap between the three systems, as it will give the plot more influence over the story surface realisation and ensure that plot improvements appear downstream. story generation without plots. diverse efforts have focused on generating stories. fan et al. (2018) re-purpose an approach for neural machine translation to translate from prompt to a story via convolutional seq2seq models. guan et al. (2020); mao et al. (2019) use a similar approach, however they incorporate structured commonsense knowledge from external datasets or knowledge bases to improve a story generated from a prompt. peng et al. (2018) add control to the story ending valence. story generation with plots. riedl and young (2010) use refinement search as a technique to balance between character and plot for solving the narrative generation problem. li et al. (2013) use plot graphs for story generation that model the intended logical flow of events in the virtual world as a set of precedence constraints between plot events. martin et al. (2018) decompose the problem of story generation into generation of successive events (event2event) followed by generation of natural language sentences from events (event2sentence). ammanabrolu et al. (2020) follow up this work by comparing five different models for event2sentence realisation, and find all to have different weaknesses. yao et al. (2019) improve their lstm-generated rocstories by ex-21 yao et al. ( 2019 ) also use word incorporation rates, but theirs are not comparable to ours, as we both include verbs as their own separate metric (due to their importance in our structure) and make this an ordered metric (rather than set intersection as they do), which is necessary because our generated stories are much longer tracting keyword-based plot-like structures, or storylines, and using these in a pipeline to generate storylines and then stories. fan et al. (2019) experiment with numerous techniques for representing story plots on the writingprompts dataset, and find semantic role labelling with entity anonymization and compression to work best. more recently, tambwekar et al. (2019) propose a goal-driven reinforcement learning approach to plot generation, and chen et al. (2019) propose a latent variable model to learn how to generate outlines for neural story generation. learning story aspects via rescoring. holtzman et al. ( 2018 ) generate continuation sentences from context sentences, and introduce using a mix of collaborative discriminators that each learn one grician maxim of conversation, and use them to rerank their rnn story model output. goldfarb-tarrant et al. (2019) use those discriminators with the system of yao et al. (2019) as part of a collaborative story writing task with an lstm and human writers. however, none of them apply rescorers to the plot. there is no other work that uses discriminators or rescores for plot structures or to try to train them based on different principles. ## conclusion and future work we have shown that content planning via an interim plot structure representation can be combined with the use of rescoring models to inject aristotelian story-writing principles into the plot. we found that this results in stories that are both more relevant and higher quality than stories that are generated directly from prompts or that use plots without aristotelian rescoring. our findings also suggest future work on additional ways to incorporate story principles into plot generation. although our aristotelian plots improved over the naive plot, there remains gaps in quality between generated and gold plot structures. there is also further work to be done in investigating what models are best able to incorporate plots, which would enable plot improvements to be even more effective.