Re3: Generating Longer Stories With Recursive Reprompting and Revision (2022)

Abstract: We consider the problem of automatically generating longer stories of over two thousand words. Compared to prior work on shorter stories, long-range plot coherence and relevance are more central challenges here. We propose the Recursive Reprompting and Revision framework (Re3) to address these challenges by (a) prompting a general-purpose language model to construct a structured overarching plan, and (b) generating story passages by repeatedly injecting contextual information from both the plan and current story state into a language model prompt. We then revise by (c) reranking different continuations for plot coherence and premise relevance, and finally (d) editing the best continuation for factual consistency. Compared to similar-length stories generated directly from the same base model, human evaluators judged substantially more of Re3’s stories as having a coherent overarching plot (by 14% absolute increase), and relevant to the given initial premise (by 20%).

## introduction generating long-term coherent stories is a longstanding challenge for artificial intelligence, requiring a comprehensive grasp of linguistic, world, and commonsense knowledge @xcite . recently, many works have automatically generated short stories ranging in length from five sentences to one or two paragraphs @xcite @xcite @xcite . while stories of such length serve as a good test bed for text generation, they are much shorter than typical short stories meant for human consumption, which are often several pages in length. in this work, we aim to bridge some of this gap by generating much longer "short" stories: the final generated stories in our experiments are 2000-2500 words. we are the first to automatically generate plot-coherent stories of such length, with further generate a setting, characters, and outline by prompting a language model. a new law grad returns home to start her career, but struggles with the broken justice system. liza turner pulled up in front of the house where she'd grown up. little had changed since she was a teenager... ## recursive reprompting and revision we now describe our recursive reprompting and revision framework (re 3 ), which decomposes the human writing process into our plan, draft, rewrite, and edit modules. see appendix k for concrete examples of each component in practice. figure 3 : illustration of the prompt constructed in re 3 's draft module to generate each next story continuation. our recursive reprompting approach combines pieces of the plan (blue) and previously generated story (grey) into a single prompt by concatenating the depicted components in order. for each point of the outline, we will generate several story passages before moving on to the next outline point. each passage is generated as a fixed-length continuation from a structured prompt, which is composed by our recursive reprompting procedure as shown in figure 3 . the prompt begins with a selection of "relevant context" shown at the top of figure 3 . as the story progresses, we dynamically update the list of character descriptions using a namedentity-recognition-based pipeline, which identifies new entities from each new story passage using flair @xcite and writes descriptions using gpt3-instruct-175b. thus "relevant context" initially contains all of the premise, setting, and characters shown in figure 2 , but subsequently selects only what is most relevant to the most recent story passage using a pretrained dense passage retrieval (dpr) model @xcite . the remainder of the prompt can be viewed as a coarse-to-fine description of the previous story, following the intuition that an author needs detailed information about the most recent passage but perhaps only higher-level information about much earlier passages. as shown in figure 3 , we include "previous sections' outlines" as a very highlevel summary of previous larger story sections, followed by a "recent story summary" written by gpt3-instruct-13b 3 of a few penultimate passages. at the end we repeat verbatim the immediately preceding passage as "autoregressive context" from which point the story should continue. finally, to enforce relevance to the current outline point, we include the "current section outline" in the prompt just before "autoregressive context." finally, the full prompt is fed to gpt3-175b to generate the next story passage. 4 ## discussion we have considered the problem of automatically generating longer stories, proposing the re 3 framework as an initial attempt at addressing the challenges of maintaining long-range coherence and premise relevance. our re 3 implementation exhibits strong performance on these metrics while generating stories over 2000 words long. at its core, re 3 is a system for emulating the human writing process for long-form generation while leveraging only general-purpose language models in the generation procedure. thus concepts from re 3 can potentially be adapted to non-story domains as well, especially the idea of dynamically re-injecting contextual information into a prompt. moreover, should human interaction be desired, re 3 is in principle highly controllable: most modules operate almost entirely in natural language. nonetheless, our main goal remains to further improve automatic long-form story generation. while re 3 's stories are an order of magnitude longer than those from prior work, most humans would still consider them to be "short stories"-and on the shorter side at that. our long term goal is to generate interesting, long-range-coherent stories of greater length-perhaps what humans might call "novellas"-and eventually full-length novels. one step in this direction could be to extend re 3 using multiple levels of hierarchical outline generation to obtain a much more detailed initial plan, as we do in appendix m to generate a 7500-word story. in our view, the greatest barrier to further increasing story length is evaluation, which frustrates efforts to benchmark systems during both test time and development. in this work, we have compared re 3 to baselines solely through human evaluation, which can be both noisy as well as costly even with non-expert annotators. while prior works have proposed some possible measures @xcite , we hope that analyzing our generated stories (both re 3 and baselines) can inspire further research on metrics for which we currently rely solely on human annotation. for example, while there exist reasonable metrics for text similarity on a sentence or paragraph level, long-form generation could benefit from metrics detecting when a longer passage begins on-topic but slowly veers off-topic, or when a passage uses on-topic vocabulary but is otherwise nonsensical in context. similarly, improved metrics for long-range factual contradictions could greatly aid efforts to improve generations' factual consistency, such as our edit module. even if new metrics do not completely replace human annotations, they could help us both to evaluate longer stories as well as conduct more detailed ablation studies with larger sample sizes. additionally, while re 3 's stories are relatively plot-coherent and faithful to the premise, substantial gaps remain along other axes compared to even beginner human writers. one such axis is long-range factual continuity: while we believe our structured detection-correction method is a humanlike approach, our current edit module is certainly not human-level. moreover, human stories exhibit long-range continuity along many axes other than just factual attributes of characters, such as overall theme; scenes and world setting; pace and tempo of storylines; and foreshadowing before major events. it remains highly nontrivial to incorporate such considerations into automatic story generation.