Improving Pacing in Long-Form Story Planning (2023)

Abstract: Existing LLM-based systems for writing long-form stories or story outlines frequently suffer from unnatural pacing, whether glossing over important events or over-elaborating on insignificant details, resulting in a jarring experience for the reader. We propose a **CONC**rete **O**utline **C**on**T**rol (CONCOCT) system to improve pacing when automatically generating story outlines. We first train a *concreteness evaluator* to judge which of two events is more concrete (low-level-detailed). This evaluator can then be used to control pacing in hierarchical outline generation; in this work, we explore a *vaguest-first* expansion procedure that aims for uniform pacing. We further use the evaluator to filter new outline items based on predicted concreteness. Compared to a baseline hierarchical outline generator, humans judge CONCOCT’s pacing to be more consistent over 57% of the time across multiple outline lengths; the gains also translate to downstream stories. All code, data, and models are open-sourced.

## introduction recent advancements in large language models have led to increased interest in long-form generation, especially in creative writing settings such as stories or books @xcite . such efforts have tackled a wide range of challenges arising in longer outputs, such as longrange coherence and internal factual consistency. another important problem in longer outputs is pacing, our focus in this work. for example, it would be an unpleasant surprise for a fantasy story to summarize a major plot point as merely e.g., "the characters went on an arduous journey." conversely, it would be very odd if an entire half of the same story were devoted to a single dialogue. in fact, pacing-related issues very frequently plague llm-generated stories and story outlines. for instance, @xcite observed that their generated outlines frequently suffer from inconsistent pacing, even after hierarchically expanding high-level events to the same final depth. poor outline pacing translates directly to the resulting story, and may be exacerbated in lengthier outlines corresponding to longer stories or books, as corroborated by @xcite when writing a full-length book with @xcite . @xcite noted that some overly detailed chapters felt like "a slog," while in other cases gpt-4 would "breeze right over big important moments with a summary." therefore, we propose the concrete outline control system (concoct) to better control pacing in llm-generated story outlines. we first train a concreteness evaluator to judge which of two event descriptions is more concrete foot_0 , constructing a large training dataset of passage summaries with varied granularity that we name gpt-booksum. our concreteness evaluator can then be used in hierarchical outline generation to control or vary pacing as desired; in this work, we demonstrate its ability to maintain uniform pacing via a vaguest-first expansion procedure. we use the evaluator both to select outline nodes to expand, as well as to filter newly generated nodes based on concreteness. compared to baseline hierarchical outlines of similar length, concoct's story outlines are judged by humans to have more consistent pacing over 60% of the time without compromising other qualities (sec. 4). downstream stories based on concoct's outlines are also judged to have more consistent pacing in over 57% of cases (sec. 4.3). length-controlled generation. several summarization methods control the length of output summaries @xcite @xcite @xcite . meanwhile, some recent story generation methods use hierarchical outlines for planning @xcite @xcite , which can grant some control over the length of story passages. however, while pacing may often correlate with word length, it is not the same. rather than controlling outline sections to have similar surface-level length, con-coct focuses on a semantic notion of granularity. ## concrete outline control we now present our method, concrete outline control (concoct). concoct first constructs a concreteness evaluator m to enable pacing control in outlines. we then use m to run a vaguest-first expansion procedure to maintain uniform pacing as well as a concreteness filter for new outline nodes. ## experiments our task is to generate a story with consistent pacing, given a brief input premise from the writing-prompts dataset @xcite . ## discussion in this work, we have introduced the concoct system for controlling pacing in hierarchical story outline generation. concoct uses a concreteness evaluator to run a vaguest-first expansion procedure and to filter new outline items for concreteness, with strong results on human evaluations on both outlines and final stories. nevertheless, pacing remains far from solved. while concoct provides effective methods for measuring and controlling pacing via our concreteness evaluator, the best objective for pacing remains an open question: uniform pacing is just one of many potential goals. for example, human authors may intentionally vary story pacing, narrating major events in great detail or fast-forwarding through less important sections. accordingly, more sophisticated pacing-aware outline expansion strategies might attempt to account for nebulous concepts like story "likability," "engagingness," or "interestingness," on top of simply maintaining uniform pacing.