Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning (2022)

Abstract: Transformer-based language model approaches to automated story generation currently provide state-of-the-art results. However, they still suffer from plot incoherence when generatingnarratives over time, and critically lack basiccommonsense reasoning. Furthermore, existing methods generally focus only on single-character stories, or fail to track charactersat all. To improve the coherence of generated narratives and to expand the scope ofcharacter-centric narrative generation, we introduce Commonsense-inference Augmentedneural StoryTelling (CAST), a framework forintroducing commonsense reasoning into thegeneration process with the option to model theinteraction between multiple characters. Wefind that our CAST method produces significantly more coherent, on-topic, enjoyable andfluent stories than existing models in both thesingle-character and two-character settings inthree storytelling domains.

## introduction ai storytelling is a crucial component of computational creativity. humans use storytelling to entertain, share experiences, educate, and to facilitate social bonding @xcite . for an intelligent system to be unable to generate a story limits its ability to interact with humans in naturalistic ways @xcite . automated story generation, the task of requiring a system to construct a sequence of sentences that can be read and understood as a story, is a grand challenge in ai. prior to the advent of neural language models, methods to model the narrative arcs of stories leveraged a variety of statistical techniques to track events and characters @xcite @xcite . the dominant approach to story generation today is to use neural language models @xcite @xcite candidate 1: alice felt so excited to watch a sunset with ann. prompt: ann brought alice to see the sunset on the beach. after that, others (alice) should: feel happy; have fun; watch the sunset; next character: alice candidate shows this character(alice): feels happy; wants to have fun; will get sunburned; match candidate 2: after a while, alice ran a little over and sat down. feel happy; have fun; watch the sunset; feels tried; wants to relax; will calm down; match story history: ann brought alice to see the sunset on the beach. alice felt so excited to watch a sunset with ann. figure 1 : overview of the cast system . 1. a text prompt and a specified character start the story generation process. 2. a language model generates candidate continuations (two are shown) with the specified character as the main character. 3. the system infers commensense attributes about the main character from each candidate sentence. 4. if enough inferences from a candidate sentence match those from the prompt sentence, the candidate is added to the story and becomes the new prompt (here, only the first candidate meets this criterion). 5. the process repeats (with the option to specify a new main character) until a story of desired length is generated. @xcite . when a language model is trained on a corpus of stories, samples from the resulting distribution tend to also be stories. these techniques have improved with the adoption of transformer-based models, such as gpt-2 @xcite and @xcite . however, these models are prone to generating repetitive or generic continuations @xcite . furthermore, as the length of the story grows, these models can lose coherence. other artifacts include new characters being arbitrarily introduced at any time and characters being forgotten. one reason for these phenomena is that language models generate continuations by sampling from a learned distribution p θ (tok n |@xmath0. human readers, however, do not perceive the coherence of a narrative as a function of the likelihood of seeing specific continuations of previous contexts. statistical sampling from a distribution is not constrained to making logical transitions because the rich relationships that readers make to perceive coherence are not modeled. previous attempts to enhance story generation coherence use conditioning on content-relevant features such as plot outlines @xcite @xcite , or character emotional arcs @xcite . these improve plot coherence through adherence to a manually-given high-level plan. a high level plan can also be automatically generated then decomposed @xcite @xcite , which elevates the challenges of maintaining coherence to a higher level of abstraction. neural language models can also be fine-tuned on other signals such as commonsense knowledge or progression rewards @xcite , which improves the distribution but still relies solely on sampling and the assumption that language models can encode complex story structure in token distributions. the latent state of neural language models used to generate subsequent story continuations are unlikely to relate to a human reader's mental model of the state of a story world. studies of human reader comprehension @xcite @xcite show that readers comprehend stories by tracking the relations between events. specifically, reader comprehension relies on the tracking of at least four types of relations between events: (1) causal consequence, (2) goal hierarchies, (3) goal initiation, and (4) character intentions. the perceived coherence of a story is thus a function of the reader being able to comprehend how events correlate to each other causally or how they follow characters' pursuits of implicit goals. we hypothesize that a story generation system that makes decisions on how to continue a story based on tracking and reasoning about events will generate more coherent stories. unfortunately, stories don't always explicitly declare the causal consequences of events or the goals and intentions of characters. that is, sentences describing character actions or external events are rarely explicitly annotated with the characters' motivations and goals. readers must infer the characters' goals, the relationship between their actions and those goals, and how their goals change as a result of the events in their world. the ability to use basic knowledge about goals and about world states falls within the study of commonsense inference. initial work in this area was limited to modeling dimensions of the "naive psychology" of characters: motivations and emotional reactions @xcite . this was later extended to more attributes-atomic @xcite and atomic 20 20 @xcite are event-centric commonsense knowledge bases that contain logical relationships between events and the mental states and attributes of their participants, represented as typed if-then relations. the former contains 9 such dimensions, and the latter 23. comet 20 20 @xcite , an extension of comet @xcite , is a transformer-based generative model trained on triples from atomic 20 20 . given a sentence, comet 20 20 infers commonsense attributes about the characters that fall into three categories: (1) social interactions, (2) physical entities, and (3) effect of events inferred from the sentence. we hypothesize that a neural language generator informed about comet-inferred event effects as well as character intentions and goals can generate more coherent narratives. to this end, we introduce commonsense inference augmented neural storytelling (cast), which infers the causal relations between events as well as the intents and motivations of characters in the story so far in order to generate story continuations that are more coherent to readers. cast is a straightforward, cognitively inspired method to scaffold the generation of story text when sampling from a language model. by chaining sentence-level comet inferences to track important implicit elements of the story over time, cast is able to make more informed choices when sampling story continuations from a neural language model of choice (gpt-2 in our experiments). it can be used to produce both single-character and multiple-character stories. we hypothesize that stricter, more explicit constraints during generation should result in more coherent narratives than generating via sampling from a distribution alone, even if the distribution is fine-tuned. an overview of our method is presented in figure 1 . to evaluate the efficacy of our proposed method, we conduct a series of human-participant experiments that measure perceptions of logical coherence of cast against three strong neural language model story generators on three different story cor-pora. results indicate that the cast method produces significantly more coherent, on-topic, enjoyable and fluent stories in both the single-character and two-character settings. this result holds even in a genre with a very different type of commonsense than that which comet is trained on (fairy tales), indicating our method's generality. ## related work in addition to the work mentioned in the introduction, we provide a detailed background on story generation systems that emphasize commonsense reasoning and other related techniques. @xcite were the first to propose to incorporate a commonsense knowledge base into the story generation pipeline. @xcite improved upon this method by using the atomic dataset to finetune gpt-2, and then fine-tuning a second time on the rocstories corpus @xcite . this system used multi-task learning during a second fine-tuning stage with an auxiliary objective to distinguish true and engineered false stories. similarly, @xcite finetune gpt-2 on rocstories to obey coherence rules generated by separately trained models. at inferencetime, the story generation model is fed the first two and the last sentences of rocstories test instances, making this an infilling rather than open-ended generation task. brahman and chaturvedi (2020) finetune gpt-2 to generate stories that follow a given emotional arc for a protagonist, using comet to infer the protagonist's emotions as labels for their training dataset. they assume five emotions (anger, fear, joy, sadness, neutral) limited to two changes throughout the story, associated with the xreact and oreact inferences produced by comet. we do not assume a fixed set of commonsense inference values, and we assume a character's state may change at each new sentence. the c2po system @xcite ) uses comet to generate successor and predecessor events instead of a language model, performing a bi-directional search from a given start event and a given end event. c2po assembles the narrative directly from the short, templated sentences produced by comet. it also assumes the end of the story is known in advance. like @xcite , it focuses on only two dimensions of comet and only works for a single character. our work models interactions between multiple characters and takes advantage of a richer set of inferences that comet and comet 20 20 provide, better aligning with the four types of relations key to reader comprehension @xcite @xcite . very recently, @xcite utilizes a bartbased commonsense inference model in conjunction with an event generation model to place eventrelated constraints on the story generation process. we acknowledge high-level similarity in approaches between our framework and this work, but we do not include this approach in our comparisons since it is concurrent. storytelling research focused on improving longrange cohesion is not limited to using commonsense resources-goldfarb-tarrant et al. (2020) perform high-level planning via plot outline generation using principles from aristotle's poetics, then use a language model in fill in details. they demonstrate strong performance on the writingprompts @xcite dataset. while one of their models' purpose is to determine whether to reuse or introduce a new character, they do not explicitly model inter-character relationships or character attributes during generation. we compare to this baseline in our experiments. ## the cast inference method we now introduce our neural storytelling framework, commonsense inference augmented neural storytelling (cast), which scaffolds the conventional text generation process by imposing constraints on the sampling process at inference time. the conventional setup for k-sentence story generation starts with a given first sentence s 1 , referred to as the prompt, and generates k -1 subsequent sentences conditioned on it. cast follows this convention, generating the ith sentence as follows: 1. we condition a fine-tuned language model on the story up to the current sentence [s 1 , . . . , s i-1 ], followed by a token signifying the main character of sentence i ( §3.1). 2. we obtain a set of commonsense inferences for s i-1 ( §3.2) and use them as constraints at the decoding stage of sampling a next-sentence candidate c from the language model ( §3.3). 3. we obtain a set of commonsense inferences for candidate c, and match commonsense inference sets between s i-1 and c using a matching criteria, producing a score for c ( §3.2). 4. if the score is above a specified threshold, c is selected to be s i and is appended to the gener- comet comet language model prefix candidate inferences prompt inferences candidate match? no yes @xmath1. owant: to have a good time' oreact: excited; happy * [char_2] * [char_1] had a big crush on [char_2]. [char_2] wanted to go to prom with [char_1]. decoding xintent: to have fun xattr: excited; romantic [char_2] ation history. otherwise, steps 2 through 4 are repeated until a viable candidate is found. 5. we repeat steps 1 through 4 until k-1 sentences have been generated. an illustration of the pipeline is given in figure 2 . in practice, when generating two-character stories, we specify main characters in an alternating manner to promote turn-taking (more details in §3.1). cast is not limited in application to the maximum story length seen during training, and can be used to generate stories of arbitrary length. ## conclusions neural language models generate content based on the likelihood of tokens given a historical context. human readers, on the other hand, use complex inferential processes to connect the relationships between events. this mismatch between generative models and reader comprehension is one of the reasons why stories generated by neural language models lose coherence over time. our cast system is a straightforward approach to enforce the constraint that a language model only generate continuations that cognitive psychology tells us will be more comprehensible. the cast method provides hard constraints to neural language model generation that results in greater story coherence, a result that holds in multiple storytelling domains. we find that perceived story enjoyability and fluency are tied to making logical sense, tracking character goals, and staying on topic; our system excels in all four of these areas.