Guiding Neural Story Generation with Reader Models (2022)

Abstract: Automated storytelling has long captured the attention of researchers for the ubiquity of narratives in everyday life. However, it is challenging to maintain coherence and stay on-topictoward a specific ending when generating narratives with neural language models. In this paper, we introduce Story generation with ReaderModels (StoRM), a framework in which areader model is used to reason about the storyshould progress. A reader model infers whata human reader believes about the concepts,entities, and relations about the fictional storyworld. We show how an explicit reader modelrepresented as a knowledge graph affords the storycoherence and provides controllability in theform of achieving a given story world stategoal. Experiments show that our model produces significantly more coherent and on-topicstories, outperforming baselines in dimensionsincluding plot plausibility and staying on topic

## introduction automated story generation is the challenge of designing an artificial intelligence system that can generate a natural language text that is perceived by readers as a story. early work on story generation used symbolic planning @xcite @xcite @xcite @xcite . these systems would be provided with a description of the initial world state-usually a list of predicates-and a goal-a description of what predicates should be true to be successful. these approaches had two benefits. first, the plots tended to be coherent because of logical constraints on the actions. second, the plots were guaranteed to end in a state in which the goal held. however, these systems require substantial knowledge engineering of logical constraints, limiting their generality, and don't always generate plot or stories in natural language. shower jenny wet bathtub water commonsense inference overlapping inference candidate entity similairty similairty jenny go to beach florida swim miami water beach jenny beach florida swim miami water shower bath tub sun shine figure 1 : the overview of storm system. our goal is to generate a story on a prompt for reaching the given goal. 1. the system builds a goal world state by converting natural language text into knowledge graph and then expands it with commonsense inference . 2. storm builds a prompt story world and then infers a set of concepts on each entity in prompt story world. 3. for every concept , storm obtains a candidate reader model by inferring commonsense relations on it. 4. storm chooses the topk concepts which maximize its similarity with goal world state. 5. a language model generates continuation options on the topk concepts, which are added to the story. recently, neural language modeling approaches @xcite @xcite @xcite @xcite have been applied to story rocstories writing prompts fairytale storm charles had always wanted to have a college degree. he was admitted to a college. finally, he graduated from college. he was glad he finally ::: got ::::: his :::::: diploma. jennifer wander in hallway. she thought she had found something on the ground. when she looked at the ground, there was nothing. she then kept walking in the dark. she cannot see any light. then, she fell on ground. she was in a lot of pain. when she got back up, she fell again. this time, she got back up and just walked away. "hello?" she asked. she walked toward kitchen. hero consumed them. then, he got very sick and died. he buried himself. then, he reborned. :: he :::::: hero invoked command to his magic sword. hero wants to prove his skills. hero wants to show off. hero wants to be in charge. hero wants to be king. he had again assumed his place as king of golden mountain. we consider the challenge of coherent and controllable text generation for neural language model based story generation. we hypothesize that neural language models, while powerful text-completion systems, are not natively well-suited for coherent story generation because a neural network trained with a cross-entropy loss function is unlikely to model the unfolding context of a story the same way as a human reader. studies of human reader comprehension @xcite show that readers comprehend stories by tracking the relations between entities and events in ways that can be expressed as a graph. the per-ceived coherence of a story is a function of the connectedness of this graph @xcite . ensuring the causality between sentences can improve the coherence of stories @xcite . inspired by cognitive science, we aim to augment neural language models with a reader model in which a story generation system infers a graph of concepts, entities, and relations that a reader is likely to believe about the story world as they read an incrementally generated story. the reader model enables the story generation algorithm to explicitly reason about the entities and relations and generate story continuations that use those entities to move the story forward; a reader can track how entities and relations change over time and thus perceive stories as more coherent. we use large language models to produce the continuation text of the story generation. however instead of only providing the previous story as context, our algorithm also selects one or more entities from the world model and uses template filling to generate candidate continuations. the reader model also provides a means for directing the generation process. in addition to a starting context prompt, we require a goal to be given. the goal is natural language text which is then converted into a knowledge graph (see fig. 1 ). the goal knowledge graph provides a rough outline of the entities and relations that need to be present in the story but without providing particulars about everything that must be in the story or the ordering in which they must occur. our contributions are as twofold: (1) we propose an automated story generation model with reader models (storm) which maintain coherence and controllability of generated stories at the same time; and (2) we conduct a thorough experimental study against strong baselines which shows that storm produces significantly more coherent and goal-directed story. ## related work and background we situate our paper in the literature of neural networks-recurrent and transformer-based-to produce stories @xcite @xcite @xcite . there are a few works that are highly related to our proposed framework, in terms of the following two dimensions: the generation controllability and the usage of commonsense knowledge. the controllability in story generation focuses on how to enable the generation process to adhere to the user's inputs. @xcite conducts generation in two steps: planning a story plots based on a prompt, then generating a story by filling the mask tokens in plots with bart @xcite . plot machines @xcite accepts as an input an un-ordered outline of concepts and conditions a language model. commonsense knowledge plays an important role in story generation. the most popular way of utilizing it is to train neural language models (e.g. gpt-2 @xcite ) on commonsense knowledge bases such as conceptnet @xcite and atomic @xcite which contains detailed information regarding well-known facts or causal relationships. thus the resulting language model, named comet @xcite , becomes capable of inferring new commonsense knowledge on novel phrases. given a sentence, comet infers commonsense attributes about the characters in three categories: (1) social interactions (i.e. xneed indicates what the character of this sentence needed), (2) physical entities, and (3) effect of events inferred from the sentence. @xcite proposes causal, commonsense plot ordering (c2po) framework which takes advantage of comet to infer predecessor and successor events and then bidirectionally search from pre-specified start event to end event, however, c2po generates plots made up of highly constrained, templated text; @xcite leverages comet to infer the character intentions and effects of actions so as to guide the generation process, but they did not consider controllability. there are also other approaches that directly incorporate commonsense knowledge graphs into the encoding process @xcite . compared with them, the novelty of our paper is to improve coherence and controllability at the same time with the help of external commonsense knowledge graph. ## story generation with reader models in this section, we introduce a framework-story generation with reader models (storm)-for generating stories with models of what the reader will believe about the fictional story world. we hypothesize that the incorporation of a reader model into the story generation process will increase story coherence. we define story coherence as the extent to which readers can identify connections between different events and entities in a story. in this work, the reader model is represented as a knowledge graph, a set of triples of the form ⟨subject, relation, object⟩. by making the beliefs about what the reader likely knows explicit, we provide mechanisms for selecting which entities to include in the continuation of the story. because the storm framework maintains a knowledge graph that approximates the reader's beliefs about the story world, we are able to compare the reader model to a desired goal world state, also described as a knowledge graph. the storm framework is thus controllable. our framework starts with a prompt and a set of goals (see fig. 2 ), which are natural language texts. the prompt and goals are transformed into knowledge graphs by extracting entities ( §3.1), referred as reader model and goal world state (see yellow bubbles in fig. 2 ). these two knowledge graphs are expanded using commonsense techniques ( §3.2; see red bubbles in fig. 2 ). for every inferred entity in the reader model, we further expand the reader model with its commonsense inference to obtain a candidate knowledge graph, which is scored based on how similar it is relative to the goal world state ( §3.3). entities of which the candidate kg shares the topk similarity with the goal world state are selected and the generation technique uses templates to generate possible story continuations ( §3.4; see blue bubbles in fig. 2 ). by targeting different entities and using template infilling, we reduce neural network hallucination of new entities and create a diverse set of story continuations. the selected continuation starts the next iteration of the generation process. story examples can be found in ## experiments after proving that the knowledge graph acquisition technique is able to capture the information that natural language story conveys in appendix c. we evaluate our system with two experiments, comparing storm to two strong neural language model story generators on the the dimensions of coherence and controllability. ## conclusions neural language models are widely used to produce text, including stories. however, they struggle with maintaining story coherence-the logical progression of events-and goal-directedness. our framework-story generation with reader models (storm)-augments neural language models with a reader model. this reader model-an explicit knowledge graph-approximates the reader's beliefs about the story world. storm increases the story coherence by producing continuations that directly reference entities in this reader model. goaldirectedness is achieved by choosing continuations that add desired entities to the reader's inferred set of beliefs. a thorough experimental study shows that storm produces significantly more coherent and goal-directed stories than two strong baselines on three datasets.