Narrative Order Aware Story Generation via Bidirectional Pretraining Model with Optimal Transport Reward (2023)

Abstract: To create a captivating story, a writer often plans a sequence of logically coherent events and ingeniously manipulates the narrative order to generate flashback in place. However, existing storytelling systems suffer from both insufficient understanding of event correlations and inadequate awareness of event temporal order (e.g., go to hospital <after> get ill), making it challenging to generate high-quality events that balance the logic and narrative order of story. In this paper, we propose a narrative order aware framework BPOT (Bidirectional Pretraining Model with Optimal Transport Reward) for story generation, which presents a bidirectional pretrained model to encode event correlations and pairwise event order. We also design a reinforcement learning algorithm with novel optimal transport reward to further improve the quality of generated events in the fine-tuning stage. Specifically, a narrative order aware event sequence model is pretrained with the joint learning objectives of event blank infilling and pairwise order prediction. Then, reinforcement learning with novel optimal transport reward is designed to further improve the generated event quality in the fine-tuning stage. The novel optimal transport reward captures the mappings between the generated events and the sentences in the story, effectively measuring the quality of generated events. Both automatic and manual evaluation results demonstrate the superiority of our framework in generating logically coherent stories with flashbacks.

## introduction as the fundamental element of numerous literary genres, stories have played a significant role in various social domains, such as literature, education, and entertainment. however, manually creating high-quality stories is often time-consuming and laborious. therefore, developing an ai system that † corresponding author. input the police were trying to catch a neighborhood thief. fine-tuned bart @xcite the thief was wearing a mask. the mask was fake. the police caught the thief. he was arrested. megatron-124m @xcite they had not arrested him yet. they put him in a jail cell for a month. the jail cell was full of people. they brought him to a jail cell. flashback @xcite the thief had stolen a wallet from a neighbor's car. the police arrested the thief. he was sentenced to community service. now, the police are trying to apprehend the thief! recently, a significant process has been made in story generation. similar to human writers, @xcite ; @xcite ; @xcite introduced a planning-based method, namely planning sketch prior to writing story. to improve the controllability of story generation @xcite , different attributes (e.g., style, psychology) were added to sketch. besides, @xcite ; @xcite leveraged external knowledge bases to generate commonsense stories. although existing methods can generate fluent stories related to the desired attribute, they pay little attention to event correlations, resulting in insufficient understanding of event correlations. this makes it difficult to maintain logical coherence as a whole story @xcite . moreover, they rarely consider the crucial role of narrative order in story generation. for example in flashback is a common writing technique that describes the past events to provide context for the current narrative. it arouses reader's interest and deepens their understanding of the narratives. to generate flashback in story, @xcite proposed temporal prompt. as shown in figure 1 , <@xmath0, which triggers model to reverse the narrative order to describe the past events for flashback generation ("catch a neighborhood thief" occurs later than "had stolen a wallet"). however, they face the problems of inefficient prompt and deteriorating logic in some cases (shown in section 4.5). it is probably because model always unidirectionally reasons event from order, lacking the process of inversely inferring order from pairwise events, resulting in inadequate awareness of event temporal order. additionally, when generating high-quality events in story, it is intrinsically more challenging to balance the logic and narrative order than merely focusing on logic. in this paper, we propose a narrative order aware framework bpot for story generation, which presents a bidirectional pretrained model to encode event correlations and pairwise event order. we also design a reinforcement learning (rl) algorithm with novel optimal transport (ot) reward to further improve the quality of generated events in the fine-tuning stage. specifically, a narrative order aware event sequence model is pretrained with the joint learning objectives of event blank infilling and pairwise order prediction. then, we employ ot to capture the mappings between the generated events and the sentences in the story. based on the mappings, we construct a novel reward to effectively measure the quality of generated events, further improving the generated event quality under the optimization of the rl algorithm. in summary, our main contributions include: ## preliminaries this section provides a description of the sketch used in this work, followed by an introduction to the basic concept of ot. ## methodology in this section, we detail our proposed narrative order aware framework bpot for story generation, which presents a bidirectional pretrained model to encode event correlations and pairwise event order. as shown in the upper of figure 3 , an rl algorithm with novel ot reward is designed to further improve the generated event quality. particularly, a narrative order aware event sequence model is pretrained with the joint learning objectives of event blank infilling and pairwise order prediction as shown in figure 2 . in the fine-tuning stage, we employ ot to capture the mappings between the generated events and the sentences in the story. based on the mappings, a novel reward is constructed to effectively measure the quality of generated events, further improving the generated event quality under the rl optimization. ## experiments in this section, we first introduce the datasets, the compared models, and the evaluation measures in the experiments. then, we show the experimental results and provide a detailed analysis. ## conclusions in this paper, we propose a narrative order aware framework bpot for story generation, which presents a bidirectional pretrained model to encode event correlations and pairwise event order. we also design an rl algorithm with novel ot reward to further improve the generated event quality in the fine-tuning stage. both automatic and manual evaluation results demonstrate the superiority of our framework in generating logically coherent stories with flashbacks. in the future, we will explore how to control the narrative order of long texts (paragraphs) or other narrative modalities (video).