Finding and Generating a Missing Part for Story Completion (2020)

Abstract: Creating a story is difficult. Professional writers often experience a writer’s block. Thus, providing automatic support to writers is crucial but also challenging. Recently, in the field of generating and understanding stories, story completion (SC) has been proposed as a method for generating missing parts of an incomplete story. Despite this method’s usefulness in providing creative support, its applicability is currently limited because it requires the user to have prior knowledge of the missing part of a story. Writers do not always know which part of their writing is flawed. To overcome this problem, we propose a novel approach called “missing position prediction (MPP).” Given an incomplete story, we aim to predict the position of the missing part. We also propose a novel method for MPP and SC. We first conduct an experiment focusing on MPP, and our analysis shows that highly accurate predictions can be obtained when the missing part of a story is the beginning or the end. This suggests that if a story has a specific beginning or end, they play significant roles. We conduct an experiment on SC using MPP, and our proposed method demonstrates promising results.

## introduction currently, because of the internet, anybody can freely publish their original stories. however, it is challenging to write something that people would like and want to read. sometimes, even professional writers fall into slumps during the writing process. numerous studies on understanding the secret of creating good stories have been conducted @xcite . rules for creating stories have been studied extensively, and "three-act structure" @xcite and "save the cat" @xcite are popular examples. these works can help guide people who want to write good stories to demonstrate their creativity. with the development of machine learning (ml) and natural language processing (nlp) technology in recent years, the creation of an automated system that supports the creative endeavors of people is now feasible @xcite @xcite . to assist people in creating stories, it is essential to train computers to understand and create stories. to measure the reading comprehension abilities of systems regarding stories, @xcite proposed the "story cloze test" (sct). in the sct, four sentences are presented, and the last sentence is excluded from an original five-sentence story. the objective of this task is to select an appropriate sentence from two options that complement the missing last sentence. based on this approach, @xcite proposed the "story completion (sc)" task in the field of generating and understanding stories. given any four sentences of a five-sentence story, the objective of this task is to generate the sentence that is not given (known as the missing plot) to complete the story. the ability to solve the sc is essential in the context of creative support. if writers cannot complete a story or a plot, a suitable model can provide them with the appropriate support. however, such applications are currently restricted because they require the user to know which part of a story is missing in advance. when considering an actual application, writers do not always know where their writing is flawed, as evidenced by the vital role of editors who work with them. of course, the editors do not just point out the missing points; they play various roles. for example, they also point out unnecessary parts. keeping this in mind, let us understand one of the roles of the editor. to overcome this limitation, we propose a new story comprehension method named "missing position prediction (mpp)," as shown in figure 1 . an incomplete story with one sentence missing is given as input. unlike in the sc, no information regarding the position of the missing content is given. this task aims to predict the position of the missing part. the ability to solve this task indicates that computers can identify flaws in a story's plot. in practical applications, by combining our novel task with an appropriate sc method, writers can benefit from computerized completion even if they do not know where a flaw is. jennifer has a big exam tomorrow. she got so stressed, she pulled an all-nighter. she went into class the next day, weary as can be. her teacher stated that the test is postponed for next week. jennifer felt bittersweet about it. jennifer has a big exam tomorrow. she got so stressed, she pulled an all-nighter. she went into class the next day, weary as can be. _______________. jennifer felt bittersweet about it. jennifer has a big exam tomorrow. she got so stressed, she pulled an all-nighter. she went into class the next day, weary as can be. jennifer felt bittersweet about it. additionally, we propose a novel method for mpp and sc. given an incomplete story, it estimates the missing part and generates a sentence to complete the story. we make our code available to support further progress on our proposed task and sc. 1 our main contributions are as follows: ## task description we begin by formulating seg and the sc, after which we formulate our proposed task. ## proposed method hierarchical approaches have demonstrated effectiveness in story generation @xcite . we propose a novel method with a hierarchical architecture for the mpp and sc. we devise a method inspired by the two-step encoder of hierarchical-seq2seq, which is a simple method for seg that we proposed in our previous study @xcite . the first encoder receives @xmath0, ..., s k-1 , s k+1 , ..., s n } and outputs the sentence embeddings {v 1 , ..., v k-1 , v k+1 , ..., v n }. next, the second encoder receives the sentence embeddings and generates a distributed representation of the entire context v context . we call the first encoder "sentence encoder," and the second encoder "context encoder." for mpp, we input v context into a linear layer and obtain a five-unit output. for sc, we input v context into a language model and obtain a sentence to complete the story. figure 2 shows an overview of the proposed method. although the output of the mpp can be used here, we prefer to have our model learn these two tasks simultaneously. we intend to take it up as future work to use predicted mpp for sc and vice versa. in proposing a new task, we believe it is useful to test how well a simple method can solve the task. analyzing the performance and characteristics of a simple method will help in the application of complex methods in future studies. thus, we propose a new task along with a simple method. ## experiment 1: missing position prediction first, we worked on learning the mpp only. in this experiment, we investigated the part of the proposed method that excludes the language model. rocstories is a well-organized corpus and is widely used in story-generation tasks; it is typically used in seg @xcite @xcite . similarly, @xcite used it for their story-completion task. furthermore, the dataset was used by @xcite for controllable story generation. @xcite tackled "counterfactual story rewriting," which is a story revising task, using their proposed timetravel dataset built using rocstories. although, initially, we did consider using other datasets, such as writingprompts, we ultimately did not use them. stories in writingprompts vary in terms of length, and therefore, the importance of a single sentence varies from one story to the other. thus, considering the requirements of our analysis, the aforementioned dataset seemed inappropriate. ## experiment 2: missing position prediction + story completion based on the results of experiment 1, we conducted another experiment in which we tackled both mpp and sc. as a context encoder, we used the gru context. we used the same dataset as in section 5.1. ## discussion the results of experiment 1 support the following two findings from @xcite : 1) the plot becomes more complicated as it progresses, thereby making the estimation of latter sentences more difficult and 2) for @xmath1, four sentences in the context are continuous. therefore, a good expression can be easily obtained, even by using an encoder that does not consider discontinuity. it is interesting to note that the beginning or the end of a story can be predicted with the highest accuracy. this appears to be related to the fact that the collection of rocstories was performed with the following in mind: "the story should read like a coherent story, with a specific beginning and ending." in other words, the story under consideration has a specific beginning and ending. thus, if the beginning or the ending is missing, it can be interpreted as that the methods treat the story as particularly unnatural and predict the missing position with high accuracy. for qualitative analysis on experiment 2, we show three examples of story pairs and human evaluations in ## conclusion to overcome the issue of conventional sc tasks that require information regarding the position of the missing part in a story, we proposed a mpp to predict the position based on the given incomplete story. our proposed method demonstrated that treating the context as a sequence is useful for solving this new task. we examined the prediction accuracy for each missing position and found that a prediction is easier if the beginning or the end of a story is missing. furthermore, we tackled the combined task of mpp and sc. we conducted a pair-wise human evaluation against a human-written story, for which our proposed method demonstrated promising results. because we limit the study to five-sentence stories, it is unlikely that humans make mistakes in the plot. however, humans may overlook plot imperfections when considering longer, more complex stories. thus, checking such mistakes is part of the editors' job. we proposed the task in the context of creative support, but it also can be positioned in the context of narrative understanding. planning a story requires a form of reasoning that can move backward as well as forward. that is why sc tasks have significant meaning in story understanding and generation, and our proposed task would be a better test of a model's abilities to understand the flow of a story. for the sake of simplicity, we proposed a simple machine-learning-based method. however, using simple bag-of-words methods or part-of-speech analysis may be effective for our proposed task. therefore, exploring the efficacy of using methods other than those based on machine learning is left as future work. however, our proposed task poses specific limitations. in our task, it is known that there is a missing position in the input story, and that there is only one such instance. in reality, an input story may be complete, that is, k is null. furthermore, there may be a case in which there are multiple missing positions, that is, a case in which k has multiple values. although dealing with these constraints is left for future studies, it is conceivable to introduce a certainty factor for the missing prediction. for example, predicting that k is null when the certainty factor is low. although we considered a constrained case of study, we believe that our proposed task is an important step toward assisting writers in the creation of stories. context since the questions were complicated, i was extremely nervous. despite believing that i've failed, i turned the exam in. the teacher handed the exams back to us the next day. i ended up receiving a b. gt i took my class final in math today. since the questions were complicated, i was extremely nervous. despite believing that i've failed, i turned the exam in. the teacher handed the exams back to us the next day. i ended up receiving a b. ours my teacher gave us a test. since the questions were complicated, i was extremely nervous. despite believing that i've failed, i turned the exam in. the teacher handed the exams back to us the next day. i ended up receiving a b. answers with reasons (a: gt, b: ours) both whether it is a class final or a given test, both stories are the same and therefore both complete. neither both doesn't make sense ours a is jumbled and does not make sense. b is logically arranged as a story. ours in "a," it wouldn't make sense that a final exam was handed back in class the next day. ours b was more appropriate since it is having a continuous flow than a context tom was at a local park. there was an egg hunt for the kids. tom decided to pick some eggs up. he enjoyed the treats in them. gt tom was at a local park. it was easter. there was an egg hunt for the kids. tom decided to pick some eggs up. he enjoyed the treats in them. ours tom was at a local park. there was an egg hunt for the kids. tom decided to pick some eggs up. tom was able to get many eggs. he enjoyed the treats in them. answers with reasons (a: gt, b: ours) both both are complete sentences ours option b is complete as it says that tom was able to get some eggs in the hunt. both both of them can be considered complete. story a tells us it is easter (and story b doesn't) while story b tells us tom picked many eggs (and story a doesn't). both of those details could be removed and the stories would still be the same.