Plug-and-Play Controller for Story Completion: A Pilot Study toward Emotion-aware Story Writing Assistance (2022)

Abstract: Emotions are essential for storytelling and narrative generation, and as such, the relationship between stories and emotions has been extensively studied. The authors of this paper, including a professional novelist, have examined the use of natural language processing to address the problems of novelists from the perspective of practical creative writing. In particular, the story completion task, which requires understanding the existing unfinished context, was studied from the perspective of creative support for human writers, to generate appropriate content to complete the unfinished parts. It was found that unsupervised pre-trained large neural models of the sequence-to-sequence type are useful for this task. Furthermore, based on the plug-and-play module for controllable text generation using GPT-2, an additional module was implemented to consider emotions. Although this is a preliminary study, and the results leave room for improvement before incorporating the model into a practical system, this effort is an important step in complementing the emotional trajectory of the story.

## introduction in this study, the authors, one of whom is a professional novelist, examined the use of natural language processing to solve the problems faced by novelists from the perspective of practical creative writing. among the diverse topics related to automatic storytelling and human creativity, "emotion" should be emphasized as an important keyword. the relationship between stories and emotions has been an essential part of the research in the field of humanities, especially in the cognitive and affective science of literature @xcite @xcite @xcite . in providing practical knowledge for authors, creative techniques emphasize the importance of being conscious of readers' emotions @xcite . the theory of the emotional arc, which states that a good story can be typified by emotional movement, is well known from the introduction by a popular american novelist, @xcite . as presented in @xcite , studies have been conducted to reveal the close relationship between emotions and stories. @xcite insisted that a key component of every character is emotion. in the context of serious storytelling, @xcite insisted that a fundamental aspect of storytelling is emotions, that is, the cognitive aspects that the story evokes in its audience. numerous efforts have been made to disclose the mystery of the relationship between emotions and stories @xcite @xcite @xcite . this study focuses on introducing emotions into a story completion (sc) task. the basic task setting in sc is shown in figure 1 . 1 in the field of story generation and understanding, @xcite proposed sc. we believe that the artificial intelligence (ai) ability to solve sc tasks is important in the context of providing creative support. if writers cannot complete a story and do not know how to proceed with a plot, a suitable model can provide them with appropriate support. the main contributions of this study are as follows: astonished excited aroused happy delighted glad pleased content satisfied at ease serene calm relaxed sleepy tired droopy bored gloomy depressed sad miserable distressed annoyed frustrated tense angry afraid alarmed (excited) (calm) (positive) (negative) arousal valence story completion with emotion control high arousal, negative valence high arousal, positive valence ⃝ pplm is used to control the emotions of the generative text. the representation of the emotions in this figure was reconstructed from an image by @xcite . this study is a preliminary study, and the results should be improved before incorporating the model into a practical system. however, we believe that this effort is an important step toward complementing the emotional trajectory of the story and worth discussing for future directions. as a complementary contribution to this study, we would like to note that a professional writer researched how to use natural language processing (nlp) technology to reflect the viewpoints of writers and researchers. we expect that this work will contribute to building a bridge toward collaborative work between professional writers and researchers in nlp and human computer interface (hci) to accelerate research in the field of story writing assistance. ## methods this section describes the proposed method in detail, emphasizing the ingenuity of its implementation. the proposed model has a novel architecture composed of two main parts for sc tasks. studies on applying unsupervised pre-trained large neural models for text infilling have been actively conducted recently @xcite @xcite . the first part of our method follows this trend and is verified using various models. in subsection 3.2, a modified version of pplm @xcite is proposed for emotionaware sc. pplm, given a prompt (user input text), generates subsequent sentences, as it uses gpt-2 as a base model and tiny attribute models. in this study, the pplm model was expanded through concatenation with other models. the model code was implemented using py-torch @xcite , which is an opensource machine-learning framework provided as a python library. 2 to make use of unsupervised pre-trained large neural models, our code was also based on huggingface transformers @xcite , which provide general-purpose architectures for natural language understanding (nlu) and natural language generation (nlg). the focus here is mainly on seq2seq language models (seq2seqlms). for seq2seqlms and its variants, the models below were used. pytorch version 1.11.0, and huggingface transformers version 4.18.0 were used. 4 the details of pre-trained models are displayed in ## discussion the no-emotion-aware baseline results indicate that bart large exhibited the highest scores for every metric. in this study, we used two sets of training parameters: one is based on summarization task-specific parameters and the other is common parameters. the result showed that the parameters for summarization work well for story completion, compared to common parameters that do not account for differences between models. future studies should search for specific parameters for each model that are more suitable for sc. in this study, pplm was extended and combined with bart, a representative model of seq2seqlms. in addition, by combining pplm with the nrc-vad lexicon, a basis was created for sc to consider valence and arousal. however, there is still a lot of room for improvement in the in text generation, it is important to control the behavior of the model using parameters such as the length penalty. two types of parameters were experimented with in this study, but further effort is required to determine the best parameter. the optimal hyperparameters seem to be naturally different for each model. it is not realistic to check storyid f2a013bd-852f-43f4-9012-4db8ae44c64e context jane had a very sick dog. her dog was old and couldn't run anymore. so that he still felt young, jane used to walk her dog in a pram. <@xmath0. missing_id 3 gt this would look strange to the public. bart base one day, her dog fell down and broke his leg. bart large her dog got very sick and couldn't run anymore. pegasus large one day, her dog got sick and had to be put down. t5 base one day, she noticed that her dog was very sick. t5 large one day, her dog got sick and couldn't walk. ## considerations by a professional writer as noted in the introduction, one of the authors of this study was a professional novelist. this work is a collaborative effort between researchers and a professional creative writer. more precisely, the first author of this paper is a professional japanese novelist as well as a researcher in the field of story understanding and generation. in section 6, the viewpoint of the researchers is discussed. in this section, the positioning and prospects of this study are discussed from the novelist's perspective. in an experiment conducted separately from this study, four professional creative writers were asked to evaluate a creative writing support system. 11 the results of that experiment confirmed that there might be a negative perception of the system's ability to control the output if there are parameters with which the user is not familiar. although it would be desirable for users to have the freedom to adjust the outcome, too many parameters make them lost. they do not know what to do, resulting in confusion on the user's part in using the system and in a negative impression. as previously mentioned, our modified pplm for controllable sc addressed in this study is difficult to adjust. moreover, in its current state, users are required to understand what "valence" and "arousal" mean. we believe that treating both dimensions rather than one dimension (positivenegative) would be important for future directions in this area, but this idea is not yet widespread. hence, it is difficult for this approach to provide professional writers with the desired results for now. at this point, there was concern that other professional writers would have a negative impression of the "creative writing support system that controls the emotions of the generated text" as a whole. that is why no human evaluation was conducted on this study, except by the novelist author. for practitioners, the extent to which ai could replace their own work is an important issue; there is also concern that it could trigger a sense of avoidance toward ai. prudence is needed in conducting research, and professional evaluations, which are important topics of discussion. some professional novelists write from beginning to end in order, while others come up with certain parts but cannot come up with the correct sentences to fill in the gaps. sc is an important task in helping the latter. from the creative writer's perspective, it is helpful to have a system that understands the meaning of one's own writing and 11 the details of the human evaluation consist the part of the doctoral dissertation of the first author. the dissertation will be publicly available in the utokyo repository, https: //repository.dl.itc.u-tokyo.ac.jp/ . ## conclusion in this study, the sc task was considered for various emotions. previous studies on emotion-aware story generation have restricted emotions to one dimension (positive-negative) or categorical ones. our aim was to control more diverse emotions, so the issue of two-dimensional control was addressed based on russell's circumplex model. our implementation made it possible to control sc using pplm. this expands the application of pplm, which was originally limited to the task of "generating the continuation of a prompt." although the goal of controlling emotions was accomplished, it was difficult to adjust the parameters. whether this difficulty in coordination can be improved through innovative implementation or demands a completely different approach requires further examination.