Affective and Dynamic Beam Search for Story Generation (2023)

Abstract: Storytelling’s captivating potential makes it a fascinating research area, with implications for entertainment, education, therapy, and cognitive studies. In this paper, we propose Affective Story Generator (AffGen) for generating interesting narratives. AffGen introduces ‘intriguing twists’ in narratives by employing two novel techniques—Dynamic Beam Sizing and Affective Reranking. Dynamic Beam Sizing encourages less predictable, more captivating word choices using a contextual multi-arm bandit model. Affective Reranking prioritizes sentence candidates based on affect intensity. Our empirical evaluations, both automatic and human, demonstrate AffGen’s superior performance over existing baselines in generating affectively charged and interesting narratives. Our ablation study and analysis provide insights into the strengths and weaknesses of AffGen.

## introduction stories have been a central part of human cultures for millennia, shaping societies, identities, and beliefs @xcite . however, the question of why some stories captivate us while others leave us indifferent remains intriguing. while humans can skillfully craft interesting narratives, even the most recent ai models cannot compose stories that can engage the reader for long enough. in this work, we address the task of automatically generating interesting stories. automatically generating interesting stories could potentially help cognitive studies by revealing patterns that make stories interesting. from an application perspective, the capability to generate interesting stories could revolutionize the fields like entertainment @xcite , education @xcite , and even therapy @xcite . while large language models (llms), such as gpt @xcite , have been de facto figure 1 : two example stories. story 1 is an interesting story with an intriguing twist (highlighted in orange color) that was produced by affgen using dynamic beam sizing. story 2 is a relatively less interesting story with a straightforward and predictable plot. the concept of "interesting stories" is also highly subjective and context-dependent @xcite . previous research in the field increases "interest" in the story by structural planning to control specific aspects of the story, e.g. modeling the emotional flow of the protagonist @xcite or incorporating flashbacks @xcite . however, such methods ignore that text complexity and quality also raise its interestingness @xcite . bradley and lang (1999) advocates for decorating the plot with affective terms to increase the suspense and intensity of the story that results in control of the audience's emotions @xcite . with this motivation, we propose affective story generator (affgen) 1 that con-trols text coherence and leverages words' affective dimensions to promote text interestingness. our method is based on two key ideas. first, in beamsearch-based decoding of language models, occasionally exploring larger beams can help in generating slightly lower probability but potentially more interesting words. second, switching between large and small beams can help in maintaining the balance between coherence and interestingness. we use these ideas to generate stories with an intriguing twist. figure 1 shows an example of an interesting story, story 1, with an intriguing twist (highlighted in orange color) that was produced by dynamically using different beam sizes. it also shows an uninteresting story, story 2, that used a comparable language model but with a constant beam size. to generate an interesting story, affgen first identifies where to generate the intriguing twist that would push the story to be more interesting. then it generates the intriguing twist using two novel techniques, i.e. dynamic beam sizing and affective reranking. in dynamic beam sizing, affgen uses a contextual bandit model @xcite to dynamically explore different beam sizes thus encouraging the model to select words that are less predictable and more intriguing without compromising coherence. in affective reranking, affgen reranks possible candidates for the sentence to be generated according to their arousal and valence scores @xcite , thereby modulating the emotional dynamics of the story. our automatic and human evaluations show that stories generated by affgen are more engaging than the baselines without sacrificing coherence. our ablation studies and analysis provide deeper insights into the functioning of affgen our contributions are: ## related works we discuss two lines of related work that are closely relevant to this study. ## problem statement given a sentence, s 1 , as a prompt that represents the first sentence of a story, our goal is to generate an interesting story represented as a sequence of generated sentences s 2 , s 3 , . . . , s n . each sentence is a sequence of tokens. in this paper, one of these generated sentences serves as the intriguing twist in the narrative. ## controlled affective story generator this section presents the controlled affective story generator (affgen), a narrative generation model designed to produce interesting stories. affgen operates in two key stages. first, it identifies the position of the sentence that should contain the intriguing twist, p it ( §4.1). then, it generates the story in the left-to-right manner using a language model. for generating sentences that do not contain the intriguing twist, it uses a standard decoding algorithm since the focus is on maintaining narrative coherence ( §4.2). for generating the sentence that contains the intriguing twist, it uses our proposed decoding algorithm based on dynamic beam sizing and affective reranking since the focus is on balancing emotional arousal, interestingness, and coherence ( §4.3). ## conclusion this paper addresses the task of generating interesting stories. for this, we present, affgen, a language model that uses a novel contextual banditbased decoding mechanism. this new decoding mechanism enables the affgen to dynamically explore different beam sizes and rerank based on affective quality of the text. our experiments indicate that affgen can generate interesting but 8 grayson wanted to bake his brother a birthday cake, so he got out his mom's recipe book and started searching for the perfect one. he decided to go all out and make a four layer chocolate cake with cream cheese frosting. his brother was so impressed that he wolfed down 2 pieces in a row.