Affective and Dynamic Beam Search for Story Generation
Authors: Tenghao Huang
Ehsan Qasemi
Bangzheng Li
He Wang
Faeze Brahman
Muhao Chen
Snigdha Chaturvedi
Venue: EMNLP 2023 Findings (2023)
Keywords: Discourse Analysis
Sentiment Analysis
Affective Computing
TL;DR: This paper introduces a new tool for generating engaging narratives, the Affective Story Generator (AffGen), which enhances the appeal of stories by introducing 'intriguing twists'.

Abstract: Storytelling's captivating potential makes it a fascinating research area, with implications for entertainment, education, therapy, and cognitive studies. In this paper, we propose Affective Story Generator (AffGen) for generating interesting narratives. AffGen introduces `intriguing twists' in narratives by employing two novel techniques—Dynamic Beam Sizing and Affective Reranking. Dynamic Beam Sizing encourages less predictable, more captivating word choices using a contextual multi-arm bandit model. Affective Reranking prioritizes sentence candidates based on affect intensity.  Our empirical evaluations, both automatic and human, demonstrate AffGen's superior performance over existing baselines in generating affectively charged and interesting narratives. Our ablation study and analysis provide insights into the strengths and weaknesses of AffGen.

Affective and Dynamic Beam Search for Story Generation
Tenghao Huang' Ehsan Qasemi'
Bangzheng Li'
He Wang" Faeze Brahman' Muhao Chen
Snigdha Chaturvedi
University of Southern California
*Columbia University
Alleen Institute for Artificial Intelligence *University of California, Davis
University of North Carolina
{ [tenghaoh, qasemi, bangzhen Juscc. edu;
hw2687@columbia edu; muhchen@ucdavis..o edu;
faezebeallenai. org;
snigdha@cs .nc. edu
Abstract
zzZ
Storytelling's s captivating potential makes fascinating research area, with implications for entertainment, education, therapy, and cogni- tive studies. this paper, we propose Affective Story Generator (AFFGeN) for generating in- teresting narratives. AFFGEN introduces 'in- triguing twists' in narratives by employing two novel techniques- DDnnamic Beam Sizing and Affective Reranking. Dynamic Beam Sizing captivating word choices using contextual multi-arm ban- dit model. Affective Reranking prioritizes sen- tence candidates based on affect intensity. Our empirical evaluations, both automatic and hu- man, demonstrate AFFGEN's superior perfor- mance over existing baselines in generating affectively charged and interesting narratives. Our ablation study and analysis provide in- sights into the strengths and weaknesses of AFFGEN.
Jason winter vacation at a ski resort. He accidentally hit avalanche trap! quickly grabbed his gear and rushed to the nearest shelter. However, the avalanche followed buried him and the shelter.
Jason winter vacation at ski resort. He had been waiting for
year. was an experienced skier, and the fresh powder was perfect for some extreme skiing. Nothing could stop him now.
Story Interesting Narratives
Story Plain Narratives
Figure Two example stories. Story is an interesting story with an intriguing twist (highlighted in orange color) that was produced by AFFGEN using dynamic beam sizing. Story is relatively less interesting story with straightforward and predictable plot.
encourages less predictable,
winners in generating coherent text, their prowess in creating narratives that captivate human inter- est leaves much to be desired. LLMs' coherence is mainly rooted in their training objective that in- centivizes text likelihood which is not necessarily correlated with human quality judgements (Holtz- man et al., 2019; Zhang al., 2021) writing style (Gehrmann et al., 2019). The concept of "interesting stories" is also highly subjective and context-dependent (Roemmele, 2021). Pre- vious research the field increases "interest" the story by structural planning to control specific aspects of the story, e.g. modeling the emotional flow of the protagonist (Luo et al., 2019; Brahman and Chaturvedi, 2020) incorporating flashbacks (Han et al., 2022). However, such methods ignore that text complexity and quality also raise its inter- estingness (Schraw et al., 2001).
1 Introduction
Stories have been a central part of human cultures for millennia, shaping societies, identities, and be- liefs (Kasunic and Kaufman, 2018). However, the question of why some stories captivate us while oth- ers leave us indifferent remains intriguing. While humans can skillfully craft interesting narratives, even the most recent AI models cannot compose stories that can engage the reader for long enough. In this work, we address the task of automatically generating interesting stories.
Automatically generating interesting stories could potentially help cognitive studies by reveal- ing patterns that make stories interesting. From an application perspective, the capability to gener- ate interesting stories could revolutionize the fields like entertainment (Akoury al., 2020; Thue et al., 2007), education (Zhao al., 2022), and even ther- apy (Gabriel and Young, 2011). 1).
Bradley and Lang (1999) advocates for decorat- ing the plot with affective terms to increase the suspense and intensity of the story that results in control of the audience's emotions (Delatorre et al., 2016). With this motivation, we propose Affective Story Generator (AFFGEN)! that con-
While large language models (LLMs), such as GPT (Radford et al., 2019), have been de facto
'Code is available at https://gittub..om tenghaohuang/AFFGGNN git

---

trols text coherence and leverages words' affective dimensions to promote text interestingness. Our method is based on two key ideas. First, in beam- search-based decoding of language models, occa- sionally exploring larger beams can help in generat- ing slightly lower probability but potentially more interesting words. Second, switching between large and small beams can help in maintaining the bal- ance between coherence and interestingness. We use these ideas to generate stories with an intrigu- ing twist. Figure shows an example of an in- teresting story, Story 1, with an intriguing twist (highlighted in orange color) that was produced by dynamically using different beam sizes. It also shows an uninteresting story, Story 2, that used a comparable language model but with constant beam size.
2 Related Works
We discuss two lines of related work that are closely relevant to this study.
Story Generation. Early research on story genera- tion explored symbolic planning methods (PÉrez and Sharples, 2001; Porteous and Cavazza, 2009; Riedl and Young, 2010) that used predefined rules and structures to generate stories. Later efforts used neural methods (Jain al., 2017; Peng et al., 2018; Fan et al., 2018; Puduppully et al., 2019; Zhai et al., 2019; Yao et al., 2019; Wang et al., 2021; Peng et al., 2022).
However, generating interesting stories has mained challenge due to the subjective nature of "iinerestingness" (Roemmele, 2021). Some pre- vious work has attempted to generate interesting stories by controlling specific aspects of the gen- erated content, such as modeling emotions (Luo el al., 2019; Brahman and Chaturvedi, 2020), flash- backs (Han et al., 2022), personas (Zhang et al., 2022), topics (Lin and Riedl, 2021), and social re- lationships (Vijjini et al., 2022). Alhussain and Azmi (2021) pointed out factors that could lead interesting narratives, such as suspense (Tan and Fasting, 1996), discourse (Genette, 1980), and char- acters (Liu et al., 2020). This work differs from these approaches in the sense that it focuses on generating interesting content by choosing more af- fective, and not necessarily high-likelihood, words.
To generate an interesting story, AFFGEN first identifies where to generate the intriguing twist that would push the story to be more interesting. Then it generates the intriguing twist using two novel techniques, i.e. Dynamic Beam Sizing and Affec- tive Reranking. dynamic beam sizing, AFFGEN uses a contextual bandit model (Thompson, 1933) to dynamically explore different beam sizes thus encouraging the model to select words that are less predictable and more intriguing without compro- mising coherence. In affective reranking, AFt'Ggn reranks possible candidates for the sentence to be generated according to their arousal and valence scores (Mohammad, 2018), thereby modulating the emotional dynamics of the story.
Sampling strategies for decoding. One of the commonly used strategies in neural text (and story) generation is Nucleus Sampling (Holtzman et al., 2019). This method involves selecting subset of the vocabulary, called the nucleus, from which the next word is sampled. Another strategy is Top-k Sampling (Fan et al., 2018), which only considers the k most probable words for the next word. Meis- ter et al. (2023) proposed an cormmattoon-hhorette strategy, Locally Typical Sampling, with the aim of making the modcl's output more human-like
Our automatic and human evaluations show that stories generated by AFFGEN are more engaging than the baselines without sacrificing coherence. Our ablation studies and analysis provide deeper insights into the functioning of AFFGEN
Our contributions are:
We propose the task of generating interesting stories.
We propose AFFGEN, a language model that uses a novel contextual bandit-basee decoding algorithm and explores dynamic beam sizes and affective reranking.
Our approach differs from these existing strate- gies in two key perspectives. First, while previous works primarily aim to encourage generation flu- ency and diversity we focus on including more affective terms during decoding. Sccond, use re-scoring, which involves adjusting the probabili- ties of the words based on additional criteria, rather than solely relying on the logits distribution gen- crated by the model. This allows us to further enhance the diversity and affective quality of the
We conduct automatic and human evaluations to empirically demonstrate that AFFGEN can produce interesting and coherent narratives.
We conduct ablation studies and analysis to further understand the working of AFFGEN.
2

---

generated text.
Our data-driven approach utilizes the Writing- Prompts dataset (Fan al., 2018), collection of auman-written stories. We use this dataset form a distribution, D(n), which corresponds to the probability of observing the intriguing twist at the th sentence. During inference, AFFGEN samples a relative position 1.r from D(n) to pin- point the location of the sentence that would be the intriguing twist in the story that will be generated
3
Problem statement
Given a sentence, $1, as a prompt that represents the first sentence of story, our goal is to generate an interesting story represented as sequence of generated sentences $2, 83. ................................ SN- Each sentence is sequence of tokens. In this paper, one of these generated sentences serves the intriguing twist in the narrative.
R1T D(n).
4 Controlled Affective Story Generator
Next, we discuss how AFFGEN generates the various sentences of the story.
This section presents the Controlled Affective Story Generator (AFrGEN), narrative generation model designed to produce interesting stories. AFFGEN operates in two key stages. First, it identifies the position of the sentence that should contain the in- triguing twist, Prr ($4.1). Then, it generates the story in the left-to-right manner using language model. For generating sentences that do not contain the intriguing twist, it uses a standard decoding al- gorithm since the focus on maintaining narrative coherence (84.2). For generating the sentence that contains the intriguing twist, it uses our proposed decoding algorithm based on Dynamic Beam Siz- ing and Affective Reranking since the focus is on balancing emotional arousal, interestingness, and coherence (84.3).
4.2 Base Storyteller
For generating sentences that do not contain an intriguing twist (s;'s Vi & {I,n)r)), the focus is maintaining narrative coherence. We use GPT-based language model (Radford al., 2019; Brown et al., 2020) which has shown promising performance on story generation (Brahman and Chaturvedi, 2020; Clark and Smith, 2021). We fine-tune the language model on a dataset of stories ($5.1) by minimizing the negative conditional log- likelihood:
NLL log Il p(w; Ju 1.-1) (1)
where w;; 's represents the tokens of the story. We use beam search for inference in this model.
4.1 Position of the intriguing twist
Narratives are highly structured texts. Freytag's pyramid (Freytag, 1908). widely recognized model of narrative structure, delineates the story into five key components: exposition, rising action, climax, falling action, and resolution.
4.3 Generating Intriguing Twist
To generate the sentence that contains the intriguing twist in the narrative, Syr, we use the fine-tuned language model from $4.2 but with a novel beam search-based decoding. Our decoding method uses Dynamic Beam Sizing and Affective Reranking to produce interesting text.
Given the prompt sentence, $1, our objective is determine the most suitable location for the cli- max or the intriguing twist, 7/T 32. N). There has been some work on identifying the cli- max or turning point a given story (Ouyang and McKeown, 2015; Wang et al., 2022; Vijayaragha- van and Roy, 2023). We employ a data-driven ap- proach inspired by the work of Wilmot and Keller (2020). Their methodology operates on the premise that if the embedding of two sentences is suffi- ciently distant, the latter sentence can be deemed unexpected or interesting with respect to the former sentence. They use this idea to identify the sentence that presents the turning point or intriguing twist in a narrative.
Dynamic Beam Sizing. The motivation behind our beam search-based decoding algorithm is that while small beam size helps in producing coher- ent text, by expanding the beam size of the PLM, we can explore slightly lower probability but po- tentially more intriguing words. However, main- taining large beam size throughout is also not desirable because not all words in a sentence need to be interesting. large beam throughout can also slow down the inference process and require more resources. So during inference, the model needs to dynamically switch between large and small beam
3

---

sizes balance the tradeoff between the coherence and interestingness of the generated text.
trigger) as feature.
3. Sequence length: Length of the partial sen- tence generated so far, SIT,t-1- Knowing where the model is, in terms of position, can help it decide whether to generate an interesting token next. 4. Perplexity: The model's perplexity on the partial sentence generated so far, S1T, This helps in maintaining coherence.
To address this, we introduce Dynamic Beam Sizing, where depending on the context, the model decides the beam size before generating a token. For practical purposes, we assume that the beam size can take one of k values (b',b2..b' "..).. and the model has to choose one. We cast the problem of choosing beam size contextual k-arm bandit problem (Langford and Zhang, 2007), where the arms of the bandit are the various beam sizes. The bandit's choice of beam size at time step or trial, t, depends on the the context the bandit. The context considers the tokens generated so far for the intriguing twist sentence, STT. We use SIT,t-1 to refer to the sequence of tokens in this partial sentence and represent the context using following features:
For choosing arm b (b', the bandit also receives a payoff. The payoff ac- counts for all the candidate sequences in the beam {c',
Each c is basically a concatena- tion of the partial sentence generated so far, 877,-11 and the 7th token in the beam. The payolf rewards beams that contain candidate sequences with high arousal scores (to promote interestingness) and low perplexity (to promote coherence). It also penalizes large beam sizes to encourage using fewer com- pute resources. Mathematically, the payoff value R(b, l), for choosing a beam size, by, at time step f, is defined as:
The arousal score of the sentence generated so far, SIT,t-1- The arousal score of a partial sentence, viewed as a sequence of tokens, s, length n is:
1. Arousal score:
R(b,, () = max ( (4) vE[1,b]
A(s) = Σ a(w;)
(2)
where or B are coefficients for each component, Acc) and ppl(c) represent the arousal score (as de- fined in Eqn. 2) and the perplexity of the candidate sequence respectively, and [by] represents the size of beam by
where a(w,) is the arousal score of the 8th token obtained from the NRC Word-Emotion Associa- tion lexicon (Mohammad, 2018). Since longer sen- tences can accumulate higher arousal scores, we divide the arousal score by a length normalizing factor (Wu et al., 2016). The length normalizing factor for a sentence of length is:
Given the set of k choices for beam sizes (b), b2 b..... 1t,, the optimal beam size by at timestep is given by
bt argmax R(b t) 1.
(5)
(5 + n)
Ip(n)
(3)
(5 + 1)
Correspondingly. the optimal payoff at time step. is R(b;, ().
where X is the normalization coefficient. 2. Event trigger likelihood: Sims et al. (2019) points out that in narratives there are certain words in sentence that trigger interesting literary events. E.g. In the sentence " ... Stephen leaned his arms on ...". The word "leaned" an event trigger. Iden- tifying such event triggers can help in locating the interesting part of a sentence, which in turn will help in deciding whether to choose larger beam. With this motivation, we train a RoBERTa (Liu et al., 2019) based predictor that given a partial sen- tence predicts whether the next token would be the trigger for an interesting literary event. We provide the partial sentence generated so far, SIT. 1. as the input to this predictor and use the likelihood assigned by it (for the next token to be an event
Using the LinUCB (Upper Confidence Bound) algorithm (Li et al., 2010), we optimize the bandit model by minimizing regret L defined as:
= DEEL t)] E|"Y_, t)]
(6)
where T is the total number of time steps or the total number of tokens SyT.
A ffective Reranking. While Dynamic Beam Sizing introduces more arousing content, it does not consider the variation of emotions associated with the content. Chung al. (2022) highlighted that variation of emotional are (Reagan et al., 2016) can make a story more engaging. We, therefore, introduce Affective Reranking.
4

---

Let (SIT', -STT .I.") be the candidate sen- lences that are generated as potential intriguing twists in the beam. The best candidate should have a high arousal score and should also have high affective contrast. We quantify affective con- trast as the difference in the valence scores of the candidate sentence and the story generated so far. Valence score of a sequence of tokens, t, is the length-normalized cumulative valence score of its individual tokens. We use the NRC-VAD lexicon (Mohammad, 2018) to obtain valence scores of tokens.
Model
PPL 4
Uni f
RUB f
Aro t
GPT2
26.77
0.021
0.1546
0.45
AFFGEN-2
40.27
0.019
0,1556
0.51
GPT3
18.90
0.028
0.1541
0,46
AFFGEN-3
25.66
0.029
0.1547
0.53*
Table Automatic evaluation of AFFGEN using Per- plexity (PPL). UNION score (Uni) (Guan and Huang, 2020), (RUB) score (Tao et al., 2018), and Arousal score (Aro). and & indicate if higher or lower scores are de- sirable. Bold fonts indicate best scores and indicates statistical significance (p <, 0.01). The results indicate that both versions of AFFGEN can generate interesting stories without compromising coherence.
We select the best candidate for the intriguing twist sentence s such that:
a baseline to compare with a large language model. For GPT3, we use the following prompt 2 (after experimentation): "Continue writing an interesting story using the following context, <context>, The total length of the story should be five sentences. The total words limit is 60 words.""
s' = argmax A(siT') (u(srr') u(SITT-1)
(7)
5 Empirical Evaluation
In this section, we describe our experiments.
5.1 Experimental Setup
5.2 Automatic Evaluation
the ROC- Stories dataset (Mostafazadeh al., 2016), large collection of 100k five-sentence 'commonsense" stories about everyday events. We held out Ik sto- ries each for validation and testing and use the first sentence of every story as the prompt. We chose this dataset because it allows us to assess the per- formance of our model's ability to learn from collection of everyday life stories and improvise them be interesting. The short nature of these sto- ries also makes the manual assessment of narrative quality feasible during human evaluation which otherwise would have been difficult. This focus on short stories, however, does not limit the potential application of our model to longer narratives.
Dataset. For experiments,
Table presents a comparison between AFFGEN and baseline methods. We use two versions our model, AFFGEN-2 and AFFGEN-3. They use fine- tuned GPT-2 and GPT-3 as the base storytellers ($4.2). We observe that both versions of AFFGEN have higher perplexity (PPL) scores than the base- lines. This, however, is expected and does not imply low coherence because AFFGEN encourages using low-likelihood words during the decoding process to generate interesting content.
For a better evaluation of coherence, we consider the UNION (UNT) (Guan and Huang, 2020) and RUBER (RUB) scores (Tao et al., 2018). UNION is reference- score specially designed for evalu- ating open-ended story generation models. RUBER hybrid of referenced and unreferenced metric used for evaluating dialog systems. We only use its unreferenced part to evaluate the quality of text (story) generated in response to a query (the story prompt). A higher value for these scores is better. We observe that for these scores versions of AFFGEN either perform better than comparable to the baselines. This indicates that AFFGEN is capable of generating coherent narratives.
Our base storyteller is trained on the ROCStories dataset. The contextual bandit model is trained in unsupervised manner, relying on the internal regret function.
piece
Implementation Details. All hyperparameters were set based on the performance the validation set. We used a 0.00015, 3 = 0.0003 in Eqn. 4 and A 1.5 in Eqn. 3. We trained the bandit model on single A5000 for 10 epochs and it chose between three beam sizes of 10, 30 and 60.
For evaluating how interesting the stories are, we measure their per-token Arousal score (Aro) (Eqn. 2) which quantifies their affect level. A higher value is better for this score. We observe that
Baselines. Our primary baseline is GPT2 fine- tuned on the RocStories dataset since is widely recognized for its story generation capabilities (Brahman and Chaturvedi, 2020). We use GPT3 as
Pleass refer to Table 7 for more pormpt details.
5

---

Evaluation Criteria
Win
Lose
Tie
Evaluation Criteria
Win
Tie
Coherence
50.5*
40.7
Coherence
14.5
38.8
46.7
Emotional Engagement
53.0*
40.3
6.7
Emotional Engagement
55.5*
19.7
Empathy
53.8
40.2
Empathy
40.8*
28.6
30.6
Interestingness
54.9"
41.1
4.0
Interestingness
45.3*
26.3
28.4
Overall Preference
52.7
39.6
7.7
Overall Preference
45.3
35.7
19.0
Table 2:
Human evaluation of AFFGEN GPT-3. AFFGEN generates better stories across all measures. indicates statistical significance (p<0.1 I coherence and p<0.05 for others).
Table
Human evaluation of AFFGEN ChatGPT.
AFFGEN generates not as coherent but more interesting and empathetic stories. * indicates statistical signifi- cance (p<0.05).
both versions of AFFGEN outperform the baselines with A achieving the highest score. This indicates that AFFGEN generates more interesting stories.
in tie. This indicates that AFFGEN does not com- promise coherence while generating stories.
Emotional Engagement evaluated how effectively story conveys a range and intensity of emotions that capture and hold the reader's attention and create sense of emotional depth and complexity. For this criterion, judges found stories generated by AFFGEN-3 be more emotionally engaging than GPT-3 53.0% and less emotionally engaging 40.3% of the cases. This demonstrates AFFGEN's stronger ability to evoke emotions in readers.
5.3 Human Evaluation
order to assess the performance of AFFGEN, comprehensive human evaluation was conducted on the Amazon Mechanical Turk (AMT) platform. A total of 100 instances were randomly selected from our test set. We feed their initial sentences prompts for generating stories using AFFGEN-3 and GPT-3, our stronger baseline. To eliminate any potential bias, the presentation order of the two stories was randomized. The Turkers then selected the better of the stories according to criteria: coherence, emotional engagement, empa- thy, interestingness, and overall preference. These criteria were chosen based on prior research con- ducted by Chhun et al. (2022). The Turkers could also select an "equally good" option. The Turk- ers were explicitly instructed to solely consider the given criterion when evaluating, except when ex- pressing an overall preference. In the appendix. Figure 4 showcases a screenshot of our AMT setup. We specifically utilized Master annotators predom- inantly from English-speaking countries (US, UK, Canada, and Australia). We evaluated 200 stories in total, and each pair was assessed by three dif- ferent annotators. We discuss the results shown in Table 2 below. All differences in this table statistically significant (p<0.1 for coherence and p<0.05 for others) and the inter-annotator agree- ment 0.58 (moderate agreement).
Empathy evaluated the story arouses the read- ers' empathy for the characters. The conflicts and challenges described in stories can create situations that make the readers project their own emotions and thoughts onto the characters, keeping them invested and engaged. For this criterion, AFFGEN- outperformed GPT-3 by large gap of 13.6% (53.8% wins and 10.2% losses). This demonstrates that AFEGEN can generate emotionally resonant content.
Interestingness evaluates the story's ability to be compelling and engaging. For this criterion also, AFFGEN-3 outperformed GPT-3 by a large gap of 13.8% (54.9% wins and 41.1% losses). This demonstrates AFFGEN's its superiority in keeping the reader's ' interest while generating stories.
Overall Preference Finally, we observed that over- all, the judges preferred AFFGEN over the baseline in 52.77 of the cases (as compared to preferring baseline over AFFGEN in 39.6% cases).
Coherence evaluated the logical flow and connec- tion between the different clements of the story. For this criterion, judges found stories generated by AFTGEN-3 be more coherent than those generated by GPT-3 in 50.5% of instances, while AhGGEN33's stories were considered less coherent in 40.7% of cases. The remaining 8.8% resulted
To conclude, the human evaluation results pro- vide strong evidence of the superiority of the AFFGEN in various critical aspects of open-ended story generation underlying its ability generate interesting and engaging stories while maintaining coherence.
6

---

UNION
RUBER
Arousal
produces less arousing stories than AFFGEN but AENG. and A produce more arous- ing stories than AFFGEN. This aligns with our initial intuition that a larger beam size helps the model generate more interesting content. However, because of large but static beam sizes, the stories generated by these two versions were less coherent than those generated by AFFGEN.
A
0.007
0.012
0.018
3FFGEN
0.002
0.007
0.024
AFTGENOO
0..055
0.006
0.047
AFFGEN AR
0.002
0.001
-0,070
Table 4: Performance of ablated versions of AFFGEN with static beam sizes relative to AFFGEN. Subseripts indicate the beam sizes. A negative score indicates that the ablated version did not perform as well as AFFGEN, These results indicate that it is important to explore large beam sizes in dynamic manner to generate interesting and coherent stories.
Next, also consider another version of AFFGEN but without Affective Reranking. The rel- ative performance of this model shown the last row of Table 4. We can see that the performance of this version is quite close to the baseline. Also, while its coherence is comparable to AFFGEN, the arousal score is particularly worse indicating the importance of this component in generating inter- esting content.
5.4 Comparison with ChatGPT
In this section we compare AFFGEN with a large language model, ChatGPT 3. We used a human evaluation setup similar to that described in $5.3. These annotations were performed by expert an- notators who were students of literature theories. For generating stories with ChatGPT, we experi- mented with different prompts and the final prompt is shown in Table 7. Table 3 shows the results. An- notators expectedly found ChatGPT's stories to be more coherent. Our initial analysis also revealed ChatGPT text to have more sophisticated structure. However, annotators found AFFGEN'S stories to and inter-
Overall, we can draw two conclusions from this ablation study. First, exploring large beam sizes and affective reranking can help in generating more interesting content. Second, is important to dy- namically switch between larger and smaller beams to balance interestingness and coherence.
5.6 Expansion to Longer Narratives
Our experiments have used RocStories which are short in nature. This focus short stories does not limit the potential application of our model to longer narratives. Jolles (2017) points out that sto- ries could be condensed into "simple forms". Story composition could be viewed as process of ex- panding these simple forms into presentable longer narratives. Table 10 presents expanded AFFGEN- generated stories and compared with vanilla Chat- GPT generated stories. With the help of the five- sentence interesting plots produced by AFFGEN, ChatGPT expand them into better stories compar- ing to vanilla ChatGPT generated stories.
be significantly more emphathy-evokin esting. Because of this, the annotators preferred AFFGEN over ChatGPT in the overall preference.
5.5 Ablation Study
We now describe our ablation study in which we investigate the importance exploring different beam sizes and affective reranking. In our experi- ments reported so far, we made AFFGEN explore three different beam sizes during decoding, In this study, design ablated versions of AFFGEN that only uses one of the three beam sizes. We call them AFFGEN10. AFFGEN30, and AFFGENco, where the subscript indicates the beam size being used. The first three rows of Table reports the relative performance of these versions with AFFGEN. All models use fine-tuned GPT-2 as the base storyteller. For all scores, negative score indicates that the ablated version did not perform well AFFGEN (and vise versa). We see that for most of the ablated versions, the UNION and RUBER scores are negative. This means that the stories generated by the ablated versions are less coherent than the full model. In terms of Arousal scores, 1FFGEN10
5.7 Dynamic Beam Sizing
We now investigate how the beam size changes as AFFGEN generates an interesting sentence. Figure shows the average beam size used to generate at different positions of a typical sentence. We observe that AFFGEN is using larger beam sizes for the first few tokens. Our manual analysis revealed that the interesting words indeed appear earlier in sentence, in general.
Since the model is capable of transitioning be- tween beam sizes, we plot heat map of the tran- sitions shown in Figure 3. Each cell shows the probability of transitioning from beam size on Y
'OpenAl. (2023). ChatGPT (May 24th version) [Large language model]. https:/chattopenai.com
7

---

Table 8 in the Appendix provides more compar- ative examples of stories generated by the base- line and AFFGEN and corresponding explanations. Analyzing explanations for story pairs we found that the judges preferred AFFGEN'S stories because they presented shift in mood enhancing the af- feet they had on the reader. AFFGEN's stories also presented unexpected twists which provide relief from the story's prevalent theme and increases its interest. In contract, the baseline stories were banal and conflict-less. Sometimes AFFGEN'S stories in- troduced a melancholic theme, but the judges still found them pleasant. This aligns with the narrato- logical theory presented by Massumi (2002) who argues that there is gap between the content and the effect on the receiver's s end. As result, audi- often rate "sad" scenes in the film the "most pleasant". Overall, AFFGEN was found to be bet- ter at generating more emotionally captivating and interesting stories leading to to better storytelling experience.
Average Beam Size
15
1
4
9
10
11
Posítion in Sentence
Figure 2: Average beam size used to generate differ- ent positions of a sentence.
1.0
0.72
0.16
0.12
Beam Size
30
0.4
10
0.2
0.0
Beam Size
5.9 Error Analysis
Figure 3: Transitional probability between beam sizes.
Using the judges' explanations provided during the human evaluation, we also conduct error analy- sis identify issues encountered during story gen- eration by AFFGEN. Table 9 in Appendix shows some examples of story pairs in which the judges did not prefer AFTGEN's stories over GPT-3's sto- ries and their explanations. We observe that while A AFFGEN introduces an intriguing twist in the story, it sometimes suffers from typical language mod- eling challenges like repetitive phrases and ideas (Story 2) and incoherence. Often the incoherence is caused by lack of commonsense knowledge like sunglasses cannot change eye colors (Story 1), and if toy breaks, cannot function (Story 3). This aligns with the proposition made by Alhussain and Azmi (2021) that coherence (and also causality) are fundamental in storytelling, Without them, the story may disintegrate into inconsistent fragments.
axis to beam size the X axis. Darker color indicates higher probabilities. We observe that in general, while AFF-GEn has a tendency stick to a chosen beam size (~ 70%), does transition to different beam sizes about 30% of the times indi- cating the importance of switching between beam sizes.
5.8 Qualitative Analysis
During the human evaluation ($5.3), when asking for preferences for the stories, we also asked the judges to provide explanations for their choices. We then analyzed the explanations to further an- alyze the stories generated by AFFGEN. Table 5 show an example of stories generated by GPT-3 and AFFGEN for the same input prompt as well as the human-provides explanation. While both stories have a happy ending, AFFGEN's story intro- duces plot complication, where the protagonist, Grayson's ss initial attempt to bake cake fails. He resolves the situation through determined efforts, creating narrative of perseverance. Compared to the baseline, the plot in AFFGEN'S story becomes more complicated and has more ups and downs, which enhances the emotional engagement and in- terest of the reader. The AMT judges noted that the AFFGEN'S story was more emotionally expressive.
Conclusion
This paper addresses the task of generating inter- esting stories. For this, we present, AFFGEN, a language model that uses novel contextual bandit- based decoding mechanism. This new decoding mechanism enables the AFFGEN to dynamically explore different beam sizes and rerank based on affective quality of the text. Our experiments in- dicate that AFEGEN can generate interesting but
8

---

Grayson wanted bake his brother birthday cake, he got out mom' recipe book and started searching for the perfect one. decided to go out and make four layer chocolate cake with cream cheese frosting. His brother was impressed that he wolfed down pieces row.
GPT-3
Grayson wanted to bake his brother birthday cake. The first attempt furned out terribly bad and ruined. Grayson was determined make the perfect cake his brother's hirthday, Undeterred, Grayson
AFFGEN
started again from scratch and was finally able make delicious cake that his brother loved.
Human Explanation
The story by AFFGEN is more emotionally expressive
Table 5: Sample stories generated by GPT-3 and AFFGEN. As indicated by the judge, AFFGEN's story is more emotionally engaging.
coherent narratives. Our ablation studies under- score the importance of dynamic beam sizing and affective reranking, and our qualitative and error analysis point to the strengths and weaknesses of our model. We hope that this ability to compose interesting narratives can open new dimensions of computational creativity, driving the generation of unique and captivating content at an unprecedented scale and speed.
model from exploring all possible values. Our experiments used GPT-2 and GPT-3 as the base storyteller for generating the stories. However, we see AFFGEN as a framework that could incor- porate other language models and future work can investigate this aspect.
Our experiments explored short and fictional nar- ratives. Future work could investigate advanced planning and strategies for composing longer sto- ries or non-fictional content.
Acknowledgement
Our dataset and experiments use only one lan- guage English. We did not investigate the model's capabilities to generate stories in other languages.
We appreciate the reviewers for their insightful comments and suggestions. Tenghao Huang and Muhao Chen were supported by the NSF Grant IIS 21 2105329,, Amazon Research Award and Ke- ston Exploratory Research Award. Ehsan Qasemi was supported by the DARPA MCS program un- der Contract No.N66001 1924033 with the United States Office Of Naval Research. Computing of this work was partly supported by a subaward of NSF Cloudbank 1925001 through UCSD.
Ethical considerations
Our experiments use publicly available dataset. Previous work (Huang et al., 2021) has shown that contains gender-related biases and storytelling models that use this dataset can replicate and am- plify these biases. Our model also encourages low- perplexity text, which could unintentionally encour- age biased, violent, sexually explicit content. Since we have not employed any bias toxicity removal methods, applications of our work should control for inappropriate content.
Limitations
Our study has the following limitations.
We assume a single sentence containing an in- triguing twist can enhance a story's interesting- ness. This paper focused on how to generate that intriguing twist. However, story can potentially benefit from multiple interesting sentences and fu- ture works can investigate into how frequently and where to generate interesting content.
References
Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng, and Mohit lyyer. 2020. STO- RIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation. Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6470-6484, Online. Association for Computational Linguisties.
We adopted a simple data-driven approach for deciding where to put the sentence that contains the intriguing twist. It samples from a distribution learned from a collection of stories. Future work could work on more sophisticated methods that consider the preceding narrative context for decid- ing when describe an interesting twist that it integrates better with the story being generated. For practical purposes, the bandit model dis- cretized beam sizes. However, beam size is con- tinuous variable, and discretizing it can restrict the
Arwa I. Alhussain and Aqil M. Azmi. 2021. Automatic story generation: A survey of approaches. ACM Comput. Surv., 54(5).
Margaret M Bradley and Peter J Lang. 1999. Affective norms for english words (anew): Instruction manual and affective ratings. Technical report. Technical re- port C-1, the center for research in psychophysiology.

---

Faeze Brahman and Snigdha Chaturvedi. 2020. Mod- eling protagonist emotions for emotion aware story- telling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5277-5294, Online. Association for Computational Linguistics.
Gerard Genette. 1980. Narrative discourse. In Narra- tive Discourse. Corelll University Press.
Jian Guan and Minlie Huang. 2020. UNION: An Un- referenced Metric for Evaluating Open-ended Story Generation. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Process- ing (EMNLP), pages 9157-9166, Online. Association for Computational Linguistics.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners, Advances in Neural Information Processing Systems, 33:1877-1901.
Rujun Han. Hong Chen, Yufei Tian, and Nanyun Peng, 2022. Go back in time: Generating flashbacks in stories with event temporal prompts. In Proceedings of the 2022 Conference of the North American Chap- of the Association for Computational Linguistics: Human Language Technologies, pages 1450-1470, Seattle, United States. Association for Computational Linguistics.
Cyril Chhun, Pierre Colombo. Fabian M. Suchanek, and Chloé Clavel. 2022. Of human criteria and au- tomatic metrics: A benchmark of the evaluation of story generation. In Proceedings of the 29th Inter- national Conference on Computational Linguisties, pages 5794-5836, Gycongju, Republic of Korea. In- ternational Committee on Computational Linguistics.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text de- generation. In International Conference on Learning Representations.
John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang. 2022. Talebrush: Sketching stories with generative pretrained language models. In Proceedings of the 2022 CHI Conference on Human Factors in Comput- ing Systems, CHI '22, New York, NY, USA. Associa- tion for Computing Machinery.
Tenghao Huang, Faeze Brahman, Vered Shwartz, and Snigdha Chaturvedi. 2021. Uncovering implicit gen- der bias in narratives through commonsense infer- ence. In Findings of the Association for Computa- tional Linguistics: EMNLP 2021, pages 3866-3873 Punta Cana, Dominican Republic. Association for Computational Linguistics.
Elizabeth Clark and Noah A Smith. 2021. Choose your own adventure: Paired suggestions in collaborative writing for evaluating story generation models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages
Parag Jain, Priyanka Agrawal, Abhijit Mishra, Mo- hak Sukhwani, Anirban Laha, and Karthik Sankara- narayanan. 2017. Story generation from sequence of independent short descriptions. arXiv preprint h0tv:77000050.1
Pablo Delatorre. Barbara Arfe, Pablo Gervás, and Manuel Palomo Duarte. 2016. A component-based architecture for suspense modelling.
André Jolles. 2017. Simple forms: Legend, saga, myth, riddle, saying, case, memorabile, fairytale, joke. Verso Books
Angela Fan, Mike Lewis, and Yann Dauphin. 2018 Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 889-899 -988, Melbourne, Australia, Association for Computational Linguistics.
Anna Kasunic and Geoff Kaufman. 2018. Learning to listen: Critically considering the role of Al in human storytelling and character creation. In Proceedings of the First Workshop on Storytelling, pages 1-13, New Orleans, Louisiana. Association for Computational Linguistics.
Gustay Freytag. 1908. Freytag's 's technique of the drama: an exposition of dramatic composition and art. Scott, Foresman and Company.
John Langford and Tong Zhang. 2007. The epoch- greedy algorithm for contextual Advances in neural information processing systems, 20(1):96-1
bandits.
Shira Gabriel and Ariana F Young. 2011. Becom- ing a vampire without being bitten: The narrative collective-assimilttion hypothesis. hypothessss Psychological sci- ence, 22(8):990-99)
Lihong Li, Wei Chu, John Langford, and Robert E Schapire. 2010. A contextual- approach to per- sonalized news article recommendation. Proceed- ings of the 19th international conference on World wide web, pages 661-670
Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical detection and visual- ization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111-116. Florence, Italy. Association for Computational Lin- guistics.
Zhiyu Lin and Mark Riedl. 2021. Plug -nd-blend: A framework for controllable story generation with blended control codes. In Proceedings of the Third Workshop on Narrative Understanding, pages 62-71, Virtual. Association for Computational Linguistics.
10

---

Danyang Liu, Juntao Li, Mlng-HHsun Yu, Ziming Huang, Gongshen Liu, Dongyan Zhao, and Rui Yan. 2020. A character-centric neural model for auto- mated story generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1725-1732.
Rafael PÊrez Y PÉrez and Mike Sharples. 2001. Mex- ica: A computer model of a cognitive account of creative writing. Journal of Experimental & Theoret- ical Artificial Intelligence, 13(2):119-139
Julie Porteous and Mare Cavazza. 2009, Controlling
narrative generation with planning trajectories: the role of constraints. In Interactive Storytelling: Sec- ond Joint International Conference on Interactive Digital Storytelling, ICIDS 2009, Guimarãex, Por- rugal, December 9-11, 2009. Proceedings 2, pages 234 245. Springer.
Yinhan Liu, Myle Ott. Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 100:790050 7907. 1 16692.
Fuli Luo, Damai Dai, Pengcheng Yang. Tianyu Liu, Baobao Chang, Zhifang Sui, and Xu Sun. 2019. Learning to control the fine-grained sentiment for story ending generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6020-6026, Florence, Italy, Asso- ciation for Computational Linguistics.
Ratish Puduppully, Li Dong, and Mirella Lapata. 2019. Data-to-text generation with content selection and planning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 6908-696 1915.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Brian Massumi. 2002. Parables for the Viriual: Move- ment, Affect, Sensation. Duke University Press, Durham, NC.
Andrew Reagan, Lewis Mitchell, Dilan Kiley, Christo- pher M Danforth, and Peter Sheridan Dodds. 2016. The emotional arcs of stories are dominated by six basic shapes. EPJ Data Science, 5(1):1-12.
Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. 2023. Locally typical sampling. Transac- tions of the Association for Computational Linguis- tics, 10:11111
Mark O Riedl and Robert Michael Young. 2010. Narra- tive planning: Balancing plot and character. Journal of Artificial Intelligence Research, 39:217-268.
Saif Mohammad. 2018 Obtaining reliable human rat- ings of valence, arousal, and dominance for 20,000 English words. In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers). pages 174-184, Melbourne, Australia. Association for Computational Linguistics.
Melissa Roemmele. 2021. Inspiration through observa- tion: Demonstrating the influence of automatically generated text on creative writing.
Gregory Schraw, Terri Flowerday, and Stephen Lehman. 2001. Increasing situational interest in the classroom. Educational Psychology Review, 13:21 1-224...
Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computationnl Linguistics: Human Language Technologies, pages 839-849.
Matthew Sims, Jong Ho Park, and David Bamman. 2019. Literary event detection. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3623-3634, Florence, Italy. Asso- ciation for Computational Linguistics.
Ed S Tan and Barbara Trans Fasting, 1996. Emotion and the structure of narrative film: Film as an emotion machine.
Jessica Ouyang and Kathleen McKeown. 2015. Mod- eling reportable events as turning points in narrative. In Proceedings of the 2015 Conference on Empiri- cal Methods in Natural Language Processing, pages
Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. 2018. Ruber: An unsupervised method for au- tomatic evaluation of open-domain dialog systems. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
Nanyun Peng, Marjan Ghazvininejad, Jonathan May, and Kevin Knight. 2018. Towards controllable story generation. In Proceedings of the First Workshop on Storytelling, pages 43-49, New Orleans, Louisiana. Association for Computational Linguistics.
William R Thompson. 1933. On the likelihood that one unknown probability exceeds another view of the evidence of two samples. Biometrika, 25(3-4):285- 294.
Xiangyu Peng, Siyan Li, Sarah Wiegreffe, and Mark Riedl. 2022. Inferring the reader: Guiding auto- mated story generation with commonsense reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 7008-7029, Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
David Thue, Vadim Bulitko, Marcia Spetch, and Eric Wasylishen. 2007. Interactive storytelling: A player modelling approach. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, volume 3, pages 43-48.

---

Prashanth Vijayarahhvvan and Deb Roy. 2023. M-sense: Modeling narrative structure in short personal nar- ratives using protagonist's s mental representations. arXiv preprint arXiv:2302.09418
pages 3346-3361, Seattle, United States. Association for Computational Linguistics.
7henjie Zhao, Yufang Hou, Dakuo Wang. Mo Yu, Chengzhong Liu, and Xiaojuan Ma. 2022. Educa- tional question generation of children storybooks via question type distribution learning and event-centric summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers). pages 5073-5085, Dublin, Ireland. Association for Computational Lin- guistics.
Anvesh Rao Vijjii, Faeze Brahman, and Snigdha Chaturvedi. 2022. Towards inter-character relationship-driven story generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pages 8970-899 8987, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
PeiFeng Wang, Jonathan Zamora, Junfeng Liu, Fangyu Liu, Muhao Chen, and Xiang Ren. 2021. Contextual- ized scene imagination generative commonsense reasoning. In International Conference on Learning Representations.
Zhilin Wang. Anna Jafarpour, and Maarten Sap. 2022. Uncovering surprising event boundaries in narratives. In Proceedings the 4th Workshop of Narrative Un- derstanding (WNU2022), pages 1-12, Seattle, United States. Association for Computational Linguistics.
David Wilmot and Frank Keller. 2020. Modelling sus- pense in short stories uncertainty reduction neural representation. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 1763-1788, Online, Association for Computational Linguistics.
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le. Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google's 's neural machine translation system: Bridging the gap between human and machine trans- lation.
Lili Yao, Nanyun Peng. Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan- and-write towards better automatic storytelling. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence.
Fangzhou Zhai, Vera Demberg, Pavel Shkadzko, Wei Shi, and Asad Sayeed. 2019. A hybrid model for globally coherent story generation. In Proceedings of the Second Workshop on Storytelling, pages 34- 45, Florence, Italy. Association for Computational Linguistics.
Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan. 2021. Trading off diversity and quality in natural language generation. In Proceed- ings the Workshop Human Evaluation of NLP Systems (HumEval), pages 25-33, Online. Associa- tion for Computational Linguistics.
Zhexin Zhang, Jiaxin Wen, Jian Guan, and Minlie Huang. 2022. Persona- guided planning for control- ling the protagonist's s persona in story generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies,
12

---

A HIT Example
Speaker User
Content
Here is an interesting story. "Melissa invited Mark the park a pienic. The weather turned bad when thunder hit. Mark and Melissa decided to wait out the storm, and when rain cleared, they had the most romantic pienic they's 'd ever experienced. watching the sun set over the park". Write an interesting story based on the context "Tom going to fish on Saturday". . The story should be five-sentence long and 60 words maxi- mum.
Figure illustrates a screenshot of the form pre- sented to the annotators on the AMT.
ChatGPT
Tom eagerly awaited Saturday, planning a day of fishing his beloved lakeside spot. As dawn broke, he cast his line, lost in the tranquility. Suddenly. dark clouds loomed, and rain poured down. Seeking refuge under tree, he met Sarah, another angler. The rain ceased, but Tom left with newfound friend and heartwarming memory.
Figure 4: Screenshot of the AMT survey used for human evaluations $5.3
Table 6: Attempt to optimize prompts through demon- stration
B Prompts for LLM
Task GPT-3 infill- ing
Prompts for LLM
"Continue wriling a story using fol- lowing context, <context>. length the story should he <n> sen- tences. The total words limit <2*12> words."
In exploration, investigated multiple prompt optimization techniques. Initially, we utilized demonstrations as potential avenue. Yet, as il- lustrated Table Tab. 6, it became evident that the resultant narrative exhibited a significant degree of repetitiveness with the given demonstration. An at- tempt to introduce diversity by leveraging varied ensemble of demonstrations did not yield improve- ments in reducing narrative redundancy. Subse- quently, explored the incorporation of specific keywords within prompts to guide the generative process. For instance, by prompting LLM "Write five-sentence interesting story with mystery", we observed that while such specificity could yield captivating narratives, they bore a recurring the- matic redundancy. To exemplify, from a sample of ten stories generated using the menystery-enhancee prompt, it was observed that seven out of ten gener- ated stories converged around the theme of hidden secret being revealed towards the conclusion. Rep- resentative narratives included scenarios such as photographer unveiling a mystery within a haunted lighthouse, a young woman unearthing a concealed garden, and an individual discerning a hidden cache of treasure, etc. In consideration of these findings, and without loss of generality, opted for the current prompt for subsequent generations.
total
GPT-3
"Continue writing an interesting story using the following context, <context>, The total length of the story should he <n> sentences. The total words limit is 60 words."
ChatGPT
"Write five-sentence interesting short story using the following sentence the first sentence: <context>."
Table 7: Prompts for Large Language Models
tive analysis in $5.8.
D Error Analysis
Table 9 shows some examples of story pairs which the judges did not prefer AFFGEN's stories over GPT-3's stories and their explanations. We present detailed qualitative analysis in $5.9.
Stories with Annotator Preferences
Table shows more sample stories generated by GPT-3 and AFFGEN. We present detailed qualita-
13

---

ID
Baseline
AFFGEN story
Explanation
Melissa invited Mark to the park for picnic. The weather turned bad when thunder hit. Mark and Melissa decided to wait out the storm, and when the rain cleared, they had the most romantic picnic they'd ' ever ex- perieneed, watching the sun over the park.
Melissa invited Mark the park pienie, They had great time, play- ing games eating the delicious food she prepared. They laughed, talked and made plans for their future. was a perfect day that they would
<AFFGEN's>> story is more romantic.
remember forever.
Ella her grandma letter, filled with stories about her first day schonl. told her about the friends she made, the lunch she had,
Ella wrote her grandma letter. Ella's grandma wrote back with Her favorite aunt died unexpectedly of cancer! Ella sad- dened to hear the news, but comforted in knowing that aunt had lived full and happy life.
Emotional condolences gives <AFFGEN's: "s> story an interesting and sad theme
pected
and she had learning. Grandma was proud her, and Ella couldn't get a letter back.
Jen big test tomorrow. Jen stayed up all night studying. but still felt un- prepared. Her roommate surprised excited with good news! She had found out that the professor had de- cided to give everyone an extra day to study for the test.
Jen has big test tomorrow and shc's s determined to it. She spends hours studying, memorizing facts and theo- ries, and rehearsing for the exam. She knows she's ready and confident that she'll get the grade she desires.
The <Baselinc> story has no bumps, it's banal
Table 8: More sample stories generated by GPT-3 and AFFGEN. As indicated by the judge, AFFGEN's story is more preferred by annotators in various aspects.
Baseline
AFFGEN story
Explanation
Hazel's 's eyes were green, but she really wanted blue cyes. The school offered special free sunglasses! Hazel was al- ways cnvious of her friends' blue eyes. With the school's 's offer of free sun- glasses, she was determined to make her dream come true. She put on the sunglasses and instantly felt like she had the most beautiful blue eyes in the world.
Hazel wished for blue eyes every night, but her green guze never changed. She felt like a different person with blue eyes; she was sure they'd 'd bring her the confidence she eraved. One day, she found my mystroouss pair of blue con- tacts. She them and was like a whole new world opened for her.
The <Baseline> story is interesting
We went out last night. My girlfriend excited to meet new friends! We had great time, and my girlfriend made few friends. We stayed out late, and although her having good time. We both agreed was great night and can't 't wait
We went out last night, dancing and laughing the night away. had the time of my life, and felt like nothing could stop us. never wanted the night to end, and I will never forget it.
The <AFFGEN > story has too much repetitions and basically dull, the first one sounds very au- thentic
tired, was
happy
again.
Joan got mad when Janet stole his gum. Not wanting be outdone. Joan cided to steal something back. Her favorite toy car broke loose. Joan snatched Janet's favorite stuffed ani- mal and ran away with it. Janet chased after Joan, but she was too late, as the toy car had already reached its destina- tion.
Joan chased Janet around play- ground, yelling for her to give back his gum. Janet laughed and kept running, eventually out of sight. Joan stomped back his friends, still mad but with a plan. He would get his gum back, no matter what.
<AFFGEN story does make sense.
Table 9: Error analysis
14

---

ID]
Vanilla ChatGPT
Expanded AFF GEN story
Melissa invited Mark to the park for pienic. It sunny Saturday, perfect for their long-awaited rendezvous. Melissa had spent hours preparing sandwiches, fruit sal- ads, and homemade lemonade. As they spread out the checkered blanket, Mark noticed a curious twinkle in Melissa's eye.
Melissa had planned the perfect pienic for Mark at their favorite park. The day started off sunny, but as they laid out their blanket and unpacked their delicious spread, omi- nous clouds gathered overhead. Suddenly, a deafening clap of thunder echoed through the park, sending them scrambling for cover.
"What's s the surprise?" he asked.
Huddled under nearby tree, Mark and Melissa exchanged nervous laughter. As the rain poured down around them, they found themselves sharing stories, dreams, and even few stolen kisses. Time seemed stand still as the storm raged on.
With a mischievous grin, she pulled out a vintage camera. "I thought we could capture memories today." " They laughed, took goofy selfies, and snapped candid shots of each other. The sun cast warm glow, creating a picturesque backdrop.
1
When the last raindrop fell and the sun emerged from behind the clouds, Mark and Melissa emerged from their shelter, hand in hand. The park glistened with fresh rain, and the sky painted itself in shades of pink and orange as the sun began its descent. They sat down on their damp blanket, feeling closer than ever, and savored their makeshift romantic picnic as they watched the sun set over the park. It turned out to be an unforgettable day, filled with unexpected magic.
Later, Melissa reached into her bag, revealing a small book. was a scrapbook, filled with their past adven- tures together. Tears through the pages, reminiscing about their journey as friends tumned soulmates.
in Mark's 's eyes as he flipped
As the sun set, they stargazed, cherishing the memories they had created that day and adding another chapter to their beautiful story.
Ella wrote her grandma a letter, her small hand gripping
Ella, with tears in her eyes, carefully penned a heartfelt let- ter to her heloved grandma, sharing her recent adventures and school triumphs. Weeks passed, and to her surprise, a letter from her grandma arrived, the envelope trembling in her hands. Ella eagerly unfolded the letter, only to be met with heart- wrenching news: her favorite aunt had suc- cumbed to an unexpected battle with cancer.
the pencil tightly. She missed her visits to Grandma's 's cozy cottage in the woods, where they would bake cookies
and read stories by the crackling fireplace. In her letter, Ella poured her beart out. She shared her adventures, like the time she found hhinbow-colored caterpillar or built secret fort in the backyard. She also drew pictures of her favorite memories. hoping Grandma could see them through the words. Days turned into weeks, and Ella anxiously checked the mailbox. Then, one sunny moming. a letter arrived. Excitement danced in her eyes she tore open. Inside was heartfelt response, filled with love and wisdom, reminding Ella of the magical moments they had shared. With Grandma's 's letter in her hand, Ella felt like she holding piece of their cherished cottage. She knew that distance could break the bond they shared, and that their love would continue to grow through letters. just like the flowers in Grandma's garden.
Tears streamed down Ella's 's cheeks as she absorbed the painful words. Memories of her aunt's 's infectious laughter and warm hugs flooded her mind. Despite the sorrow, she found solace in her grandma's 's words. Her aunt had lived a life filled with boundless love, adventures, and cherished moments. She had been a beacon of positivity, spreading happiness wherever she went.
2
Ella vowed to honor her aunt's legacy by embracing life with the same zest and determination. Though her heart ached, she knew her aunt's 's spirit would continue inspire and guide her through the ups and downs of life.
Jen had big test lomorrow, and her nerves were getting the best her. She had spent countless hours studying. poring over textbooks. and reviewing notes, but anxiety still gnawed at her. The weight of the impending test loomed over her like dark cloud.
Jen had a big test tomorrow, and anxiety gnawed at her as she burned the midnight oil, pouring over textbooks and notes. Hours passed, her eyelids heavy, but she still felt unprepared. Doubt clung her like stubhorn shadow. Just as the clock struck midnight, her roommate, Sarah, burst into the room with a grin that could light up sta- dium. Breathless and excited, she shared the unexpected news: the professor had decided to give everyone an extra day to study for the test.
She tried to calm herself with deep breaths and a reassur- ing pep talk. "You'v ve prepared as best as you can, Jen," she whispered to herself. "You know the material. You've 've got this."
3 Jen's friends, knowing the stress she was under, decided to surprise her. They showed up at her doorstep with a homemade dinner, colorful sticky notes of encouragement, and their unwavering support. Together, they created a cozy study space filled with laughter and motivation.
Jen's 's heart swelled with relief and gratitude. It like a lifeline thrown to drowning swimmer. Tears of joy welled up in her eyes she hugged Sarah tightly. The universe had given her a second chance, and she wasn't 't going to waste She spent the next day revising her notes and feeling more confident with every passing hour. When she finally walked into the exam hall, she was armed not only with knowledge but also with the indomitahle spirit of resilience that had seen her through the long night.
As the night wore on, Jen's anxicty began to subside. With her friends by her side, she folt a renewed sense of confidence and determination. She knew that no matter what the test held, she had the support of some incredible friends who believed in her.
Table 10: Comparison betwen vanilla ChatGPT stories and expanded AFFGEN's stories We used ChatGPT for story expansion. we instructed the model with the following prompt "Expand the story to 150 words. The story should be interesting. MAAr-GGN-geneaan tend to be straightforward and lack of dramatic flair, the narratives produced by the expanded AFFGEN approach exhibit dramatic undertones, contributing to more captivating reading experience.
plots>.". While narratives generated by the vanilla CharGPT model
15


Reviews:
[{"review_id": "63ZAQIr4gz", "reviewer": "Reviewer_ME8S", "summary": "", "questions": "In Table 1, which language model is used for calculating PPL (for evaluating fluency)?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nThank you for your thoughtful review and constructive suggestions. We appreciate the time you have invested in evaluating our work. We address the concerns you raised below:\n\n**1) Regarding reviewer's comment on over-optimizing our approach toward 5-sentence short stories and its possible lack of generalizability to longer stories**, there are a couple of things we want to clarify:\n\nStory generation is a challenging task. One of the challenges is maintaining a  balance between interestingness and coherence. While coherence is not a big issue anymore for modern models for shorter stories, it is still a major challenge for longer stories. Therefore, we decided to work with short stories to focus our attention on interestingness which is a currently difficult problem.\n\nWe argue that generating short, interesting stories can be a vital starting point in the broader quest to produce longer engaging texts. Since the community is still in the early stages of understanding what constitutes an interesting narrative,  this work can contribute valuable insights and methodologies for this purpose.However, this focus on short stories does not limit the potential application of our model to longer narratives.\n\nInspired by literature criticism research, we also argue that longer stories could be created by expanding on the short stories that our model can generate. Here we would like to report additional experiment results: By prompting ChatGPT “Write a story based on the context we provide '', we expanded AFFGEN generated stories to long stories and compared with vanilla ChatGPT generated stories. The human evaluation shows that expanded-AFFGEN long stories have **40% win, 15% lose, 35%** tie rate compared with ChatGPT generated stories. We will include expert annotation for this experiment in the camera-ready version. By focusing on generating engaging short stories, this work lays the foundation for future research generating more complex, long-form content.\n\n**2) Regarding reviewer's comment on under-optimizing LLM’s prompt**, We want to clarify that \n\n**i)** AFFGEN is generalizable to different prompts: We recognize that diverse prompts to the story-infiller can have different effects on the generated content. We want to clarify that the strength of AFFGEN is providing intriguing twists regardless of base stories as the example shown below: \n\n`ChatGPT`: Tom is going to fish on Saturday. Beneath a rock, he finds a peculiar map. It leads to a forgotten cave. Inside, ancient markings puzzle him. A glint reveals a hidden compartment. A single question: \"What lies beyond?\" Determined, he started his adventure to unveil the truth.\n\n`AFFGEN`: Tom is going to fish on Saturday. He arrives at his favorite spot, waits, and casts line. <Two people start yelling loudly, threatening with violence.>. Perplexed, Tom notices their eyes flicker towards the trees. Hidden figures manipulating a bizarre scene? He decides to uncover the enigmatic pranksters.\n\nNote that the sentence wrapped in angle brackets is the intriguing twist generated by AFFGEN. We want to stress that ChatGPT's stories are indeed more coherent and it uses fancier words which has certainly impressed all users. However, there is more to storytelling than using fancy words. As the example illustrates, while ChatGPT gives a common treasure hunting story, AFFGEN writes a suspenseful story with the help of the intriguing twist. We want to highlight the generalizability of AFFGEN, which leverage words’ affective dimensions to promote text interestingness regardless of the prompt. \n\n**ii)** Specific prompts can indeed give you interesting stories but they get repetitive in spirit. We generated 10 stories using the prompt “Write a 5-sentence interesting short story with a mystery”. However, we found out that 7 out of 10 stories are talking about a secret being “unraveled” at the end. E.g. A photographer discovering a secret in a haunted lighthouse, a young woman discovering a hidden garden, and a man finding the hidden treasure… Our concern is that repetitive storylines would limit the scope of this research. \n\nAs mentioned in line 385, we chose the prompt in the paper after experimentations without loss of generality. However, we do recognize that optimizing instruction could potentially improve the quality of generated stories. Future research could study how prompt engineering, alignment, and human-in-the-loop storytelling can empower the task of generating interesting stories, but is out of scope for the current paper.\n\n**3) Regarding reviewer’s question on interpreting ablation study**, We would like to better support our ablation study with the following examples:\n\n`AFFGEN_large_beam_size_60`: Ella wrote her grandma a letter. The big scary wolf attack scare hit hard.\n\n`AFFGEN_small_beam_size_10`: Ella wrote her grandma a letter. The letters would get lost.\n\nGiven the example above, when we fix a large beam size, the generated text is interesting but not coherent. The reason behind the poor coherence is because during decoding, AFFGEN with a fixed large beam size chose a series of low-probability but high arousal tokens. While promoting the interestingness of the sentence, it hurt the story coherence and became not interesting. On the other hand, AFFGEN with a small fixed beam size generates content that is coherent but not as interesting. This shows the importance of not fixing the beam size and dynamically switching while the sentence is being generated.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Improving the interestingness of the story is an interesting area of text generation but it is not well studied. The proposed algorithm is relatively simple: just generating one sentence containing an intriguing twist. The results show the method to be effective, significantly improving upon the base model with the standard decoding algorithm. The result that dynamic beam sizing improves interestingness compared to using a constant large beam size is somewhat surprising. Because the motivation to use dynamic beam size is that a larger beam size can generate more intriguing words but at the expense of coherence. The motivation for using dynamic beam size (as explained in Section 4.3) is to achieve a better trade-off between coherence and interestingness (as well as achieving better decoding efficiency). ", "reasons_to_reject": "The main weakness is that the proposed algorithm seems to be tailored to the five-sentence story dataset, which is the only test dataset. For example, many examples in the paper show that by generating the second sentence with an intriguing twist, the rest of the story becomes more interesting. This may be true for the five-sentence story, but it's hard to imagine this is the case for long stories. Also, the affective reranking contains many hand-crafted rules, which also seem to be tailored to the five-sentence story dataset.\n\nAnother weakness is that the comparison between GPT3 and ChatGPT seems to be unfair. While the authors build many hand-crafted rules  for affective reranking, the prompts for GPT3/ChatGPT are not well optimized. I played with ChatGPT and I can easily get more interesting stories by changing for prompt slightly from the prompt in Table 8. For example, by changing \"Write a 5-sentence interesting short story\" to \"Write a 5-sentence interesting short story with a mystery\".\n\nI think the most interesting component is the dynamic beam sizing. While the ablation study shows it's better than a constant large beam size (in contrast to the original motivation), it does not give more insights into the underlying reason. So it is still not clear whether this is a generalizable method outside of the five-sentence dataset.\n\nFinally, the clarity can be improved. It's not very clear how the contextual bandit model works. \n\n", "questions_to_authors": "In Table 1, which language model is used for calculating PPL (for evaluating fluency)?", "paper_topic_and_main_contributions": "This paper introduces a new decoding algorithm to generate interesting stories. The algorithm works by generating most of the sentences using the beam search but generating a single sentence with an intriguing twist with the new decoding algorithm. Which sentence contains the intriguing twist is sampled from a prior distribution. The new decoding algorithm works by dynamically choosing a beam size and reranking the results with hand-crafted rules. The proposed algorithm is compared to GPT2, GPT3 (base model without applying the new decoding algorithm), and ChatGPT as baselines. The authors claim that both automatic evaluation and human evaluation show that the proposed algorithm can generate more interesting stories than the baselines. Ablation study shows that dynamic beam sizing helps improve the interestingness. ", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IFQdZTY7yZ", "reviewer": "Reviewer_pHn9", "summary": "", "questions": "1. In Sec 4.1, how do you form D(p) from the Writing-Prompt dataset? How do you identify or quantify intriguing twists in a sentence?\n2. Both the arousal and valence scores are based on a lexicon. Are there out-of-vocabulary cases during evaluation? If so, how do you deal with it?\n3. What is a valence score? Maybe add some explanations\n4. Line 387, in the prompt example for GPT3, where is the first sentence of the story?\n5. Why only compare to GPT 2 & 3?  Are they the SOTA in story generations?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer, \n\nThank you for your thoughtful review and constructive suggestions. We appreciate the time you have invested in evaluating our work. We address the concerns you raised below:\n\n**1) Insertion Point of the Twist**\n\nAFFGEN does not employ an additional model to identify the twist's insertion point. As mentioned in section 4.1, we use a data-driven methodology to shape a distribution $D_{IT}$, which characterizes the typical positions of intriguing twists within sentences. At the inference stage, we draw from $D_{IT}​$ to determine the insertion point of the twist. This strategy ensures that our approach remains both uncomplicated and lightweight, circumventing any additional training overhead. \n\n**2) Complexity of Generating 'Interesting' Stories**  \n\nOn “too complicated”: Generating compelling narratives is intrinsically challenging. Our approach, driven by linguistic and literary criticism theories, aims to regulate arousal,valence and coherence through our tailored decoding techniques. This is a technically challenging task and so the proposed approach might seem “complicated”. \n\nOn “not interesting”: We are not sure what the reviewer means by this. We look forward to further clarification on this. \n\nOn definition of interesting: The term 'interesting' carries with it multiple dimensions. We used arousal and valence as one way of characterizing interestingness and show that this works. We recognize the potential for future endeavors to further dissect and model the multifaceted nature of 'interestingness’ but it is out of scope of the current paper. \n\n**3) Considering Arousal and Valence Over the Entire Sentence**  \nWe did conduct experiments that contemplated this very approach. However, our results showed that directly modulating valence and arousal across the entire sentence has less than 20% chance generating interesting stories. We observe that content diversity is also limited, especially on model’s choices on the main verb of the sentence. This motivates us to dive deep to token level to carefully balance between coherence and interestingness.\n\n**4) Including other SOTA methods**  \n\nWe would like to highlight that the generation of intriguing stories is still in an early stage. There are limited methods specifically tailored to this task, which has restricted the breadth of our comparative study. We strategically chose GPT models  as our baseline comparisons because they are widely recognized as the state-of-the-art (SOTA) story generators.  [Modeling Protagonist Emotions for Emotion-Aware Storytelling](https://aclanthology.org/2020.emnlp-main.426.pdf), [Choose Your Own Adventure: Paired Suggestions in Collaborative Writing for Evaluating Story Generation Models](https://aclanthology.org/2021.naacl-main.279.pdf), and [The Authenticity Gap in Human Evaluation](https://aclanthology.org/2022.emnlp-main.406/). Outperforming GPT-2, GPT-3, and ChatGPT, as AFFGEN has demonstrated, is indicative of the effectiveness and the potential of our approach. We welcome concrete suggestions for any other method that is a better fit for a baseline.  \n\n**5) Clarification on Table 1**  \n\nTable 1’s goal is not necessarily to showcase significant gains in coherence over baselines but rather to underscore that in our pursuit of enhancing story interestingness, we have not compromised on the coherence. In particular, Table 1 evaluates interestingness (via Arousal score) and coherence (via PPL, Uni and Rub). For interestingness, we can see from the table that AFFGEN’s improvement over the baselines is statistically significant. For coherence, AFFGEN performs at par with the baselines. Some baselines have better scores but they are not always statistically significant. This indicates that AFFGEN produces interesting but coherent stories. We will add more clarification about this. \n\n**6) Getting intriguing twist distribution from WritingPrompts dataset**  \n\nAs we pointed out in line 201, we followed the methodology described in [Wilmont and Keller 2020](https://doi.org/10.18653/v1/2020.acl-main.161)’s paper. They used a hierarchical language model that encodes stories and computes surprise and uncertainty reduction through entropy-driven modeling to identify the twist of the story.\n\n**7) OOV scenario for NRC-VAD lexicon group**  \n\nWe measure arousal and valence scores based on NRC VAD Lexicon, which is built through human ratings of valence, arousal, and dominance for more than 20,000 English words. VAD’s vocabulary size is quite large and it is has been the standard way to quantify valence and arousal of sentences. [A survey of textual emotion recognition and its challenges](https://ieeexplore.ieee.org/abstract/document/9330790/), [Semeval-2018 task 1: Affect in tweets](https://aclanthology.org/S18-1001/), and [Humor Knowledge Enriched Transformer for Understanding Multimodal Humor](https://ojs.aaai.org/index.php/AAAI/article/view/17534). Given the comprehensiveness and high coverage of the NRC VAD Lexicon, this knowledge source should be representative enough to handle most cases. On the contrary, there could always be rare out-of-vocabulary cases which we believe the impact would be minor. \n\n**8) Clarification on valence score**  \n\nAs referenced between lines 343 to 347, the valence score of a sequence of tokens is the cumulation of the valence scores of its individual tokens. Drawing from the NRC VAD lexicon, valence gauges the emotional sentiment of a word: a high score signifies positive sentiments like happiness or joy, while a low score indicates negative sentiments such as sadness or anger.\n\n**9) Regarding missing first sentence in GPT-3 prompt**  \n\nWe presented the complete prompt in Table 8, Appendix. In line 387, due to the space limit, we removed the place-holder for the first sentence of the story. However, since it is causing confusion, we will put it back. \n\n**10) Regarding SOTA in story generations**  \n\nYes, GPT-style models are currently SOTA in story generations as we discussed in `7)`\n\nOur method is technically sophisticated but this is out of necessity. As pointed out by [Roemmele (2021)](https://roemmele.github.io/publications/human_computer_authoring.pdf), the notion of interesting stories is subjective and contextual. So simple fine-tuning is not enough for this task (as also demonstrated by GPT-2 result in Table 1). Existing LMs can generate coherent texts but do not understand what \"interestingness\" means to humans.  Instead, there is a need to carefully design methods that can generate interesting stories without compromising coherence. AFFGEN meticulously balances text's interestingness (via affective reranking and large beams) and coherence (via switching to smaller beams).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper proposes Dynamic Beam Sizing and Affective Reranking methods to make the generated sentences more interesting. The evaluation shows that the proposed method can improve the intrigue and coherence of a generated story.\n\n----- After rebuttal -----\n\nThe paper introduces the Dynamic Beam Sizing and Affective Reranking techniques as a means to enhance the appeal of generated sentences. The evaluation demonstrates the effectiveness of these approaches in augmenting the intrigue while keeping the coherence of generated stories. Key contributions lie in its simplicity and adaptability. The method employs a count-based model to identify one twist sentence and subsequently inserts the intriguing twists. This infusion elevates the arousal and valence of the sentences and meanwhile introduces variations to the rest of the story. The inclusion of dynamic beam sizing further bolsters the method's adaptability to different scenarios.", "reasons_to_reject": "1. Overall, the idea seems too complicated and not interesting. AFFGEN needs to train a model to identify the insertion position of the twist and uses an off-the-shelf model to predict trigger likelihood, but simply refer to the arousal and valence scores to define \"interesting\". Why not directly consider the arousal and valence scores over the whole sentence?\n2. The evaluation results are not convincing. \n    - The comparison only involved GPT 2 & 3 but no other SOTA methods.\n    - Table 1 doesn't show significant gains from AFFGEN to the baselines.\n3. Many technical details are missing from the paper. See `Questions For The Authors`.\n\n----- After rebuttal -----\n\nMore efforts are needed in selecting better baseline results. If there are no other comparable methods, we could probably carefully prompt GPT 2 & 3 to make it a fair comparison.  For example, since the proposed method explicitly leverage the concepts of arousal and valence, we can also include them in the prompt sentence, something like \"Generate a story that has high arousal and valence scores. The total length ....\".", "questions_to_authors": "1. In Sec 4.1, how do you form D(p) from the Writing-Prompt dataset? How do you identify or quantify intriguing twists in a sentence?\n2. Both the arousal and valence scores are based on a lexicon. Are there out-of-vocabulary cases during evaluation? If so, how do you deal with it?\n3. What is a valence score? Maybe add some explanations\n4. Line 387, in the prompt example for GPT3, where is the first sentence of the story?\n5. Why only compare to GPT 2 & 3?  Are they the SOTA in story generations?", "paper_topic_and_main_contributions": "The paper is dedicated to enhancing story generation, aiming to make narratives more captivating and emotionally engaging. The core concept revolves around introducing intriguing twists during sentence generation. To achieve this, the authors introduce the AFFGEN model, which can predict an appropriate position within sentences and then insert the intriguing twist. Additionally, the authors propose two strategies, namely Dynamic Beam Sizing and Affective Reranking, to select words during decoding that elevate the sentence's arousal and valence scores while reducing perplexity and make the sentence more interesting. The study provides a comparison of AFFGEN with the latest GPT2 and GPT3 language models, evaluating both automatically and through human assessment. The results demonstrate that AFFGEN outperforms these models, producing narratives that are more interesting and cohesive.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Line 366, focus --> focuses\n2. There're formatting issues around line 208 - 209"}, "extra_text": {}}, {"review_id": "rgGTf8Ne9c", "reviewer": "Reviewer_Y6wU", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer, \n\nThank you for your thoughtful review and constructive suggestions. We appreciate the time you have invested in evaluating our work. We address the concerns you raised below:\n\n**Regarding comparing with fine-tuned model**  \nWe compared with GPT-large which is fine-tuned on RocStories. And AFFGENl outperforms this well-finetuned GPT2-large, as demonstrated in Table 1. We want to highlight that fine-tuning a model beyond the setup discussed in our paper  is not trivial as it requires a dataset of interesting stories only, which does not exist. As highlighted by [Roemmele (2021)](https://roemmele.github.io/publications/human_computer_authoring.pdf), the notion of “interesting stories'' carries inherent subjectivity and is greatly influenced by context. Future work can look into creating resources of interesting stories but it extends beyond the purview of our current research.\n\nOur method is technically sophisticated but this is out of necessity. As pointed out by [Roemmele (2021)](https://roemmele.github.io/publications/human_computer_authoring.pdf), the notion of interesting stories is subjective and contextual. So simple fine-tuning is not enough for this task. Existing LMs can generate coherent texts but do not understand what \"interestingness\" means to humans.  Instead, there is a need to carefully design methods that can generate interesting stories without compromising coherence. AFFGEN meticulously balances text's interestingness (via affective reranking and large beams) and coherence (via switching to smaller beams).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The author believes that creativity is the deficiency of the current generation model, so the author directly chooses to fine-adjust GPT-2 and GPT-3 to build AFFGEN, and the results also show that AFFGEN is better than GPT-2 and GPT-3 in creativity, which provides a new inspiration for story generation.", "reasons_to_reject": "The proposed AFFGEN model requires the exploration of different beam sizes and affective rearrangements. It's unclear whether adding these two technologies will outperform a well-fine-tuned model while keeping the system design simple.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The author aims to generate interesting stories through twists and turns. They proposed AFFGEN(Controlled Affective Story Generator) to improve the interestingness of text by controlling the coherence of text and utilizing the emotional dimension of vocabulary. Dynamic beam size occasionally explores longer search beams, and affective rearrangement reorders candidate sentences based on arousal and value scores. Their automated ratings on the dataset showed that AFFGEN generated stories that were high in Arousal score but higher than baselines in arousal score. The manual assessment considers how well the generated stories are scored on six dimensions, including coherence, emotional engagement, empathy, interestingness, and overall preference. The results showed that AFFGEN achieved leading human evaluations most of the time.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents experiments in generating more or less interesting 5-sentences stories by manipulating the generator's beam search size dynamically and introducing a specific generation step for 'twist sentence' candidates and some lexicon-based reranking, which outperform GPT models in the automatic and human evaluation. Reviewers liked the task idea and some of the proposals in the paper, but criticized the very tailord nature of the task (exactly 5-sentence stories, in which for example a twist in sentence 2 seems most important), the lack of experiments using more effective prompts for the GPT comparisons, and the relatively weak results in Table 1 - as the rebuttal points out, the main positive result here hinges only on the arousal score, which is highly dependent on individual lexical items based on Mohammad 2018, and may have some connection with the lexicon-based reranking. Having read all of the rebuttals I am not completely convinced by the authors' argument about limitation to 5-sentence stories based on LLMs' difficulties in creating longer coherent stories - especially when heuristics are involved, a tailored approach will likely fare better when the outputs are more templatic in nature, and an attempt at a few-shot approach with examples of what the authors want from the model is also missing here. The latter point is especially important given the claim that LMs don't understand what humans find interesting - they cannot know what the authors are targeting exactly, unless we give them examples. At the same time, I agree that the experiments form an interesting first step in exploring this task and the nature of 'twists', so although reviewers were consistently on the fence with excitement scores of 3, if there is space, I would rather see this paper published than rejected. If it is not accepted to the main conference, I think it would make a good candidate for a Findings paper.
Decision: Accept-Findings