SlowGuess/ACL_2020_OCR huggingface dataset:
David Wilmot and Frank Keller
10 Crichton Street, Edinburgh EH8 9AB, UK
david.wilmot@ed.ac.uk, keller@inf.ed.ac.uk
Suspense is a crucial ingredient of narrative fiction, engaging readers and making stories compelling. While there is a vast theoretical literature on suspense, it is computationally not well understood. We compare two ways for modelling suspense: surprise, a backward-looking measure of how unexpected the current state is given the story so far; and uncertainty reduction, a forward-looking measure of how unexpected the continuation of the story is. Both can be computed either directly over story representations or over their probability distributions. We propose a hierarchical language model that encodes stories and computes surprise and uncertainty reduction. Evaluating against short stories annotated with human suspense judgements, we find that uncertainty reduction over representations is the best predictor, resulting in near human accuracy. We also show that uncertainty reduction can be used to predict suspenseful events in movie synopses.
As current NLP research expands to include longer, fictional texts, it becomes increasingly important to understand narrative structure. Previous work has analyzed narratives at the level of characters and plot events (e.g., Gorinski and Lapata, 2018; Martin et al., 2018). However, systems that process or generate narrative texts also have to take into account what makes stories compelling and enjoyable. We follow a literary tradition that makes And then? (Forster, 1985; Rabkin, 1973) the primary question and regards suspense as a crucial factor of storytelling. Studies show that suspense is important for keeping readers' attention (Khrypko and Andreae, 2011), promotes readers' immersion and suspension of disbelief (Hsu et al., 2014), and plays a big part in making stories enjoyable and interesting (Oliver, 1993; Schraw et al., 2001). Computationally less well understood, suspense has
only sporadically been used in story generation systems (O'Neill and Riedl, 2014; Cheong and Young, 2014).
Suspense, intuitively, is a feeling of anticipation that something risky or dangerous will occur; this includes the idea both of uncertainty and jeopardy. Take the play Romeo and Juliet: Dramatic suspense is created throughout — the initial duel, the meeting at the masquerade ball, the marriage, the fight in which Tybalt is killed, and the sleeping potions leading to the death of Romeo and Juliet. At each moment, the audience is invested in something being at stake and wonders how it will end.
This paper aims to model suspense in computational terms, with the ultimate goal of making it deployable in NLP systems that analyze or generate narrative fiction. We start from the assumption that concepts developed in psycholinguistics to model human language processing at the word level (Hale, 2001, 2006) can be generalised to the story level to capture suspense, the Hale model. This assumption is
similar concepts to model suspense in games (Ely et al., 2015; Li et al., 2018), the Ely model. Common to both approaches is the idea that suspense is a form of expectation: In games, we expect to win or lose instead in stories, we expect that the narrative will end a certain way.
We will therefore compare two ways for modelling narrative suspense: surprise, a backward-looking measure of how unexpected the current state is given the story so far; and uncertainty reduction, a forward-looking and measure of how unexpected the continuation of the story is. Both measures can be computed either directly over story representations, or indirectly over the probability distributions over such representations. We propose a hierarchical language model based on Generative Pre-Training (GPT, Radford et al., 2018) to encode story-level representations and develop an
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1763-1788 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
inference scheme that uses these representations to compute both surprise and uncertainty reduction. For evaluation, we use the WritingPrompt corpus of short stories (Fan et al., 2018), part of which we annotate with human sentence-by-sentence judgements of suspense. We find that surprise over representations and over probability distributions both predict suspense judgements. However uncertainty reduction over representations is better, resulting in near human-level accuracy. We also show that our models can be used to predict turning points, i.e., major narrative events, in movie synopses (Papalampidi et al., 2019).
In narratology, uncertainty over outcomes is traditionally seen as suspenseful (e.g., O'Neill, 2013; Zillmann, 1996; Abbott, 2008). Other authors claim that suspense can exist without uncertainty (e.g., Smuts, 2008; Hoeken and van Vliet, 2000; Gerrig, 1989) and that readers feel suspense even when they read a story for the second time (Delatore et al., 2018), which is unexpected if suspense is uncertainty; this is referred to as the paradox of suspense (Prieto-Pablos, 1998; Yanal, 1996). Considering Romeo and Juliet again, in the first view suspense is motivated by primarily by uncertainty over what will happen. Who will be hurt or killed in the fight? What will happen after marriage? However, at the beginning of the play we are told "from forth the fatal loins of these two foes, a pair of star-crossed lovers take their life", and so the suspense is more about being invested in the plot than not knowing the outcome, aligning more with the second view: suspense can exist without uncertainty. We do not address the paradox of suspense directly in this paper, but we are guided by the debate to operationalise methods that encompass both views. The Hale model is closer to the traditional model of suspense as being about uncertainty. In contrast, the Ely model is more in line with the second view that uncertainty matters less than consequentially different outcomes.
In NLP, suspense is studied most directly in natural language generation, with systems such as Dramatis (O'Neill and Riedl, 2014) and Suspenser (Cheong and Young, 2014), two planning-based story generators that use the theory of Gerrig and Bernardo (1994) that suspense is created when a protagonist faces obstacles that reduce successful outcomes. Our approach, in contrast, models sus
sense using general language models fine-tuned on stories, without planning and domain knowledge. The advantage is that the model can be trained on large volumes of available narrative text without requiring expensive annotations, making it more generalisable.
Other work emphasises the role of characters and their development in story understanding (Bamman et al., 2014, 2013; Chaturvedi et al., 2017; Iyyer et al., 2016) or summarisation (Gorinski and Lapata, 2018). A further important element of narrative structure is plot, i.e., the sequence of events in which characters interact. Neural models have explicitly modelled events (Martin et al., 2018; Harrison et al., 2017; Rashkin et al., 2018) or the results of actions (Roemmele and Gordon, 2018; Liu et al., 2018a,b). On the other hand, some neural generation models (Fan et al., 2018) just use a hierarchical model on top of a language model; our architecture follows this approach.
3 Models of Suspense
In order to formalise measures of suspense, we assume that a story consists of a sequence of sentences. These sentences are processed one by one, and the sentence at the current timepoint  $t$  is represented by an embedding  $e_t$  (see Section 4 for how embeddings are computed). Each embedding is associated with a probability  $P(e_t)$ . Continuations of the story are represented by a set of possible next sentences, whose embeddings are denoted by  $e_{t+1}^i$ .
The first measure of suspense we consider is surprise (Hale, 2001), which in the psycholinguistic literature has been successfully used to predict word-based processing effort (Demberg and Keller, 2008; Roark et al., 2009; Van Schijndel and Linzen, 2018a,b). Surprise is a backward-looking predictor: it measures how unexpected the current word is given the words that preceded it (i.e., the left context). Hale formalises surprise as the negative log of the conditional probability of the current word. For stories, we compute surprise over sentences. As our sentence embeddings  $e_t$  include information about the left context  $e_1, \ldots, e_{t-1}$ , we can write Hale surprise as:
$$
S _ {t} ^ {\text {H a l e}} = - \log P \left(e _ {t}\right) \tag {1}
$$
An alternative measure for predicting word-by-word processing effort used in psycholinguistics is entropy reduction (Hale, 2006). This measure is
forward-looking: it captures how much the current word changes our expectations about the words we will encounter next (i.e., the right context). Again, we compute entropy at the story level, i.e., over sentences instead of over words. Given a probability distribution over possible next sentences  $P(e_{t+1}^{i})$ , we calculate the entropy of that distribution. Entropy reduction is the change of that entropy from one sentence to the next:
$$
H _ {t} = - \sum_ {i} P \left(e _ {t + 1} ^ {i}\right) \log P \left(e _ {t + 1} ^ {i}\right) \tag {2}
$$
$$
U _ {t} ^ {\text {H a l e}} = H _ {t - 1} - H _ {t}
$$
Note that we follow Frank (2013) in computing entropy over surface strings, rather than over parse states as in Hale's original formulation.
In the economics literature, Ely et al. (2015) have proposed two measures that are closely related to Hale surprise and entropy reduction. At the heart of their theory of suspense is the notion of belief in an end state. Games are a good example: the state of a tennis game changes with each point being played, making a win more or less likely. Ely et al. define surprise as the amount of change from the previous time step to the current time step. Intuitively, large state changes (e.g., one player suddenly comes close to winning) are more surprising than small ones. Representing the state at time  $t$  as  $e_t$ , Ely surprise is defined as:
$$
S _ {t} ^ {\text {E l y}} = \left(e _ {t} - e _ {t - 1}\right) ^ {2} \tag {3}
$$
Ely et al.'s approach can be adapted for modelling suspense in stories if we assume that each sentence in a story changes the state (the characters, places, events in a story, etc.). States  $e_t$  then become sentence embeddings, rather than beliefs in end states, and Ely surprise is the distance between the current embedding  $e_t$  and the previous embedding  $e_{t-1}$ . In this paper, we will use L1 and L2 distances; other authors (Li et al., 2018) experiment with information gain and KL divergence, but found worse performance when modelling suspense in games. Just like Hale surprise, Ely surprise models backward-looking prediction, but over representations, rather than over probabilities.
Ely et al. also introduce a measure of forward-looking prediction, which they define as the expected difference between the current state  $e_{t}$  and
the next state  $e_{t + 1}$
$$
\begin{array}{l} U _ {t} ^ {\mathrm {E l y}} = \mathbb {E} \left[ \left(e _ {t} - e _ {t + 1} ^ {i}\right) ^ {2} \right] \\ = \sum_ {i} P \left(e _ {t + 1} ^ {i}\right) \left(e _ {t} - e _ {t + 1} ^ {i}\right) ^ {2} \tag {4} \\ \end{array}
$$
This is closely related to Hale entropy reduction, but again the entropy is computed over states (sentence embeddings in our case), rather than over probability distributions. Intuitively, this measure captures how much the uncertainty about the rest of the story is reduced by the current sentence. We refer to the forward-looking measures in Equations (2) and (4) as Hale and Ely uncertainty reduction, respectively.
Ely et al. also suggest versions of their measures in which each state is weighted by a value  $\alpha_{t}$ , thus accounting for the fact that some states may be more inherently suspenseful than others:
$$
\begin{array}{l} S _ {t} ^ {\alpha \mathrm {E l y}} = \alpha_ {t} \left(e _ {t} - e _ {t - 1}\right) ^ {2} \\ U _ {t} ^ {\alpha \mathrm {E l y}} = \mathbb {E} \left[ \alpha_ {t + 1} \left(e _ {t} - e _ {t + 1} ^ {i}\right) ^ {2} \right] \tag {5} \\ \end{array}
$$
We stipulate that sentences with high emotional valence are more suspenseful, as emotional involvement heightens readers' experience of suspense. This can be captured in Ely et al.'s framework by assigning the  $\alpha$ s the scores of a sentiment classifier.
3.2 Modelling Approach
We now need to show how to compute the surprise and uncertainty reduction measures introduced in the previous section. This involves building a model that processes stories sentence by sentence, and assigns each sentence an embedding that encodes the sentence and its preceding context, as well as a probability. These outputs can then be used to compute a surprise value for the sentence.
Furthermore, the model needs to be able to generate a set of possible next sentences (story continuations), each with an embedding and a probability. Generating upcoming sentences is potentially very computationally expensive since the number of continuations grows exponentially with the number of future time steps. As an alternative, we can therefore sample possible next sentences from a corpus and use the model to assign them embeddings and probabilities. Both of these approaches will produce sets of upcoming sentences, which we can then use to compute uncertainty reduction. While we have so far only talked about the next sentences, we will also experiment with uncertainty reduction computed using longer rollouts.
Figure 1: Architecture of our hierarchical model. See text for explanation of the components word_enc, sent_enc, and story_enc.
Our overall approach leverages contextualised language models, which are a powerful tool in NLP when pretrained on large amounts of text and fine tuned on a specific task (Peters et al., 2018; Devlin et al., 2019). Specifically, we use Generative Pre-Training (GPT, Radford et al., 2018), a model which has proved successful in generation tasks (Radford et al., 2019; See et al., 2019).
Hierarchical Model Previous work found that hierarchical models show strong performance in story generation (Fan et al., 2018) and understanding tasks (Cai et al., 2017). The language model and hierarchical encoders we use are unidirectional, which matches the incremental way in which human readers process stories when they experience suspense.
Figure 1 depicts the architecture of our hierarchical model. It builds a chain of representations that anticipates what will come next in a story, allowing us to infer measures of suspense. For a given sentence, we use GPT as our word encoder (word_enc in Figure 1) which turns each word in a sentence into a word embedding  $w_{i}$ . Then, we use an RNN (sent_enc) to turn the word embeddings of the sentences into a sentence embedding  $\gamma_{i}$ . Each sentence is represented by the hidden state of its last word, which is then fed into a second RNN
(story_enc) that computes a story embedding. The overall story representation is the hidden state of its last sentence. Crucially, this model also gives us  $e_t$ , a contextualised representation of the current sentence at point  $t$  in the story, to compute surprise and uncertainty reduction.
Model training includes a generative loss  $\ell_{gen}$  to improve the quality of the sentences generated by the model. We concatenate the word representations  $w_{j}$  for all word embeddings in the latest sentence with the latest story embedding  $e_{\max(t)}$ . This is run through affine ELU layers to produce enriched word embedding representations, analogous to the Deep Fusion model (Gulçehre et al., 2015), with story state instead of a translation model. The related Cold Fusion approach (Sriram et al., 2018) proved inferior.
Loss Functions To obtain the discriminatory loss  $\ell_{disc}$  for a particular sentence  $s$  in a batch, we compute the dot product of all the story embeddings  $e$  in the batch, and then take the cross-entropy across the batch with the correct next sentence:
$$
\ell_ {d i s c} \left(e _ {t + 1} ^ {i = s}\right) = - \log \frac {\exp \left(e _ {t + 1} ^ {i = s} \cdot e _ {t}\right)}{\sum_ {i} \exp \left(e _ {t + 1} ^ {i} \cdot e _ {t}\right)} \tag {6}
$$
Modelled on Quick Thoughts (Logeswaran and Lee, 2018), this forces the model to maximise the dot product of the correct next sentence versus other sentences in the same story, and negative examples from other stories, and so encourages representations that anticipate what happens next.
The generative loss in Equation (7) is a standard LM loss, where  $w_{j}$  is the GPT word embeddings from the sentence and  $e_{\max(t)}$  is the story context that each word is concatenated with:
$$
\ell_ {g e n} = - \sum_ {j} \log P \left(w _ {j} \mid w _ {j - 1}, w _ {j - 2}, \dots ; e _ {\max  (t)}\right) \tag {7}
$$
The overall loss is  $\ell_{disc} + \ell_{gen}$ . More advanced generation losses (e.g., Zellers et al., 2019) could be used, but are an order of magnitude slower.
4.2 Inference
We compute the measures of surprise and uncertainty reduction introduced in Section 3.1 using the output of the story encoder story_enc. In addition to the contextualised sentence embeddings  $e_t$ , this requires their probabilities  $P(e_t)$ , and a distribution over alternative continuations  $P(e_{t+1}^i)$ .
We implement a recursive beam search over a tree of future sentences in the story, looking between one and three sentences ahead (rollout). The
1Model code and scripts for evaluation are available at https://github.com/dwlmt/Story-Untangling/tree/acl-2020-dec-submission
probability is calculated using the same method as the discriminatory loss, but with the cosine similarity rather than the dot product of the embeddings  $e_t$  and  $e_{t+1}^i$  fed into a softmax function. We found that cosine outperformed dot product on inference as the resulting probability distribution over continuations is less concentrated.
Dataset The overall goal of this work is to test whether the psycholinguistic and economic theories introduced in Section 3 are able to capture human intuition of suspense. For this, it is important to use actual stories which were written by authors with the aim of being engaging and interesting. Some of the story datasets used in NLP do not meet this criterion; for example ROC Cloze (Mostafazadeh et al., 2016) is not suitable because the stories are very short (five sentences), lack naturallness, and are written by crowdworkers to fulfill narrow objectives, rather than to elicit reader engagement and interest. A number of authors have also pointed out technical issues with such artificial corpora (Cai et al., 2017; Sharma et al., 2018).
Instead, we use WritingPrompts (Fan et al., 2018), a corpus of circa  $300\mathrm{k}$  short stories from the /r/WritingPrompts subreddit. These stories were created as an exercise in creative writing, resulting in stories that are interesting, natural, and of suitable length. The original split of the data into  $90\%$  train,  $5\%$  development, and  $5\%$  test was used. Pre-processing steps are described in Appendix A.
Annotation To evaluate the predictions of our model, we selected 100 stories each from the development and test sets of the WritingPrompts corpus, such that each story was between 25 and 75 sentence in length. Each sentence of these stories was judged for narrative suspense; five master workers from Amazon Mechanical Turk annotated each story after reading instructions and completing a training phase. They read one sentence at a time and provided a suspense judgement using the five-point scale consisting of Big Decrease in suspense (1% of the cases), Decrease (11%), Same (50%), Increase (31%), and Big Increase (7%). In contrast to prior work (Delatorre et al., 2018), a relative rather than absolute scale was used. Relative judgements are easier to make while reading, though in practice, the suspense curves generated are very similar, with a long upward trajectory and flattening or dip near the end. After finishing a story, annotators had
GRU	LSTM
Loss	5.84	5.90
Discriminatory Acc.	0.55	0.54
Discriminatory Acc. k = 10	0.68	0.68
Generative Acc.	0.37	0.46
Generative Acc. k = 10	0.85	0.85
Cosine Similarity	0.48	0.50
L2 Distance	1.73	1.59
Number of Epochs	4	2
Table 1: For accuracy the baseline probability is 1 in 99;  $k = 10$  is the accuracy of the top 10 sentences of the batch. From the best epoch of training on the Writing-Prompts development set.
to write a short summary of the story.
In the instructions, suspense was framed as dramatic tension, as pilot annotations showed that the term suspense was too closely associated with murder mystery and related genres. Annotators were asked to take the character's perspective when reading to achieve stronger inter-annotator agreement and align closely with literary notions of suspense. During training, all workers had to annotate a test story and achieve  $85\%$  accuracy before they could continue. Full instructions and the training story are in Appendix B.
The inter-annotator agreement  $\alpha$  (Krippendorff, 2011) was 0.52 and 0.57 for the development and test sets, respectively. Given the inherently subjective nature of the task, this is substantial agreement. This was achieved after screening out and replacing annotators who had low agreement for the stories they annotated (mean  $\alpha < 0.35$ ), showed suspiciously low reading times (mean RT  $< 600$  ms per sentence), or whose story summaries indicated low-quality annotation.
Training and Inference The training used SGD with Nesterov momentum (Sutskever et al., 2013) with a learning rate of 0.01 and a momentum of 0.9. Models were run with early stopping based on the mean of the accuracies of training tasks. For each batch, 50 sentence blocks from two different stories were chosen to ensure that the negative examples in the discriminatory loss include easy (other stories) and difficult (same story) sentences.
We used the pretrained GPT weights but fine-tuned the encoder and decoder weights on our task. For the RNN components of our hierarchical model, we experimented with both GRU (Chung et al.,
2015) and LSTM (Hochreiter and Schmidhuber, 1997) variants. The GRU model had two layers in both sen_enc and story_enc; the LSTM model had four layers each in sen_enc and story_enc. Both had two fusion layers and the size of the hidden layers for both model variants was 768. We give the results of both variants on the tasks of sentence generation and sentence discrimination in Table 1. Both perform similarly, with slightly worse loss for the LSTM variant, but faster training and better generation accuracy. Overall, model performance is strong: the LSTM variant picks out the correct sentence  $54\%$  of the time and generates it  $46\%$  of the time. This indicates that our architecture successfully captures the structure of stories.
At inference time, we obtained a set of story continuations either by random sampling or by generation. Random sampling means that  $n$  sentences were selected from the corpus and used as continuations. For generation, sentences were generated using top- $k$  sampling (with  $k = 50$ ) using the GPT language model and the approach of Radford et al. (2019), which generates better output than beam search (Holtzman et al., 2018) and can outperform a decoder (See et al., 2019). For generation, we used up to 300 words as context, enriched with the story sentence embeddings from the corresponding points in the story. For rollouts of one sentence, we generated 100 possibilities at each step; for rollouts of two, 50 possibilities and rollouts of three, 25 possibilities. This keeps what is an expensive inference process manageable.