Screenplay Summarization Using Latent Narrative Structure (2020)

Abstract: Most general-purpose extractive summarization models are trained on news articles, which are short and present all important information upfront. As a result, such models are biased on position and often perform a smart selection of sentences from the beginning of the document. When summarizing long narratives, which have complex structure and present information piecemeal, simple position heuristics are not sufficient. In this paper, we propose to explicitly incorporate the underlying structure of narratives into general unsupervised and supervised extractive summarization models. We formalize narrative structure in terms of key narrative events (turning points) and treat it as latent in order to summarize screenplays (i.e., extract an optimal sequence of scenes). Experimental results on the CSI corpus of TV screenplays, which we augment with scene-level summarization labels, show that latent turning points correlate with important aspects of a CSI episode and improve summarization performance over general extractive algorithms leading to more complete and diverse summaries.

## introduction automatic summarization has enjoyed renewed interest in recent years thanks to the popularity of modern neural network-based approaches @xcite @xcite and the availability of large-scale datasets containing hundreds of thousands of document-summary pairs @xcite @xcite @xcite . most efforts to date have concentrated on the summarization of news articles which tend to be relatively short and formulaic following an "inverted pyramid" structure which places the most essential, novel and interesting elements of a story in the beginning and supporting material and secondary details afterwards. the rigid structure of news articles is expedient since important passages can be identified in predictable locations (e.g., by performing a "smart selection" of sentences from the beginning of the document) and the structure itself can be explicitly taken into account in model design (e.g., by encoding the relative and absolute position of each sentence). in this paper we are interested in summarizing longer narratives, i.e., screenplays, whose form and structure is far removed from newspaper articles. screenplays are typically between 110 and 120 pages long (20k words), their content is broken down into scenes, which contain mostly dialogue (lines the actors speak) as well as descriptions explaining what the camera sees. moreover, screenplays are characterized by an underlying narrative structure, a sequence of events by which , key events that segment the story into sections). we next create a summary by selecting informative scenes, i.e.,semantically related to turning points. a story is defined @xcite , and by the story's characters and their roles @xcite ). contrary to news articles, the gist of the story in a screenplay is not disclosed at the start, information is often revealed piecemeal; characters evolve and their actions might seem more or less important over the course of the narrative. from a modeling perspective, obtaining training data is particularly problematic: even if one could assemble screenplays and corresponding summaries (e.g., by mining imdb or wikipedia), the size of such a corpus would be at best in the range of a few hundred examples not hundreds of thousands. also note that genre differences might render transfer learning (pan and yang, 2010) difficult, e.g., a model trained on movie screenplays might not generalize to sitcoms or soap operas. given the above challenges, we introduce a number of assumptions to make the task feasible. firstly, our goal is to produce informative summaries, which serve as a surrogate to reading the full script or watching the entire film. secondly, we follow gorinski and lapata (2015) in conceptualizing screenplay summarization as the task of identifying a sequence of informative scenes. thirdly, we focus on summarizing television programs such as csi: crime scene investigation @xcite which revolves around a team of forensic investigators solving criminal cases. such programs have a complex but well-defined structure: they open with a crime, the crime scene is examined, the victim is identified, suspects are introduced, forensic clues are gathered, suspects are investigated, and finally the case is solved. in this work, we adapt general-purpose extractive summarization algorithms @xcite to identify informative scenes in screenplays and instill in them knowledge about narrative film structure @xcite @xcite . specifically, we adopt a scheme commonly used by screenwriters as a practical guide for producing successful screenplays. according to this scheme, wellstructured stories consist of six basic stages which are defined by five turning points (tps), i.e., events which change the direction of the narrative, and determine the story's progression and basic thematic units. in figure 1 , tps are highlighted for a csi episode. although the link between turning points and summarization has not been previously made, earlier work has emphasized the importance of narrative structure for summarizing books (mihalcea and ceylan, 2007) and social media content @xcite . more recently, @xcite have shown how to identify turning points in feature-length screenplays by projecting synopsis-level annotations. crucially, our method does not involve manually annotating turning points in csi episodes. instead, we approximate narrative structure automatically by pretraining on the annotations of the tripod dataset of @xcite and employing a variant of their model. we find that narrative structure representations learned on their dataset (which was created for feature-length films), transfer well across cinematic genres and computational tasks. we propose a framework for end-to-end training in which narrative structure is treated as a latent variable for summarization. we extend the csi dataset @xcite with binary labels indicating whether a scene should be included in the summary and present experiments with both supervised and unsupervised summarization models. an overview of our approach is shown in figure 2 . our contributions can be summarized as follows: (a) we develop methods for instilling knowledge about narrative structure into generic su-pervised and unsupervised summarization algorithms; (b) we provide a new layer of annotations for the csi corpus, which can be used for research in long-form summarization; and (c) we demonstrate that narrative structure can facilitate screenplay summarization; our analysis shows that key events identified in the latent space correlate with important summary content. ## related work a large body of previous work has focused on the computational analysis of narratives @xcite . attempts to analyze how stories are written have been based on sequences of events @xcite , plot units @xcite @xcite and their structure @xcite , as well as on characters or personas in a narrative @xcite @xcite @xcite and their relationships @xcite @xcite . as mentioned earlier, work on summarization of narratives has had limited appeal, possibly due to the lack of annotated data for modeling and evaluation. @xcite summarize short stories based on importance criteria (e.g., whether a segment contains protagonist or location information); they create summaries to help readers decide whether they are interested in reading the whole story, without revealing its plot. @xcite summarize books with an unsupervised graph-based approach operating over segments (i.e., topical units). their algorithm first generates a summary for each segment and then an overall summary by collecting sentences from the individual segment summaries. focusing on screenplays, gorinski and lapata (2015) generate a summary by extracting an optimal chain of scenes via a graph-based approach centered around the main characters. in a similar fashion, @xcite create video summaries for tv series episodes; their algorithm ranks sub-scenes in terms of importance using features based on character graphs and textual cues available in the subtitles and movie scripts. @xcite introduce the moviegraphs dataset, which also uses character-centered graphs to describe the content of movie video clips. our work synthesizes various strands of re-search on narrative structure analysis @xcite , screenplay summarization (gorinski and lapata, 2015), and neural network modeling @xcite . we focus on extractive summarization and our goal is to identify an optimal sequence of key events in a narrative. we aim to create summaries which re-tell the plot of a story in a concise manner. inspired by recent neural network-based approaches @xcite @xcite , we develop supervised and unsupervised models for our summarization task based on neural representations of scenes and how these relate to the screenplay's narrative structure. contrary to most previous work which has focused on characters, we select summary scenes based on events and their importance in the story. our definition of narrative structure closely follows @xcite . however, the model architectures we propose are general and could be adapted to different plot analysis schemes @xcite . to overcome the difficulties in evaluating summaries for longer narratives, we also release a corpus of screenplays with scenes labeled as important (summary worthy). our annotations augment an existing dataset based on csi episodes @xcite , which was originally developed for incremental natural language understanding. ## problem formulation let d denote a screenplay consisting of a sequence of scenes @xmath0, s 2 , . . . , s n }. our aim is to select a subset @xmath1, . . . , s k } consisting of the most informative scenes (where @xmath2. note that this definition produces extractive summaries; we further assume that selected scenes are presented according to their order in the screenplay. we next discuss how summaries can be created using both unsupervised and supervised approaches, and then move on to explain how these are adapted to incorporate narrative structure. ## experimental setup crime scene investigation dataset we performed experiments on an extension of the csi dataset foot_5 introduced by @xcite . it consists of 39 csi episodes, each annotated with word-level labels denoting whether the perpetrator is mentioned in the utterances characters speak. we further collected scene-level binary labels indicating whether episode scenes are important and should be included in a summary. three human judges performed the annotation task after watching the csi episodes scene-by-scene. to facilitate the annotation, judges were asked to indicate why they thought a scene was important, citing the following reasons: it revealed (i) the victim, (ii) the cause of death, (iii) an autopsy report, (iv) crucial evidence, (v) the perpetrator, and (vi) the motive or the relation between perpetrator and victim. annotators were free to select more than one or none of the listed reasons where appropriate. we can think of these reasons as high-level aspects a good summary should cover (for csi and related crime series). annotators were not given any information about tps or narrative structure; the annotation was not guided by theoretical considerations, rather our aim was to produce useful csi summaries. ## results and analysis is narrative structure helpful? we perform 10-fold cross-validation and evaluate model performance in terms of f1 score. in ## conclusions in this paper we argued that the underlying structure of narratives is beneficial for long-form summarization. we adapted a scheme for identifying narrative structure (i.e., turning points) in hollywood movies and showed how this information can be integrated with supervised and unsupervised extractive summarization algorithms. experiments on the csi corpus showed that this scheme transfers well to a different genre (crime investigation) and that utilizing narrative structure boosts summarization performance, leading to more complete and diverse summaries. analysis of model output further revealed that latent events encapsulated by turning points correlate with important aspects of a csi summary. although currently our approach relies solely on textual information, it would be interesting to incorporate additional modalities such as video or audio. audiovisual information could facilitate the identification of key events and scenes. besides narrative structure, we would also like to examine the role of emotional arcs @xcite in a screenplay. an often integral part of a compelling story is the emotional experience that is evoked in the reader or viewer (e.g., somebody gets into trouble and then out of it, somebody finds something wonderful, loses it, and then finds it again). understanding emotional arcs may be useful to revealing a story's shape, highlighting important scenes, and tracking how the story develops for different characters over time. hao zheng and mirella lapata. 2019. sentence centrality revisited for unsupervised summarization. arxiv preprint arxiv:1906.03508. qingyu zhou, nan yang, furu wei, shaohan huang, ming zhou, and tiejun zhao. 2018. neural document summarization by jointly learning to score and select sentences. arxiv preprint arxiv:1807.02305.