SlowGuess/ACL_2020_OCR huggingface dataset:
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch.
Patrice Pavis. 1998. Dictionary of the theatre: Terms, concepts, and analysis. University of Toronto Press.
Vladimir Iakovlevich Propp. 1968. Morphology of the Folktale. University of Texas.
Andrew J. Reagan, Lewis Mitchell, Dilan Kiley, Christopher M. Danforth, and Peter Sheridan Dodds. 2016. The emotional arcs of stories are dominated by six basic shapes. EPJ Data Science, 5(31):1-12.
Whitman Richards, Mark Alan Finlayson, and Patrick Henry Winston. 2009. Advancing computational models of narrative. Technical Report 63:2009, MIT Computer Science and Atrificial Intelligence Laboratory.
David E. Rumelhart. 1980. On evaluating story grammars. Cognitive Science, 4(3):313-316.
Evan Sandhaus. 2008. The New York Times Annotated Corpus. Linguistic Data Consortium, Philadelphia, 6(12).
Roger C. Schank and Robert P. Abelson. 1975. Scripts, plans, and knowledge. In Proceedings of the 4th International Joint Conference on Artificial Intelligence, pages 151-157, Tblisi, USSR.
Shashank Srivastava, Snigdha Chaturvedi, and Tom Mitchell. 2016. Inferring interpersonal relations in narrative summaries. In Proceedings of the 13th AAAI Conference on Artificial Intelligence, pages 2807-2813, Phoenix, Arizona. AAAI Press.
Kristin Thompson. 1999. Storytelling in the new Hollywood: Understanding classical narrative technique. Harvard University Press.
Tsvetomira Tsoneva, Mauro Barbieri, and Hans Weda. 2007. Automated summarization of narrative video on a semantic level. In International Conference on Semantic Computing (ICSC 2007), pages 169-176. IEEE.
Josep Valls-Vargas, J. Zhu, and Santiago Otonanon. 2014. Toward automatic role identification in unannotated folk tales. In Proceedings of the 10th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, pages 188-194.
Paul Vicol, Makarand Tapaswi, Lluis Castrejon, and Sanja Fidler. 2018. Moviegraphs: Towards understanding human-centric situations from videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8581-8590.
Christopher Vogler. 2007. Writer's Journey: Mythic Structure for Writers. Michael Wiese Productions.
Kurt Vonnegut. 1981. Palm Sunday. RosettaBooks LLC, New York.
Figure 4: Average composition of a CSI summary based on different crime-related aspects.
Hao Zheng and Mirella Lapata. 2019. Sentence centrality revisited for unsupervised summarization. arXiv preprint arXiv:1906.03508.
Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. Neural document summarization by jointly learning to score and select sentences. arXiv preprint arXiv:1807.02305.
A Narrative Structure Theory
The initial formulation of narrative structure was promoted by Aristotle, who defined the basic triangle-shaped plot structure, that has a beginning (protasis), middle (epitasis) and end (catastrophe) (Pavis, 1998). However, later theories argued that the structure of a play should be more complex (Brink, 2011) and hence, other schemes (Freytag, 1896) were proposed with fine-grained stages and events defining the progression of the plot. These events are considered as the precursor of turning points, defined by Thompson (1999) and used in modern variations of screenplay theory. Turning points are narrative moments from which the plot goes in a different direction. By definition these occur at the junctions of acts.
Currently, there are myriad schemes describing the narrative structure of films, which are often used as a practical guide for screenwriters (Cutting, 2016). One variation of these modern schemes is adopted by Papalampidi et al. (2019), who focus on the definition of turning points and demonstrate that such events indeed exist in films and can be automatically identified. According to the adopted scheme (Hauge, 2017), there are six stages (acts) in a film, namely the setup, the new situation, progress, complications and higher stakes, the final push and the aftermath, separated by the five turning points presented in Table 1.
Figure 5: F1 score  $(\%)$  for directed neural TEXT-RANK and SUMMER for unsupervised summarization with respect to different  $\lambda_{1}$  values. Higher  $\lambda_{1}$  values correspond to higher importance in the next context for the centrality computation of a current scene.
B CSI Corpus
As described in Section 4, we collected aspect-based summary labels for all episodes in the CSI corpus. In Figure 4 we illustrate the average composition of a summary based on the different aspects seen in a crime investigation (e.g., crime scene, victim, cause of death, perpetrator, evidence). Most of these aspects are covered in  $10 - 15\%$  of a summary, which corresponds to approximately two scenes in the episode. Only the "Evidence" aspect occupies a larger proportion of the summary  $(36.1\%)$  corresponding to five scenes. However, there exist scenes which cover multiple aspects (an as a result are annotated with more than one label) and episodes that do not include any scenes related to a specific aspect (e.g., if the murder was a suicide, there is no perpetrator).
We should note that Frermann et al. (2018) discriminate between different cases presented in the same episode in the original CSI dataset. Specifically, there are episodes in the dataset, where except for the primary crime investigation case, a second one is presented occupying a significantly smaller part of the episode. Although in the original dataset, there are annotations available indicating which scenes refer to each case, we assume no such knowledge treating the screenplay as a single unit — most TV series and movies contain substories. We also hypothesize that the latent identified TP events in SUMMER should relate to the primary case.
Figure 6: Examples of inferred TPs alongside with gold-standard aspect-based summary labels in CSI episodes at test time. The TP events are identified in the latent space for the supervised version of SUMMER  $(+\mathrm{P}, +\mathrm{R})$
In all unsupervised versions of TEXTRANK and SUMMER we used a threshold  $h$  equal to 0.2 for removing weak edges from the corresponding fully connected screenplay graphs. For the supervised version of SUMMER, where we use additional regularization terms in the loss function, we experimentally set the weights  $a$  and  $b$  for the different terms to 0.15 and 0.1, respectively.
We used the Adam algorithm (Kingma and Ba, 2014) for optimizing our networks. After experimentation, we chose an LSTM with 64 neurons for encoding the scenes in the screenplay and another identical one for contextualizing them. For the context interaction layer, the window  $l$  for computing the surrounding context of a screenplay scene was set to  $20\%$  of the screenplay length as proposed in Papalampidi et al. (2019). Finally, we also added a dropout of 0.2. For developing our models we used PyTorch (Paszke et al., 2017).
We illustrate in Figure 5 the performance (F1 score) of the directed neural TEXTRANK and SUMMER models in the unsupervised setting with respect to different  $\lambda_{1}$  values. Higher  $\lambda_{1}$  values correspond to higher importance for the succeeding scenes and respectively lower importance for
the preceding ones, since  $\lambda_{1}$  and  $\lambda_{2}$  are bounded  $(\lambda_{1} + \lambda_{2} = 1)$ .
We observe that performance increases when higher importance is attributed to screenplay scenes as the story moves on  $(\lambda_1 > 0.5)$ , whereas for extreme cases  $(\lambda_1 \to 1)$ , where only the later parts of the story are considered, performance drops. Overall, the same peak appears for both TEXTRANK and SUMMER when  $\lambda_1 \in [0.6, 0.7]$ , which means that slightly higher importance is attributed to the screenplay scenes that follow. Intuitively, initial scenes of an episode tend to have high similarity with all other scenes in the screenplay, and on their own are not very informative (e.g., the crime, victim, and suspects are introduced but the perpetrator is not yet known). As a result, the undirected version of TEXTRANK tends to favor the first part of the story and the resulting summary consists mainly of initial scenes. By adding extra importance to later scenes, we also encourage the selection of later events that might be surprising (and hence have lower similarity with other scenes) but more informative for the summary. Moreover, in SUMMER, where the weights change in a systematic manner based on narrative structure, we also observe that scenes appearing later in the screenplay are selected more often for inclusion in the summary.
As described in detail in Section 3.3, we also
infer the narrative structure of CSI episodes in the supervised version of SUMMER via latent TP representations. During experimentation (see Section 5), we found that these TPs are highly correlated with different aspects of a CSI summary. In Figure 6 we visualize examples of identified TPs on CSI episodes during test time alongside with gold-standard aspect-based summary annotations. Based on the examples, we empirically observe that different TPs tend to capture different types of information helpful for summarizing crime investigation stories (e.g., crime scene, victim, perpetrator, motive).