“Let Your Characters Tell Their Story”: A Dataset for Character-Centric Narrative Understanding (2021)

Abstract: When reading a literary piece, readers often make inferences about various characters’ roles, personalities, relationships, intents, actions, etc. While humans can readily draw upon their past experiences to build such a character-centric view of the narrative, understanding characters in narratives can be a challenging task for machines. To encourage research in this field of character-centric narrative understanding, we present LiSCU – a new dataset of literary pieces and their summaries paired with descriptions of characters that appear in them. We also introduce two new tasks on LiSCU: Character Identification and Character Description Generation. Our experiments with several pre-trained language models adapted for these tasks demonstrate that there is a need for better models of narrative comprehension.

## introduction previous works in literary analysis have discussed that the development of the plot and the main character(s) are among the most important components that contribute to a good piece of fiction @xcite . in particular, character(s) are central to narratives since their motivations, traits, and actions determine the flow of the plot. hence, understanding and critically analyzing characters is an important facet of literary scholarship. in computational narratives, prior work has exploited the potential of character-centric natural language understanding @xcite @xcite . however, these works are limited to only understanding certain aspects of characters and do not do an in-depth and systematic study. to facilitate character-centric narrative understanding, we present liscu -a new dataset in 1 data and code are available at: https://github.com/ fabrahman/char-centric-story the plantation's cajun work boss, beau baton, has been murdered just before the novel begins. candy marshall, the partial owner and general overseer of the plantation, discovers beau's dead body outside of mathu's house. candy believes that mathu killed beau, but as mathu is virtually her foster father she wants to make every effort to protect him. @xcite with numerous men and guns at the crime scene, candy believes that the local sheriff will not be able to solve the crime. miss merle spreads candy's plan through the community. within a few hours, eighteen men have gathered at mathu's house. @xcite the sheriff still believes that mathu murdered beau, but can do nothing so everyone just waits to see if fix bauton and a lynch mob will show up. beau bauton's brother, gil, is a star football player on the louisiana state university team who plays closely with a black halfback named cal. when gil learns of his brother's murder, he treats cal coldly. […] ## background the field of computational narrative understanding studies how to algorithmically represent, understand, and generate stories. early computational studies on narratives had focused on learning procedural scripts and event sequences @xcite @xcite , narrative chains or schemas @xcite , and plot units @xcite @xcite . computational linguists have also worked on character-centric modelling of narratives @xcite . the character-centric perspective aims to understand characters -their personas, roles, goals, relationships, emotions, etc. previous works have proposed methods to detect characters and infer latent personas in movie plot summaries and fictional novels @xcite @xcite , model inter-character relationships @xcite @xcite , and emotions @xcite . earlier works have also considered constructing social networks of characters @xcite from novels @xcite and films @xcite . another line of work related to ours is on summarization of novels @xcite . this work built a dataset of novel-summary pairs and used unsupervised summarization models such as textrank @xcite and @xcite . instead of summarizing full novels, @xcite proposed a contentselection approach to create a gold-standard set of extractive summaries by aligning chapter sentences with abstractive summary sentences. in a more related work, zhang et al. (2019) collected a dataset of fictional stories along with author-written summaries. they proposed an extractive ranking and a classification approach to select a subset of salient attributes from a list of candidate attributes (extracted from the story) that describe a character's personality. while this work presented a collection of personality-related phrases as a potential summary for the actual novel, our dataset contains literature summaries and char-acter descriptions, and we aim to generate natural language texts that analyze the narrative from the perspective of the characters. such an analysis is more in-depth than a collection of phrases. ## the liscu dataset we now describe our literature summary and character understanding (liscu) dataset. liscu is a dataset of literature summaries paired with descriptions of characters that appear in the summaries. fig. 1 shows an example of our dataset. next, we describe the data collection pipeline for liscu ( §3.1), followed by details on the reproducibility of the data collection process ( §3.2). ## liscu task definitions we introduce two new tasks on the liscu dataset: ## character identification we present two approaches to address this task: (1) solving it as a multiple-choice classification problem, and (2) using a generative classifier that generates, instead of identifying, the character name, as shown in fig. 3 . in the multiple-choice approach, we use the standard setup introduced in bert @xcite where the text from c i , d c * masked and s (with custom prefix tokens) are concatenated as input, and the [cls] token is projected to a final logit. we apply a softmax function to the logits to obtain the scores for each c i . for training practicalities, we limit the number of choices to 4 during training (using the earliest window of choices which include the correct one). during inference, we can generate the logits for all the answer choices since they are independent before the final softmax. to establish a baseline performance, we experiment with finetuning roberta @xcite , and albert @xcite which have been shown to perform well in several classification tasks. however, both these models cannot process inputs longer than 512 tokens and the concatenated inputs are generally much longer. so we also tried longformer @xcite , a bert-like model with an attention mechanism designed to scale linearly with sequence length, thus allowing the model to encode longer documents. however, despite trying various hyperparameters, longformer was not able to match the scores in our experiments. our second approach, a generative classifier, is inspired by @xcite who studied transfer learning by converting nlp problems into a text-to-text format. the generative classifier addresses the character identification problem by directly generating the character name ĉ, given all character names (answer choices), the masked character description, and the summary (see fig. 3 ). during inference, we compute the model's probability of each of the answer choices, and output the one with the highest probability. we use this procedure to train several strong baselines built on top of the following pre-trained transformer-based models: bart @xcite , t5 @xcite , and longformer @xcite . implementation details. the roberta and al-bert multiple-choice classifiers were trained for 6 epochs, initial learning rate 1e-5 (adam optimizer), batch size 16. the generative classifier using bart was trained for 5 epochs, initial learning rate 5e-6, batch size 8. we used the transformer package @xcite for training. the t5 model was trained for 12 epochs on a tpu using the default parameters from the t5 repository (learning rate 1e-3 with adafactor, batch size 8). @xcite we truncate the summaries (and descriptions) to satisfy model-specific maximum input length. ## character description generation we present several strong baselines for generating character descriptions by fine-tuning pre-trained transformer-based language models (lm) @xcite . we study two types of models: (1) a standard left-to-right lm, namely gpt2-l (radford et al., 2019) which is trained with lm objective to predict the next word; and (2) two encoder- experiment with full literary pieces. we also run an experiment on a subset of our data with accompanying full-text of the literary pieces. since it is infeasible to use the full texts as input given the memory constraints of current models, we coarsely select spans of the full-text beginning 50 tokens before, and 50 tokens after the occurrence of character's name. we use a longformer model where the input is simply the concatenation of the selected spans. due to the small size of the this subset, we perform a 5-fold cross validation starting from a pre-trained model fine-tuned on summary-description pairs. 14 implementation details. we use the transformer library @xcite . each baseline was trained for 5 epochs with effective batch size of 8, and initial learning rate of 5e-6. we use the maximum input length of 1024 for gpt2, and 2048 for bart 15 and the variant of longformer with truncated input. for experiment with original books, we use 16, 384 which is the maximum allowable input length for longformer. during inference, we use beam search decoding with 5 beams. ## conclusion understanding and critically analyzing fictional characters is an important element of understand-ing a literary piece. human readers build a mental model of characters, understand what they look like, their role in the literary piece, and assess their psychology, motivations, and consequences of their behavior. however, building such a deep understanding of fictional characters in narratives is hard for machine reading systems. to encourage progress in character-centric understanding of narratives, we present liscu, a dataset of literature summaries paired with descriptions of characters that appear in them. we use liscu to propose two tasks that explore the ability of the modern neural models to understand the narrative from the perspective of characters. performing human assessments on the model outputs show that there is still a lot of room for improvement on these tasks.