What’s New? Identifying the Unfolding of New Events in a Narrative (2023)

Abstract: Narratives include a rich source of events unfolding over time and context. Automatic understanding of these events provides a summarised comprehension of the narrative for further computation (such as reasoning). In this paper, we study the Information Status (IS) of the events and propose a novel challenging task: the automatic identification of new events in a narrative. We define an event as a triplet of subject, predicate, and object. The event is categorized as new with respect to the discourse context and whether it can be inferred through commonsense reasoning. We annotated a publicly available corpus of narratives with the new events at sentence level using human annotators. We present the annotation protocol and study the quality of the annotation and the difficulty of the task. We publish the annotated dataset, annotation materials, and machine learning baseline models for the task of new event extraction for narrative understanding.

## introduction the task of narrative understanding is a challenging topic of research and has been studied in numerous domains @xcite . recent studies include important applications of this task in supporting professionals in mental health. @xcite @xcite . automatic narrative understanding may provide a summarized comprehension of the users' recollections that can be used to engage in personal and grounded dialogues with the narrator. narrative understanding has been approached in different ways @xcite @xcite . a research direction in this field focuses on extracting the sequence of events that are mentioned in the narrative to obtain a summarized understanding of the whole narrative and its characters @xcite . in these works, the event is mostly represented by a predicate along with its corresponding subject and object dependencies. this definition relies on two assumptions a) the predicate represents an action/occurrence relation between the subject and the object dependencies; b) reoccurring characters across different events are the protagonists of the narrative. there have been interesting studies on different aspects of events in a narrative such as linking the correlated events as a chain @xcite , learning semantic roles of participants @xcite , commonsense inference @xcite , and temporal common-sense reasoning @xcite . in order to obtain a concise and salient understanding of the narrative through the events, it is necessary to identify and select the events that relate to a new happening/participant in the narrative and have novel contributions. the process of recognizing a new event implicitly involves the event coreference resolution task, which consists of detecting the mentions of the same event throughout the content @xcite . essentially, an event that is referring to a previous event is not considered new. nevertheless, even if an event appears in the narrative for the first time it might be part of commonsense knowledge, and thus not provide any new information. in this paper, we address the problem of identifying new events as they unfold in the narrative. this task is inspired and motivated by the need to a) extract salient information in the narrative and position them with respect to the rest of the discourse events and relations, and b) acquire new events from a sequence of sentential units of narratives. this task can facilitate higher levels of computation and interaction such as reasoning, summarization, and human-machine dialogue. last but not least, we believe this task is a novel and very challenging machine learning task to include in natural language understanding benchmarks. we assess whether an event is new in a narrative according to their information status (is) @xcite . is refers to whether a piece of information, which can be represented as an entity or other linguistic forms, is new or old. we consider an event new if it has not been previously observed in the context and provides novel information to the reader; that is, its information (the event and/or participants) is not presented priorly in the discourse stretch, and it can not be inferred through commonsense. for instance, bob saw alice is a new event if it is the first time that alice is introduced in the narrative or the first time bob saw her. however, once this event is selected as new, bob looked at alice will not be a new event anymore. furthermore, if bob married alice is considered as a new event, alice is bob's wife can be inferred through commonsense and thus is not a new event. an example of new and old events is presented in figure 1 . while there are eight events in the narrative sentences, two of them do not represent any novel information and thus are not new. for this purpose, we developed an unsupervised model to extract markable event candidates from the narratives. we parsed a publicly available dataset of narratives, send @xcite , and using the developed model, extracted all the markable events for each sentence. in the next step, we designed and conducted an annotation task using five human annotators to select the events in each sentence that are discourse-new with respect to the narrative context. in order to validate the annotation protocol and evaluate the results, we developed several neural and non-neural baselines for the task of new event extraction in both candidate-selection and sequence-tagging settings. the contributions of this paper can be summarized as follows: ## definition of new event we introduce the task of identifying the new events in a narrative to obtain a distilled and concise representation of the whole narrative and its characters. we follow the definition of an event that was used by @xcite based on the verb and its dependencies. that is, a verb is a core element of an event and supports the relation among its dependencies such as subject, object/oblique nominals which are considered as the participants of the event @xcite . @xcite defined the notion of old or new information status (is) with respect to two aspects of the hearer's beliefs and the discourse model. new information according to the hearer's belief is the one that is assumed not to be already known for the hearer, while discourse-new information is the one that has not been mentioned or has not occurred value #narratives (train:valid:test) 193 (114:40:39) #subject (# female) 49 (30) avg. narrative len. 28.10 utterances avg. utterance len. 15.44 tokens #vocabulary 4,416 unique tokens we extend the definition of the new category in entities @xcite to events. we define new events as those that are not mentioned in the narrative context and can not be inferred through commonsense by the reader. in this work, we do not consider further distinctions such as old or mediated. task design to reduce the annotators' workload, we developed a baseline model inspired by @xcite to automatically parse and extract all event candidates for each sentence in the narrative as the triplets of (subject, predicate, object). in the cases where more than 5 candidates were extracted for a sentence, we created 5 clusters using levenshtein distance (yujian and bo, 2007) (hierarchical clustering) and the candidate with the most number of tokens in each cluster was selected to be presented to the annotator. we randomly sampled 21 narratives from the send dataset and reserved them as backup data (13 narratives from the train set, 4 from the valid set, and 4 from the test set). using the extraction pipeline, we extracted all subject-predicate-object triplets as event candidates in the remaining 172 narratives at the sentence level. annotation ui the user interface (ui) of the annotation platform is presented in figure 2 . throughout the task, the annotator is presented with a brief version of the task guidelines on the top of the display (with access to the complete version). the narrative is presented on the left side of the screen with the current sentence in black and the context in grey. the narrative is updated progressively sentence-by-sentence while the annotator has access to the previous sentences of the context. for each sentence, the annotation question, the list of the triplet candidates and the possibility to select and add continuous span from the text are presented on the right side. annotation task during the task, the annotators were presented with a narrative one sentence at a time and the corresponding list of candidates. they were asked to control if any of the candidate triplets in the list is valid (i.e. it reflects the infor-mation in the sentence correctly); and whether it provides new information with respect to the previous narrative context, that can not be inferred through commonsense. in the case of valid and new information, the annotators were asked to select that candidate as a new event. furthermore, if there were no candidates extracted for a sentence or the new information in a sentence was not presented as a valid candidate, the annotator was asked to add the new information by simply copying the segment that conveys it from the sentence and adding it as continuous span text. ## baselines for new event detection we developed neural and non-neural baselines to validate the outcome of the annotation task, and, as baselines for the novel task of new event detection in a narrative. considering the two annotation formats of selecting candidates and adding continuous spans, we formalize the task using two settings of candidate selection and sequence tagging. ## conclusions in this work, we study the events in narratives according to their information status. we introduce the new task of identifying new events as they unfold in the narrative. in our definition of the event, the verb is the central element that represents a relation/happening that engages its dependencies such as subject, object, or oblique nominals. meanwhile, we define an event as new if it provides novel information to the reader with respect to the discourse (discourse-new) and if such information can not be inferred through commonsense. we annotated a complete dataset of personal narratives with new events at the sentence level using human annotators. we then developed several neural and non-neural baselines for the task of new event detection in both settings of candidate selection and sequence tagging. we share the annotated dataset and the base-lines with the community. we believe this task can be a novel and challenging task in narrative understanding and can facilitate and support other tasks in natural language understanding, human-machine dialogue, and natural language generation.