Salience-Aware Event Chain Modeling for Narrative Understanding (2021)

Abstract: Storytelling, whether via fables, news reports, documentaries, or memoirs, can be thought of as the communication of interesting and related events that, taken together, form a concrete process. It is desirable to extract the event chains that represent such processes. However, this extraction remains a challenging problem. We posit that this is due to the nature of the texts from which chains are discovered. Natural language text interleaves a narrative of concrete, salient events with background information, contextualization, opinion, and other elements that are important for a variety of necessary discourse and pragmatics acts but are not part of the principal chain of events being communicated. We introduce methods for extracting this principal chain from natural language text, by filtering away non-salient events and supportive sentences. We demonstrate the effectiveness of our methods at isolating critical event chains by comparing their effect on downstream tasks. We show that by pre-training large language models on our extracted chains, we obtain improvements in two tasks that benefit from a clear understanding of event chains: narrative prediction and event-based temporal question answering. The demonstrated improvements and ablative studies confirm that our extraction method isolates critical event chains.

## introduction human languages always communicate about evolving events. hence, it is important for nlp systems to understand how events are procedurally described in text. in this context, identifying patterns of event chains is important but challenging, as it requires knowledge of inter-event relations such as temporal or causal relations. modeling high-quality event chain patterns from text would be a first step toward the more general goal of event schema induction, which involves generating high-level representations of event relations and structures. the extracted chain patterns can in turn represent useful information for core natural language tasks such as question answering @xcite , semantic role labeling @xcite , story generation @xcite , and reading comprehension @xcite . generally speaking, "events" correspond to what we perceive as happening around us. according to the theory of embodied cognition @xcite , our understanding of the world is shaped by various aspects of our entire bodies, involving also our language comprehension. however, it is currently difficult to gather enough data from other modalities to model real world "events," and written text, especially in the news domain, seems to be our best option. previous attempts have been made to generate event chains by modeling narratives in news, stories and documentaries @xcite @xcite . the problem with such data is that the articles are usually a mixture of the true narrative flow with other content, which serves to explain the context or provide side information. most prior approaches do not take into account the centrality or salience of events or the discourse structures that describe those events. accordingly, overlooking such important discourse properties when choosing the events of a sequence introduces noise in the modeling of event chains, and causes trivial or irrelevant content to be captured and inferred in narrative understanding tasks. in this work, we explore the use of salience identification @xcite and discourse profiling @xcite to help isolate the main event chains from other distracting events, and show the effect on two recent datasets related to narrative understanding and temporal understanding. more specifically, we obtain event chains from documents and perform different methods of filtering, then build event language models, which we use to predict story ending events from the rocstories dataset @xcite and answer temporal event questions from the torque dataset @xcite . by comparing the use of event language models built on differently filtered event sources, we show that filtering out distracting narrative information can indeed benefit the modeling of event relations, thus leading to more reliable extraction of useful event chain patterns. the main contributions in this work are threefold. (1) to support narrative understanding tasks with high-quality event chains, we develop a new event chain extraction method that is discourse-aware, and particularly, salience-aware. ## event chains and narratives information extraction techniques have evolved to extract event mentions as well as their ordering information @xcite @xcite , hence enabling the automated induction of raw event chains from text. tools such as tear @xcite have been developed, which we can use to extract both the events and temporal ordering from text documents, like those shown in figure 1 . these sequences of events, connected in a temporal order, form linear event chains, which can be viewed as a form of linear representation of the progress of described events in a narrative. however, events that co-occur or are described together in an article are often not equally important. some events are salient -they are relevant to the main topic of a text context, or play essential roles with regard to the central goal of an event chain @xcite . for example, from the paragraph in figure 1 , the events detained, alerted and questioned are salient components that constitute the progress of a described story. other events may describe how the salient events came to be known or involve other events that happened alongside salient events but are not critical to understanding the core actions of the story. in figure 1 , said and establish are not salient. the goal of event salience detection systems @xcite is to identify events that are essential components of the narrative, which can help us filter out trivial or distracting event mentions from event chains. from another related perspective, modeling discourse structures of the article is also helpful for understanding which parts of the text directly report on main events and which parts serve other supportive roles. discourse profiling @xcite seeks to analyze the functional role of each sentence, which is different from the event-level prediction of salience. van dijk (1988)'s theory of news discourse outlines an ontology of sentence types that are seen in most news articles. according to the theory, the first sentence in figure 1 is a main event sentence, while the second is an evaluation sentence. we may classify the members of van dijk's ontology into types that are core to understanding the event sequence of a story (e.g. main event) and types that are not (e.g. evaluation), and further refine extracted event sequences. though salience and discourse structures represent different perspectives of narrative analysis, they can be concurrently modeled in an event chain extraction system, leading to more effective filtering of mined event chain representations. ## method to obtain the interesting event chains from a document, we first use the tear tool by @xcite to generate a temporal relation graph. then we apply different levels of filtering on the extracted events, which correspond to the nodes in the event graph structure. finally, we extract linear chains of events from the filtered graph structure by following along the directed edges. in our evaluation, we will use these linear chains instead of raw text to pre-train language models such as roberta @xcite . our overall pipeline is shown in figure 2 . details of each step in the pipeline are described as follows. ## experiments it is difficult to directly evaluate the quality of event chains in an intrinsic way. some works on event schemas ask human annotators to rate qualities of generated chains @xcite , which can be subjective. we instead turn to extrinsic evaluation tasks that depend on implicitly understanding typical sequences of meaningful events in order to be completed usefully. torque @xcite , short for temporal ordering questions, is a machine reading comprehension benchmark that requests a model to answer questions regarding temporally specified events (e.g., "what happened before the snow started?") in a reference article. we hypothesize that models trained on unfiltered event chains are less likely to focus on the relevant events requested by the question, but this seeks to be improved by our filtered event chains. rocstories @xcite is a narrative prediction dataset consisting of five-sentence short stories, where each sentence of a story contains a core event. a test set included with roc-stories contains two candidate endings to each partially complete story, where one of them is plausible. while prior work has successfully leveraged event chains to infer the endings @xcite , we believe a model trained on relevant chains of events should be able to better distinguish the relevant ending from the irrelevant one than a model trained on event chains that contain irrelevant events. other works such as @xcite have directly tried to maximize performance on this corpus; we don't seek to directly compete with that work here, but rather use rocstories as a means of establishing that our isolation of important events at the event language model level does positively influence event ending prediction, indicating that the sequences we find do indeed contain more relevant events. in this section we verify the value of our eventfiltering model on these tasks. we also evaluate our proposed discourse parsing model to show the usefulness of combining salience information at multiple levels. ## conclusion we propose an event chain extraction pipeline, which leverages both salience identification and discourse profiling to filter out distracting events. we demonstrate the effectiveness of the approach by using the produced event chains to train/fine-tune language models, which leads to improved performance on temporal understanding of events and narrative prediction. our case study on salienceaware discourse parsing shows the advantage of combining event-level and sentence-level salience information. we plan to use these event chain patterns on other narrative understanding and generation tasks, such as constrained story generation @xcite , event script generation @xcite , and implicit event prediction @xcite .