CAN ar E x: Contextually Aware Narrative Extraction for Semantically Rich Text-as-data Applications (2022)

Abstract: Narrative modelling is an area of active research, motivated by the acknowledgement of narratives as drivers of societal decision making. These research efforts conceptualize narratives as connected entity chains, and modeling typically focuses on the identification of entities and their connections within a text. An emerging approach to narrative modelling is the use of semantic role labeling (SRL) to extract Entity-Verb-Entity (E-V-Es) tuples from a text, followed by dimensionality reduction to reduce the space of entities and connections separately. This process penalises the semantic richness of narratives and discards much contextual information along the way. Here, we propose an alternate narrative extraction approach - CANarEx, incorporating a pipeline of common contextual constructs through co-reference resolution, micro-narrative generation and clustering of these narratives through sentence embeddings. We evaluate our approach through testing the recovery of “narrative time-series clusters”, mimicking a desirable text-as-data task. The evaluation framework leverages synthetic data generated using a GPT-3 model. The GPT-3 model is trained to generate similar sentences using a large dataset of news articles. The synthetic data maps to three topics in the news dataset. We then generate narrative time-series document cluster representations by mapping the synthetic data to three distinct signals synthetically injected into the testing corpus. Evaluation results demonstrate the superior ability of CANarEx to recover narrative time-series through reduced MSE and improved precision/recall relative to existing methods. The validity is further reinforced through ablation studies and qualitative analysis.

## introduction narratives are accounts of connected events, and are a basic mechanism of communication and of understanding our world @xcite . "narrative modelling" is the discovery of underlying narratives (such as "politician is liar" or "inflation is high"), as well as the analysis of how their shape and prominence shifts over time @xcite . narrative modelling offers the potential to go beyond simple issue tagging or topic modelling to get at the richer question of how ideas are connected and framed, and when these relationships form, arise, and change. in the realm of the social sciences, quantitative and causal analysis relies on structured, tabular data, with researchers recognising the potential for 'text as data' to greatly expand the realm of analysis into news, opinion, and political discourse @xcite . in particular, if semantically related narratives can be reliably identified in a large corpus over time using narrative modelling, then established time-series techniques in the social sciences can be leveraged in a wide array of applications @xcite . recent work on narrative extraction such as @xcite demonstrates the potential of such approaches to discourse analysis, when based on more traditional nlp techniques. the advances in transformer-based language models such as bert @xcite and gpt-3 @xcite present the opportunity to vastly increase the power of such methods, incorporating a greater understanding of "context" into the model. context is especially important to narratives since they are fundamentally composed of sequences of coupled language components. we demonstrate and evaluate, canarex, a contextually-aware narrative modelling approach which employs state-of-the-art transformer models to unlock the potential of the context-sensitivity of these models. we show the superiority of canarex's contextually-aware narrative modeling technique to (already powerful) existing methods in the literature. we discuss how these models can be used to conduct previously infeasible analysis of the shifts in narratives through an illustration in the social sciences by understanding social discourse over time. we employ generative text models tuned on narratives extracted from original news, opinion and parliamentary speeches to develop a highly realistic narrative time-series recovery evaluation task. ## related work automated extraction of narratives and other artifacts for text as data is an area of ongoing research @xcite , spanning from classical nlp techniques to the new transformer architectures. given the typical multi-entity, discursive nature of most narratives, identifying entities and interactions between entities are common building blocks of narrative modelling approaches. these interactions can be framed as semantic triples extracted using a semantic parser and can take forms such as {subject-predicate-object}, {entity-verb-entity}, with the key idea being that the subject and object of the triple represent entities and the verb or predicate of triplet represents the interaction (or event) between them @xcite @xcite . these e-v-es provide a succinct summary of the underlying events, actors addressing "who does what to whom" @xcite and thus provide a framework that enables further downstream tasks. in terms of the "minimal model of narrativity" @xcite , the triples can be mapped to the features 'agents' and 'events'. @xcite propose a narrative modelling framework with a particular focus on identifying economic activities (debates, conflicts, compromise) around social processes such as industrial regeneration. towards this end, their work prioritises identifying the points of view of each entity called attribution (beliefs, thoughts, speeches) in a given narrative alongside identifying the entities and events involved. their pipeline includes manually annotated custom event labelling, pre-trained entity recognition(ner) @xcite models and a semantic role labelling (srl) ensemble comprising of semafor @xcite and deepsrl @xcite for event and attribute extractions of the narratives. the results from the semantic frames (predicate with entities) is split into entities, extracted with the ner, and events through mapping to the custom event labels which has the event type attribution. through this process, their framework is able to attribute the statements and intents within a narrative to specific actors. the 'relatio' framework takes a more generalized approach to narrative modelling, seeking to extract narratives from any text data, and without extensive custom labelling. 'relatio' @xcite framework enables identifying the opinions, inflection points and changing trends from public discourse through narrative extraction from news. they use the {entity-verb-entity} narrative structure, or the e-v-es using srl @xcite to identify the entities in the discourse. they have an additional step to convert the extracted high dimensional entities to lower dimensional entities using clustering of word vectors @xcite . the workflow is presented in figure 1 . the separation of the entities from their verbs whilst clustering results in loss of contextual information. as seen from these examples, modelling the multi-entity, discursive nature of narratives typically involves identifying entities and interactions between these entities, with the rest of the pipeline reflecting the specific goals of the model. the implementations of these different tasks of the pipeline have evolved to reflect the new transformer architectures @xcite and associated improvements. wankmuller @xcite provided a review of the possibilities of transfer learning with transformers in the domain of social science studies, highlighting the higher prediction power and increased efficiencies of these pretrained language models (plms) compared to the historically common supervised learning models. the canarex framework builds on the work of relatio by incorporating more features to support complex narratives, and by updating the existing components to leverage transformer based models. ## canarex here we propose the 'contextual narrative extraction' (canarex) framework that enables contextual narrative extraction. the framework is represented in figure 3 . ## evaluation we evaluate the performance of canarex by testing the recovery of the time-series narrative across the corpus. to do this, we rerun our pipeline on time-series clusters of synthetic data and compare the results with the output of relatio, our baseline model. ## conclusion in this work we introduced the canarex framework for contextually aware narrative extraction for text-as-data applications. the contextual aspects of the narrative are retained via the co-reference resolution step and by the use of the complete e-v-es triplets for clustering. we also present an optional step that enables filtering of the extracted micro-narratives. we demonstrate the framework on two datasets -parliamentary proceedings and news articles. further, we generate synthetic test data using a novel gpt-3 based technique for evaluation of the framework, and demonstrate better performance than the baseline model. the results are reinforced through an ablation study exercising the various components of the pipeline, and a qualitative study demonstrating the applicability of the framework.