When and Where Did it Happen? An Encoder-Decoder Model to Identify Scenario Context (2024)

Abstract: We introduce a neural architecture finetuned for the task of scenario context generation: The relevant location and time of an event or entity mentioned in text. Contextualizing information extraction helps to scope the validity of automated finings when aggregating them as knowledge graphs. Our approach uses a high-quality curated dataset of time and location annotations in a corpus of epidemiology papers to train an encoder-decoder architecture. We also explored the use of data augmentation techniques during training. Our findings suggest that a relatively small fine-tuned encoder-decoder model performs better than out-of-the-box LLMs and semantic role labeling parsers to accurate predict the relevant scenario information of a particular entity or event.

## introduction we present an approach to contextualizing information extraction (ie) that focuses on enhancing events and entities with scenario context: the location and time relevant to extracted elements. knowing when and where an event occurs has become increasingly relevant due to the wide adoption of large-scale machine reading technology. decision makers in high-stakes areas, 1 like epidemiology, public health or climate sciences, are increasingly turning to natural language processing (nlp) technologies to help guide their decision making through automatic evidence discovery and aggregation. in light of this, properly scoping automated ie becomes very valuable to the users of these tools. one example of a domain when scenario context information is relevant is the modeling of epidemic dynamics, where the literature describes different outbreaks using different mathematical models, such as variations of the susceptible-infectedrecovered (sir) compartmental model. the dif-1 https://www.darpa.mil/program/ automating-scientific-knowledge-extraction-and-modeling ferent scenarios have different parameters and it is useful to contextualize the relevant event to have a better picture of the scenario described. another example is the domain of climate and climate change, where changes in the climate of different geographical regions over time is studied by the geosciences community. scenario information is often explicitly found in the periphery of the text describing an extraction, but not necessarily in the same sentence-thus, it is a form of inter-sentence relation extraction (see figure 1 for examples). in this work, we tackle the problem as a generative task using an encoder-decoder transformer based on t5 @xcite . given the locations and temporal phrases in an input passage, we prompt the model to chose and generate the relevant scenario information with respect to a specific entity or event. the main contributions of this work are the following: all artifacts used to train and evaluate the model are publicly available. foot_0 ## related work annotating when and where an event occurs is closely related to semantic role labeling (srl) @xcite temporal annotations (b) location annotations figure 1 : example annotations in our dataset. predicates highlighted in yellow represent 'events' with scenario context information assigned to them, text highlighted with cyan represents temporal context, and text in green location context. the arrows connect a context expression to an event they are associated with. the scenario context to event associations are effectively many-to-many relations. figure 1a shows a passage with several temporal scenario contexts and figure 1b shows several location scenario contexts. @xcite @xcite ). an important distinction, however, is that srl operates on parsing the structure of a sentence and assign roles to phrases within it. our proposed approach is generative in nature, and is not constrained to the structural elements identified during parsing. ## dataset in order to train our model, we created a dataset that contains location and temporal context annotations at both intra-and inter-sentential levels. we focused on 22 epidemiology research articles, including ones that involve modeling the dynamics of infection and outbreak case-studies, published between 2020-2022. these articles often describe parallel scenarios to compare and contrast the behavior of different outbreaks, making the inference of the scenario context of relevant concepts and events in these papers non-trivial. correctly understanding the location and time period for specific events is important for the accuracy of any inference drawn from these studies. we excluded any temporal mentions that were abstract or relative, and any location mentions that were modifiers or adjectives. figure 1 shows an example of an annotated passage of each kind. the dataset comprises 383 passages, ranging in length from a single sentence to a couple of para-graphs. a total of 1,382 relations were annotated with 833 (60.3%) location context relations and 549 (39.7%) temporal context relations. a key aspect of the dataset is the presence of annotations for inter-sentential relations-i.e., relations where the event/concept of interest and its scenario context are in different sentences. these comprise 18% of scenario context relations. approaches that rely on syntax, e.g., srl, are not well suited for inter-sentential relations. figure 2 shows the distribution of sentence distances for the inter-sentential subset of annotations. a random sample consisting of 13% of the relations was used to measure inter-annotator agreement using the cohen's kappa method @xcite , resulting in a score of 𝜅 = 0.79. 3 . details about the annotation guidelines are found in appendix a ## experiments and results we trained foot_1 an encoder-decoder model @xcite based on t5 to generate the location and temporal information relevant to a specific event from its surrounding context. for each relation in the dataset, we prompted (see appendix b for details) the model to decode the context information of the specific event. each event may have zero, one or more context relations of each type. the model decoded all of them simultaneously. we held out a random sample of 20% of the annotions for testing and fine-tuning t5-base. baselines we compare our methods with a decoder-based llm approach and an srl system. ## conclusions and future work in this work, we introduced an encoder-decoder model finetuned to generate location and temporal context associated with a particular concept or event. we are releasing a dataset of hand-curated annotations from a collection of academic papers in the epidemiology domain that describe the dynamics of outbreaks in different locations and times. we found that our method more accurately recognizes the relevant context than out-of-the-box llms or srl. we also explored the use of data augmentation methods, finding that they resulted in modest improvements in temporal context extraction. there are at least two promising avenues for future work. the first is expanding the dataset to include more curated annotations from additional domains. this will foster the development of more accurate models with better generalization capabilities. the second is exploring other network architectures, such as span-prediction or decoderonly models. the former is useful for attributing the source of the context prediction and the latter can benefit from the transfer learning potential exhibited by open-source llms.