Generative Approaches to Event Extraction: Survey and Outlook (2024)

Abstract: enter abstract here

## introduction event extraction is one of the core applications in natural language processing (nlp), aiming to create structured representations of events described in unstructured text. the task revolves around the identification and categorisation of predefined types of events within texts. this is typically broken down into identifying and categorising so-called event triggers and their respective arguments, along with their relevant properties and relationships, such as time, location, and participants. recently, generative language models have seen widespread uptake across many subfields of nlp, and event extraction is no exception. generative approaches to event extraction sometimes deviate from the traditional way of identifying and categorising events and their arguments, introducing new opportunities and challenges with respect to both training and evaluation. this paper provides an overview of the current landscape of generative approaches to event extraction by focusing on a representative set of techniques across different dimensions. we survey how the task of event extraction is approached across the range of decoder-only and encoder-decoder models with regards to generating the extracted event fields as natural language -as opposed to the traditional sequence labelling or boundary identification approaches. for readers seeking a broader overview of event extraction approaches, the surveys by @xcite , @xcite and @xcite can be explored. event extraction (ee) is traditionally approached as a sequence labelling problem. the annotations identify specific text spans that highlight event triggers with their associated arguments. this leads to the task being broken up into two parts as shown in figure 1: (1) event detection (ed) where event triggers are identified and the event is categorised into a type. an event trigger typically corresponds to the word(s) in the text that most clearly describes an event. in the example of figure 1 , "leave" evokes an "end-position"-type event. ## datasets in this section, we discuss the event datasets most commonly used across the different generative approaches we assess in section 3. while some of the datasets cover multiple languages, most primarily focus on english language sources. the highly influential automatic content extraction (ace) program released manual event annotations for text spans at the sentence-level, also including rich information about entities, temporal expressions, and relations between entities. the event annotation in the ace tradition has become a de facto standard for the evaluation of event extraction systems in the field of nlp. the 5 th iteration of the dataset, @xcite , consists of broadcast transcripts in addition to newswire and newspaper texts. it provides manual annotation for entities, relations, and events for joint evaluation of multiple information extraction tasks in multiple languages (arabic, chinese, and english) at the sentence level. the ace dataset is annotated for 8 general event types (e.g. life, conflict, transaction), along with 33 subtypes (e.g. conflict.attack), and 22 argument roles (e.g. attacker, agent, and recipient). the english version of the dataset comprises 599 documents. depending on the pre-processing approach, ace features two main variants, where ace covers only events with single-token triggers, and ace+ keeps all events with either single-or multi-token triggers. four f1 scores are usually reported on ace: the trigger identification, the trigger classification into an event type, the argument identification, and the argument classification into a role. the evaluation of ace and similar datasets is structured primarily for sequence labelling models. it typically involves comparing the predicted position offsets (specific locations of event mentions in the text) with the corresponding correct offsets. consequently, if a name appears multiple times within a sentence, only one of those occurrences is considered correct. 1 since generative models only extract and generate out-of-context surface forms without incorporating position offsets, evaluating them on datasets like ace may give these approaches an unfair advantage. the current best practice for generative approaches is to search for the text generated by the model in the input text, transform the output to offsets to simulate a sequence labelling model, and subsequently evaluate it as such. more recently, the ere annotation effort (entities, relations, and events, @xcite has contributed both data and annotation guidelines for event extraction purposes. the ere effort has evolved from the light ere to rich ere datasets, advancing from simple ace-based annotations to more complex handling of entities and events, ultimately enabling document-level event co-reference. the ere effort covers english, spanish and chinese documents from discussion forums, newswire, and proxy sources. the rich ere extends the annotation scheme of ace, covering 9 main event types and 38 event subtypes. in light ere, only asserted events are annotated (events that have occurred), with each event trigger linked to a single event. in contrast, rich ere allows for event triggers to be annotated for multiple events and includes annotations for event modality, capturing events that did not actually occur. another sentence-level event dataset is maven (massive event detection dataset, @xcite . it aims to alleviate problems of data scarcity and low coverage and contains 111 611 distinct events across 4480 human-annotated documents in total, corresponding to event-related articles from english wikipedia. it comprises 168 hierarchically organised event types derived from framenet @xcite , intended to cover generaldomain events. @xcite introduce a document-level annotated dataset based on english wikipedia articles and their referenced news articles called wiki-events. while only containing 246 documents with 8544 sentences, the dataset serves as an essential benchmark for event extraction systems beyond the sentence-level. each document is annotated with event types, event mentions (triggers and arguments), and co-references across sentences, even in sentences lacking an explicit event trigger. annotating co-references enables a fairer evaluation of generative models, as an extracted argument is considered correct if the model generates any coreference of the gold argument. in wikievents parlance, these are referred to as coref scores. the annotators also aimed to annotate the most informative event mention, giving precedence to name mentions over nominal mentions rather than focusing solely on the mention closest to the trigger word. this allows for another evaluation mode for wikievents termed informative argument extraction, where models are evaluated on their ability to extract the most informative argument mention. the annotations of the dataset resemble ace, but expand the number of sub-events from 33 to 67 following the kairos ontology. 2 additionally, wikievents has a more fine-grained event-type hierarchy. for instance, whereas ace identifies the event type and subtype such as conflict.attack, wikievents introduces event types at three levels, such as conflict.attack.detonateexplode. in recent years, the fourth message understanding conference (muc-4, @xcite dataset has resurfaced in research on documentlevel event extraction. the dataset is based on english newswire provided by the federal broadcast information services. it is annotated with the event types arson, attack, bombing, kidnapping, robbery, forced work stoppage, covering political conflicts in latin america. muc-4 contains 1700 documents, which may be associated with zero or more events of each type. moreover, the event type is associated with a template, each with the same set of 24 argument roles to be filled with either a numeric value, a categorical value, a text string, or a canonical form extracted or derived from the text. however, beyond event type classification, most recent works on the dataset are based on a simplified set of template slots restricted to five argument roles, where all have text string values that can be directly extracted from the source document @xcite . given that a large proportion of the documents are linked to empty templates, indicating the absence of relevant events, the ed task is important for muc-4. some of the models discussed in this survey employ additional datasets alongside those outlined above. rams @xcite ) covers 9124 events from news articles and is annotated in a 5sentence window around each event trigger. phee @xcite is a biomedical domain-specific dataset focused on drug safety, consisting of nearly 5000 sentences extracted from public medical case reports. finally, casie @xcite , consisting of 5000 news articles, with 1000 of these annotated on the sentence-level for cyber-attack events. ## models the majority of the models we survey follow a similar pattern as shown in figure 2 : the input text is fed to an encoder-decoder transformer that is fine-tuned to generate a representation of the events conveyed in said text. most of them can be divided into one of two groups according to how the events are represented in the generated text. to structure this survey, we first consider the representation of events in the output of the model; two main approaches exist: i) either an event is represented using a formal structure template in line with text2event @xcite , or (ii) the event is represented using a natural language template in line with bart-gen @xcite . we present these two distinct approaches in two separate sections. a classification of all models is also given in figure 3 . note that the organisation of this survey results from the fact that most of the models within the scope of this work build upon text2event and bart-gen. however, this structure does not necessarily reflect a deep fundamental difference between the two sections. the first model using a generative transformer to address ee falls partly outside this dichotomy. tanl @xcite introduces an ed model and an eae model using t5 @xcite . however, tanl does not focus solely on event extraction and can be trained on multiple information extraction (ie) tasks (named entity recognition, coreference resolution, etc). this is a recurring pattern in the papers we survey; different ie tasks are often similar enough that a single architecture can be reused. tanl goes one step further by simultaneously training on multiple tasks before extract the event information in the following text: he claimed iraqi troops had destroyed five tanks. ? 1 attacked ? 2 using ? 3 at ? 4 place instruction (optional) source text template (optional) encoder-decoder (e.g. bart, t5) ((attack destroyed) (attacker iraqi troops) (target five tanks)) he claimed iraqi troops had [destroyed|attack] five tanks. iraqi troops attacked five tanks using ? 3 at ? 4 place formal structure template augmented natural language natural language template input(s) output or figure 2: schema of a standard generative event extraction model. on the left-hand side are common features given to the model as input. some models rely only on the source text being present. a trigger word can be marked in the source text if the task being worked on is argument extraction. the inputs are generally given to an encoder-decoder model, which then generates a representation of the event. three examples of possible outputs are shown on the right-hand side. tanl uses augmented natural language, while models based on generating a formal structure or natural language templates are described respectively in sections 3.1 and 3.2 respectively. note that some models do not follow this general pattern (see for example, qga-ee). for example, if the word "destroyed" appeared twice in the given example, the model would be able to distinguish between the two and tag only the relevant one. ## summary and outlook this paper has surveyed the uptake of generative approaches to event extraction in nlp, presenting a range of different methods from encoder-decoders to decoder-only models. while some approaches take entire documents into account and others focus on the sentence-level, all evaluate performance based on matching predicted strings towards the strings found in the original input text. we argue that the field has yet to embrace generative approaches to ee fully. sticking to the traditional formulation of an "extraction" task makes it difficult to take full advantage of the capabilities of generative models like llms. the wide context windows of current llms also make them more suited for capturing more general or "complex events" -to use the words of @xcite -rather than the more granular and predicatecentered events typically targeted in the field so far. going forward, we hope to see new formulations of the task itself, focusing on more high-level event analysis or understanding. by moving away from span-based and sentence-level annotations to more abstract and document-level annotations, with an evaluation methodology that correspondingly focuses on semantics rather than string matching towards a source text, we believe that the field can have a version of event analysis that will be more useful for many downstream applications @xcite and more attuned to the strengths and possibilities of generative approaches and llms. in fact, the arguments for such a shift from an "extractive" to an "abstractive" view could also be made for many other ie tasks in nlp where both modelling and evaluation are traditionally tied to span-based text annotations.