COFFEE : A Contrastive Oracle-Free Framework for Event Extraction (2023)

Abstract: Event extraction is a complex task that involves extracting events from unstructured text. Prior classification-based methods require comprehensive entity annotations for joint training, while newer generation-based methods rely on heuristic templates containing oracle information such as event type, which is often unavailable in real-world scenarios. In this study, we consider a more realistic task setting, namely the Oracle-Free Event Extraction (OFEE) task, where only the input context is given, without any oracle information including event type, event ontology, or trigger word. To address this task, we propose a new framework, COFFEE. This framework extracts events solely based on the document context, without referring to any oracle information. In particular, COFFEE introduces a contrastive selection model to refine the generated triggers and handle multi-event instances. Our proposed COFFEE outperforms state-of-the-art approaches in the oracle-free setting of the event extraction task, as evaluated on two public variants of the ACE05 benchmark. The code used in our study has been made publicly available.

## introduction the event extraction task aims to identify events and their arguments from the given textual input context @xcite @xcite . conventionally, this task can be decomposed into four sub-tasks @xcite : (i) detecting the trigger word that most directly describes the event; (ii) event type classification for defining its event-specific attributes; (iii) argument identification and (iv) argument classification that maps the argument entities to the corresponding role attributes based on the structure of each event type, namely event schema. for instance, figure 1 shows the input context of an event extraction example that contains two events: figure 1: an event extraction example with two events: transport and attack. in the 'transport' event, 'went' is the trigger word, and 'home' is the 'destination' argument. in the 'attack' event, 'killed' is the trigger word while 'father-in-law' and 'home' are the 'agent' and 'place' arguments, respectively. a 'transport' event triggered by the trigger word 'went' and an 'attack' event triggered by the trigger word 'killed', where 'transport' and 'attack' are two event types. many prior studies formulate the event extraction task as a token-level classification problem, which extracts event triggers and arguments using sequence tagging models based on tailor-designed neural networks @xcite @xcite @xcite @xcite . however, such methods cannot leverage rich label semantics since the target outputs (e.g., event triggers and arguments) are fixed tagging labels. recently, with advances in generative pre-trained language models, several generation-based approaches @xcite @xcite have been applied to solve this problem. these approaches transform the event extraction task into a conditional generation task. by utilizing the autoregressive generation nature of generative pre-trained language models (e.g., bart-gen @xcite , degree @xcite ) and some manual prompts, it becomes possible to harness the semantics of labels and conduct both entity extraction and classification in an autoregressive manner simultaneously. while impressive results are reported, we identify two major limitations of the current generationbased event extraction methods. firstly, most of these methods rely on heuristic templates and extensive human knowledge engineering. according to the experiments conducted by @xcite , a slight change in the template might lead to significant performance changes, thus raising the issue of using sub-optimal templates. secondly, most of these generation-based approaches still require certain oracle information, such as event type and event schema, which necessitate extensive manual annotations. for example, the degree model's inference process, as demonstrated by @xcite , requires manually designed event-specific templates for each example and iterates over all event types. on the other hand, text2event @xcite also constrains the generation with manually designed templates, which require event schema to be given. however, obtaining this oracle information, such as event type and schema, is unrealistic for a real-world inference system to achieve automatically. hence, this paper aims to address the oracle-free event extraction (ofee) task where only the input context is given. in this study, we propose a novel contrastive oracle-free framework for event extraction (coffee), which addresses the event extraction task without using any oracle information. our coffee consists of two parts, a generator that performs the extraction of events and a selector that aims to refine the generated results. the generator of our coffee generates both the candidate triggers and event arguments, where the shared generator allows for cross-task knowledge sharing between these sub-tasks. the selector of our coffee learns to re-rank and select the candidate triggers to obtain more accurate trigger predictions, which is inspired by @xcite . one challenge of the sentence-level event extraction is that a sentence may contain more than one event record @xcite (e.g., the example in figure 1 ), and event specific templates can help the model to identify and extract events in a targeted manner. prior approaches tackling this challenge have necessitated either multilabel tagging @xcite , event-specific templates @xcite , or multi-turn question answering techniques @xcite . in contrast, our proposed model can concurrently generate and se-lect multiple event candidates, encompassing both the event trigger and its associated type, thereby effectively addressing the aforementioned challenge. the contribution of this work is as follows: ## task definition conventionally, the event extraction task entails the following terminologies @xcite @xcite . given the input context c, which is a sequence of tokens [c 1 , • • • , c n ], the conventional event extraction task aims to identify the trigger words, classify the events triggered by these words and extract the arguments in each of the events with their corresponding roles @xcite . assume that an input sentence context c contains |e| different events, then its ground truth triggers y t can be represented as [y t 1 , • • • , y t |e| ], where y t i denotes the i-th trigger word and event type of the given context sentence. for each event, there is a list of ground truth arguments, denoted by y a , which is a list of ⟨role, argument⟩ pairs, i.e., y @xmath0, a 1 ⟩, • • • , ⟨r m , a m ⟩], where a j is the j-th entity participating in the event and r j is the corresponding role type for that entity. for this conventional event extraction task, the current state-of-the-art generation-based approaches rely on manual templates, which require trigger words or event types to be given, to simplify this task @xcite . however, in a realistic scenario, although argument roles are event-specific, gold trigger words or event type information may not be readily available during event argument extraction we focus on the oracle-free event extraction (ofee) task, which presents a more practical scenario by only providing the input context during inference. the goal of ofee is to infer event triggers and arguments without relying on pre-defined event specific templates, making it more challenging to solve due to the absence of external guidance or oracle information. ## methodology as mentioned in the task definition, our goal is to extract event frames without using any templates. this adds complexity to the generation model, particularly when dealing with contexts containing multiple events, such as the example given in fig- ## conclusion in this work, we study a more realistic setting of the event extraction task, namely the oracle-free event extraction, where no additional information beyond the context is required for event inference. to address this task, we propose a generationbased event extraction framework called coffee. our coffee introduces a contrastive selector to improve trigger extraction performance by reranking and automatically determining the number of triggers to be selected in a given context. additionally, we investigate the dependence of current generation-based models on extra knowledge, such as designed event-specific templates, event trigger keywords, and event descriptions. our results show that this reliance on templates and human-designed trigger sets is unnecessary, and a pure oracle-free model applied directly can perform very well on general event extraction. in the future, we plan to extend sentence-level event extraction to documentlevel and explore zero-shot settings to handle the emergence of unseen events.