CEAN : Contrastive Event Aggregation Network with LLM -based Augmentation for Event Extraction (2024)

Abstract: Event Extraction is a crucial yet arduous task in natural language processing (NLP), as its performance is significantly hindered by laborious data annotation. Given this challenge, recent research has predominantly focused on two approaches: pretraining task-oriented models for event extraction and employing data augmentation techniques. These methods involve integrating external knowledge, semantic structures, or artificially generated samples using large language models (LLMs). However, their performances can be compromised due to two fundamental issues. Firstly, the alignment between the introduced knowledge and event extraction knowledge is crucial. Secondly, the introduction of data noise during the augmentation is unavoidable and can mislead the model’s convergence. To address these issues, we propose a Contrastive Event Aggregation Network with LLM-based Augmentation to promote low-resource learning and reduce data noise for event extraction. Different from the existing methods introducing linguistic knowledge into data augmentation, an event aggregation network is established to introduce event knowledge into supervised learning by constructing adaptively-updated semantic representation for trigger and argument. For LLM-based augmentation, we design a new scheme including a multi-pattern rephrasing paradigm and a data-free composing paradigm. Instead of directly using augmentation samples in the supervised task, we introduce span-level contrastive learning to reduce data noise. Experiments on the ACE2005 and ERE-EN demonstrate that our proposed approach achieves new state-of-the-art results on both of the two datasets.

## introduction event extraction, as a fundamental task of information extraction, aims at acquiring structured information about periodical incidents from plain * corresponding author text @xcite @xcite . as shown in figure 1a , it is usually accomplished by trigger extraction and argument extraction. most of the methods are based on supervised learning, which require adequate high-quality labeled data. however, data annotation is usually arduously expensive, which means only insufficient data can be used to train a supervised model. in such low-resource scenarios, event extraction models often suffer from poor performances, especially when facing the hard samples with rare patterns. to alleviate data sparsity, some methods leverage large-scale unsupervised data with pretraining tasks such as semantic structure analysis @xcite . for such methods, a crucial and challenging issue is the alignment between rich knowledge lying in unsupervised data and event knowledge. alternatively, data augmentation methods are proposed by rephrasing the eventrelated fragments and adjunct fragments, which are lexically different but semantically consistent. as shown in figure 1b , token-level augmentations are based on linguistic knowledge such as synonym replacement, which results in limited diversity improvement. sentence-level augmentations use text generation techniques such as back translation to produce more diverse samples. with recent success on text generation, llms can be introduced for more flexible and diverse augmentation. meanwhile, data noise is unavoidably expanded. subtle lexical changes in token-level possibly result in event structure deviations. new events can be unexpectedly introduced by sentence-level augmentation without annotated. in supervised learning, model's convergence can be misled by data noise in the training data. however, few existing studies of event extraction focus on the data noise alleviation. this paper proposes a contrastive event aggregation network with llm-based augmentation (cean). firstly, we establish an event aggregation network with a knowledge bank by constructcleaned and noisy augmentation are marked in green and purple. by replacing the "to" with "from", the role of "singapore" deviates from "destination" to "origin". by using "retired" in the rephrasing, a new event of "end-postion" is unexpectedly introduced. ## related work traditional extraction studies are based on elaborate feature engineering @xcite , which are replaced by deep learning-based methods with convolutional neural network @xcite , recurrent neural network @xcite , transformer @xcite @xcite @xcite @xcite etc. according to the procedure of trigger and argument extraction, existing studies can be classified into pipeline-based and joint-based methods, which arrange the two extraction subtasks either in a serial or parallel paradigm, respectively @xcite . mostly, the extraction task can be solved in classification manner or generative manner @xcite . for classification manner, the typical studies employ transformer-based encoder with modules such as feed forward network @xcite or global pointer @xcite @xcite to conduct token classification or sequence labeling. for generative manner, many approachs with transformer decoder architecture are recently proposed to solve the extraction tasks by machine reading comprehension or sequence-to-structure generation @xcite @xcite . with the surge in popularity of llms especially chatgpt @xcite , some research has attempted to conduct zero-shot or few-shot event extraction by harnessing the powerful capabilities of llms, only achieving unsatisfactory performances @xcite @xcite . to tackle the data sparsity, unsupervised pretraining on tasks such as abstract meaning representation are explored @xcite . additionally, data augmentations are conducted by external knowledge introduction @xcite @xcite , mask token prediction @xcite , backtranslation @xcite , blank infilling @xcite etc. recenet studies prove that tokenlevel augmentation can only bring poor diversity improvement @xcite , and possibly results in event information deviation @xcite . sentence-level augmentation can better improve diversity, but also unavoidably bring data noise @xcite . recent studies introduce llms in sentence-level augmentation and receive performances improvement @xcite @xcite . retrievalbased methods are also proposed for low-resource learning of information extraction @xcite . ## contrastive event aggregation network this section describes our proposed model cean for closed-domain event extraction. detailed architecture is introduced as two parts, event aggregation network and span-level contrastive learning. the proposed approach is in pipeline-based paradigm, with only minor differences in model architecture and training methods for trigger and argument extraction. therefore, we use the notations with a superscript of * to represent the layers, tensors or functions that occur in both trigger and argument extraction. the notations marked with a superscript of t or a are specifically used for trigger extraction or argument extraction, respectively. data-related notations are summarized in ## llm-based data augmentation this section describes llm-based data augmentation shown in figure 3 . the paraphrase patterns and the prompt engineering for two paradigms are introduced, followed by evaluation metric. ## conclusion we propose a contrastive event aggregation network with llm-based augmentation to promote low-resource learning and reduce data noise for event extraction. cean introduces event knowledge into supervised learning by establishing knowledge bank for triggers and arguments. we design an llm-based augmentation scheme including a multi-pattern rephrasing paradigm and a datafree composing paradigm to improve lexical diversity. we introduce span-level contrastive learning to reduce data noise unavoidably originated in the augmentation. experiments on the ace2005 and ere-en datasets demonstrate that our proposed approach achieves new state-of-the-art results.