Continual Event Extraction with Semantic Confusion Rectification (2023)

Abstract: We study continual event extraction, which aims to extract incessantly emerging event information while avoiding forgetting. We observe that the semantic confusion on event types stems from the annotations of the same text being updated over time. The imbalance between event types even aggravates this issue. This paper proposes a novel continual event extraction model with semantic confusion rectification. We mark pseudo labels for each sentence to alleviate semantic confusion. We transfer pivotal knowledge between current and previous models to enhance the understanding of event types. Moreover, we encourage the model to focus on the semantics of long-tailed event types by leveraging other associated types. Experimental results show that our model outperforms state-of-the-art baselines and is proficient in imbalanced datasets.

## introduction event extraction @xcite aims to detect event types and identify their event arguments and roles from natural language text. given a sentence "the oklahoma city bombing conspirator is already serving a life term in federal prison", an event extraction model is expected to identify "bombing" and "serving", which are the event triggers of the "attack" and "sentence" types, respectively. also, the model should identify arguments and roles of corresponding event types such as "conspirator" and "oklahoma city" are two arguments involved in "attack" and as argument roles of "attacker" and "place", respectively. conventional studies @xcite @xcite @xcite model event extraction as a task to extract from the pre-defined event types and argument roles. in practice, new event types and argument roles emerge continually. we define a new problem called continual event extraction for this scenario. compared to conventional studies, continual event extraction expects the model not only to detect new types and identify corresponding event arguments and roles but also to remember the learned types and roles. this scenario belongs to continual learning @xcite , which learns from data streams with new emerging data. to alleviate the so-called catastrophic forgetting problem @xcite , existing works focus on event detection and apply knowledge transfer or prompt engineering @xcite @xcite . on one hand, they do not consider the task of argument extraction, making them incomplete in event extraction. on the other hand, they ignore that the semantic understanding by the model deviates from correct semantics when new types emerge, which we call semantic confusion. first, semantic confusion is caused by the annotations of previous types and new types do not generate at the same time. as shown in figure 1(a) , a sentence may have multi-type annotations. however, current training data only contains new annotations, and the model misunderstands the text "died" with the previous annotation "die" as a negative label "na". similarly, the model that is only trained on the previous data would identify the new types as negative labels. existing works @xcite @xcite simply transfer all learned knowledge to the current model, which would disturb new learning. the second problem is the imbalanced distribution of event types in natural language text. figure 1(b) shows the number distribution of event types in three widely used event extraction datasets. the model is confused with the semantics of the longtailed event types in two aspects. on one hand, it suffers from the lack of training on long-tailed event types due to their few instances. on the other hand, the semantics of long-tailed types would be disturbed by popular types during training. this paper proposes a novel continual event extraction method to rectify semantic confusion and address the imbalance issue. specifically, we propose a data augmentation strategy that marks pseudo labels of each sentence to avoid the disturbance of semantic confusion. we apply a pivotal knowledge distillation to further encourage the model to focus on vital knowledge during training at the feature and prediction levels. moreover, we propose prototype knowledge transfer, which leverages the semantics of other associated types to enrich the semantics of long-tailed types. our main contributions are outlined as follows: ## experiments and results in this section, we evaluate our model and report the results. the source code is accessible online. foot_0