Improving Cascade Decoding with Syntax-aware Aggregator and Contrastive Learning for Event Extraction (2023)

Abstract: “Cascade decoding framework has shown superior performance on event extraction tasks. How-ever, it treats a sentence as a sequence and neglects the potential benefits of the syntactic struc-ture of sentences. In this paper, we improve cascade decoding with a novel module and a self-supervised task. Specifically, we propose a syntax-aware aggregator module to model the syntaxof a sentence based on cascade decoding framework such that it captures event dependencies aswell as syntactic information. Moreover, we design a type discrimination task to learn better syn-tactic representations of different event types, which could further boost the performance of eventextraction. Experimental results on two widely used event extraction datasets demonstrate thatour method could improve the original cascade decoding framework by up to 2.2% percentagepoints of F1 score and outperform a number of competitive baseline methods. Introduction”

## introduction as an important yet challenging task in natural language processing, event extraction has attracted much attention for decades @xcite @xcite @xcite @xcite . this task aims at predicting event types, triggers and arguments from a given sentence. we display three examples in figure 1 . given an example sentence (a) "in 2018, chuangwei tech acquired equity of qianhong electronics for 1.5 billion ...", an event extraction system is able to recognize the trigger "acquired", that corresponds to the event type "invest", and the argument "chuangwei tech", that plays the subject role of "sub" in the event. a great number of methods have been developed for event extraction. early methods formulate the event extraction as a sequence labeling task, where each token is considered as a candidate for labeling. they perform trigger extraction and argument extraction with joint learning @xcite @xcite , which easily causes the label conflict issue. considering the precedence relationship between the components in an event, pipeline methods are explored to perform trigger and argument extraction in separate stages @xcite @xcite . but the error is accumulated along with the pipeline. recently, a cascade decoding framework @xcite is proposed to extract events with a cascade tagging strategy, which could not only handle the label conflict issue, but also avoid error propagation. in above methods, a sentence is treated as a sequence, and methods suffer from the low efficiency problem in capturing long-range dependency. we take sentence (a) in figure 1 as an example. the argument "1.5 billion" is far from the trigger "acquired" based on the sequential order while they are closely connected via the dependency arc. therefore, it is necessary to take advantage of the syntactic structure to capture the relations between triggers and arguments. some researches managed to include syntactic information of sentences in event extraction. chen et al. @xcite first employed dependency trees to conduct event extraction. @xcite and @xcite treated each dependency tree as a graph and adopted graph convolution network (gcn) @xcite to represent the sentence. more recent studies strengthened the graph representation via gate mechanism to filter out noisy syntactic information @xcite or empowered the graph encoder with more advanced transformer @xcite . these methods could effectively solve the long-range dependency issue. however, they either follow the joint learning paradigm or pipeline paradigm thus still encounter the issue of label conflict or error propagation. in this paper, we develop our approach modeling syntactic information for event extraction based on cascade decoding framework. to achieve this, two main challenges should be addressed. first, cascade decoding represents event types, triggers as well as arguments in the format of a triple. it sequentially predicts components in triples as subtasks and learns the implicit dependencies of the subtasks. it is not trivial to design a syntax encoder which is customized for the cascade decoders. in this paper, we propose a novel syntax-enhanced aggregator which could not only integrate the information from the precedent subtask with the current subtask but also model the syntactic structure of sentences. moreover, this module could fuse the heterogeneous features together. in detail, our aggregator processes both subtask dependency and syntactic information via two channels. the final representation will be fused based on the alignment between tokens of a sentence and components in a dependency tree. such aggregators are deployed in cascade decoders. second, existing methods involving syntactic structure rarely consider the interaction among event types. as examples (a) and (b) shown in figure 1 , the sentences of the same event type usually have similar syntactic structure despite different involved entities. in contrast, the sentences of the different event types usually have different syntactic structure despite similar involved entities, as examples (a) and (c). we design contrastive learning of syntactic representation to capture the interactions between sentences. specifically, we define a type discrimination task to distinguish whether two sentences belong to the same event type based on their syntactic representations. this is jointly trained with event extraction task. we conduct experiments on two event extraction datasets, fewfc @xcite and duee @xcite . the experiments show that compared with original cascade framework, our method can clearly perform better on both datasets. our method also outperforms competitive baseline methods that represent the state-of-the-art on event extraction tasks. to reveal the working mechanism of our method, we also conduct ablation study and visualization that shed light on where the improvement comes from. we summarize the contributions of this paper as follows: (1) we propose a novel syntax-enhanced aggregator to model the syntactic structure of sentences, which is a good fit for the cascade decoding framework. this aggregator is able to model syntax and fuse with dependencies of events. (2) to c c l 2 0 2 3 further benefit from the syntax modeling, we design a type discrimination task to refine the syntactic representation via contrastive learning. (3) we empirically show the effectiveness of our method on two datasets. our proposed method outperforms the baseline methods with remarkable margins based on f1 score of all measurement metrics. ## our approach the cascade decoding framework that we described in section 2.2 decodes different components of events in a cascading manner, the inputs of which are hidden representations of tokens featured with subtask dependencies. our approach follows the framework, but we improve it by introducing a module to fuse the syntactic information over the decoding process and a self-supervised task to further refine the syntactic representation. specifically, we propose syntax-enhanced aggregators to take place of the original aggregators. the proposed aggregator elaborately models the syntactic structure of the sentence and fuses syntax with the original hidden representation, as we will explain in section 3.1. to better capture the interactions among event types, we design a type discrimination task to distinguish whether the representations belonging to the same type are syntactically close or not, which will be presented in section 3.2. eventually, event detection and type discrimination generate their training objectives and we join them together, as we will describe in section 3.3. the overall architecture of our approach is displayed in figure 2 . ## experiments in this section, we conduct experiments to evaluate the proposed method. we first introduce our experiment settings including datasets and evaluation metrics, comparable methods, and implementation details in section 4.1, section 4.2, and section 4.3. next, we discuss our main results in section 4.4. we show further analysis in section 4.5 ## conclusions in this paper, we improved cascade decoding with syntax-aware aggregator and contrastive learning for event extraction. we demonstrated the effectiveness of our proposed method on two datasets. the results showed that our method outperforms all baseline methods based on f1 score. considering that many scenes have relatively high requirements for real-time performance, we will explore to optimize the computational complexity of the model and improving the universality of the model in the future.