A Novel Joint Framework for Multiple C hinese Events Extraction (2020)

Abstract: Event extraction is an essential yet challenging task in information extraction. Previous approaches have paid little attention to the problem of roles overlap which is a common phenomenon in practice. To solve this problem, this paper defines event relation triple to explicitly represent relations among triggers, arguments and roles which are incorporated into the model to learn their inter-dependencies. The task of argument extraction is converted to event relation triple extraction. A novel joint framework for multiple Chinese event extraction is proposed which jointly performs predictions for event triggers and arguments based on shared feature representations from pre-trained language model. Experimental comparison with state-of-the-art baselines on ACE 2005 dataset shows the superiority of the proposed method in both trigger classification and argument classification.

## introduction event extraction (ee) is of utility and challenge task in natural language processing (nlp). it aims to identify event triggers of specified types and their arguments in text. as defined in automatic content extraction (ace) program, the event extraction task is divided into two subtasks, i.e., trigger extraction (identifying and classifying event triggers) and argument extraction (identifying arguments and labeling their roles). chinese event extraction is a more difficult task because of language specific issue in chinese @xcite . since chinese does not have delimiters between words, segmentation is usually a necessary step for further processing, leading to word-trigger mismatch problem @xcite . the approaches based on word-wise classification paradigm commonly suffer from this. for instance, two characters in one word "打死" (hit and die) trigger two different events: an "attack"event triggered by "打" (hit) and a "die" event triggered by "死" (die). it is hard to extract accurately when a trigger is part of a word or cross multiple words. to avoid this issue, we formulate chinese event extraction as a character-based classification task. in addition, another interesting issue in event extraction which is rarely followed requires more efforts. it is the roles overlap problem that we concern in this paper, including the problems of either roles sharing the same argument or arguments overlapping on some words. there are multiple events existing in the one sentence, which commonly causes the roles overlap problem and is easy to overlook @xcite . plays the role "artifact". we observe that the above two arguments overlap on word "中国" (china)，which is more challenging for traditional methods to simultaneously identify these two arguments, especially for those being long noun phrases. research shows that there exist about 10% events in ace 2005 dataset @xcite having the roles overlap problem @xcite . moreover, the results of event extraction could affect the effectiveness of many other nlp tasks, such as the construction of knowledge graph. if there exist roles overlap problems in events, the model identities accurately when it predicts any one argument or role, which leads to omission and incompleteness of information for knowledge graph construction and is obviously far from real-world applications. therefore, the roles overlap problem is of great importance and needs to be seriously addressed. it is thus appealing to design a single architecture to solve the problem. although there exist prior studies that mention the roles overlap problem on ace 2005 dataset, they share the limitations that include either depending on elaborate engineering features (i.e, hand-crafted features @xcite , dependency paths @xcite , etc.) or following the pipelined approach @xcite . to overcome the issues of such prior works, in this paper, we propose a single framework to jointly extract triggers and arguments. inspired by the effectiveness of pre-trained language models, we adopt bidirectional encoder representation from transformer (bert) as the encoder to obtain the shared feature representations. specifically, the relations among triggers (t), ## related work ee is an important task which has attracted many attentions. there are two main paradigms for ee: a) the joint approach that predicts event triggers and arguments jointly, and b) the pipelined approach that first identifies trigger and then identifies arguments in separate stages @xcite . the advantages of such a joint system are twofold: (1) mitigating the error propagation from the upstream component (trigger extraction) to the downstream classifier (argument extraction), and (2) benefiting from the inter-dependencies among event triggers and argument roles @xcite . traditional methods that rely heavily on hand-craft features are hard to transfer among languages and annotation standards @xcite @xcite . the neural network based methods that are able to learn features automatically @xcite @xcite @xcite have achieved significant progress. most of them have followed the pipelined approach. some improvements have been made by jointly predicting triggers and arguments @xcite @xcite and introducing more complicated architectures to capture larger scale of contexts. these methods have achieved promising results in ee. unfortunately, roles overlap problem has been put forward @xcite , but there are only few works in the literature to study this. @xcite construct a multi-task learning with crf enhanced model to jointly learn sub-events. however, their method relies on hand-crafted features and patterns, which makes them difficult to be integrated into recent neural models. the similar work to ours is @xcite al.(2019 in recent years, pre-trained language models are successful in capturing words semantic information dynamically by considering their context. @xcite pre-train a deep lstm encoder from an attentional sequence-to-sequence model for machine translation (mt) to contextualize word vectors. elmo (embeddings from language models) improve 6 challenging nlp problems by learning the internal states of the stacked bidirectional lstm (long short-term memory) @xcite . open ai gpt (generative pre-training) improves the state-of-the-art in 9 of 12 tasks @xcite . bert obtains new state-of-the-art results on 11 nlp tasks @xcite . ## extraction model this section describes our approach that is designed to extract events occurring in plain text. we now define the scope of our work. the task of argument extraction is defined as automatically extracting event relation triples defined. in our model, instead of treating entity if there is no word triggering any event as well) with each set of binary classifiers severing for a role to determine the starts and ends of all arguments that play it. the roles overlap problem could be solved since the prediction could belong to different arguments and roles. besides, our jmcee enables to identify those arguments being long noun phrases like "来往于中国和澳 大利亚之间的乘客" (passengers who traveled between china and australia), which tackles the word boundary problem often encountered in chinese. compared with sentence-level sequential modeling methods, our approach also avoids suffering low efficiency in capturing very long-range dependencies in previous works @xcite . ## conclusions in this paper, we propose a simple yet effective joint chinese multiple events extraction framework which jointly extracts triggers and arguments by adopting a pre-trained bert encoder without elaborate engineering features. our contribution in this work is as follows: experiments have shown that our method outperforms conventional methods. we believe our proposed framework could be applied to many other nlp tasks for exploiting inner composition structure during extraction, such as entity relation extraction. our future work will focus on data generation to enrich training data and try to extend our framework to the open domain.