Treasures Outside Contexts: Improving Event Detection via Global Statistics (2021)

Abstract: Event detection (ED) aims at identifying event instances of specified types in given texts, which has been formalized as a sequence labeling task. As far as we know, existing neural-based ED models make decisions relying entirely on the contextual semantic features of each word in the inputted text, which we find is easy to be confused by the varied contexts in the test stage. To this end, we come up with the idea of introducing a set of statistical features from word-event co-occurrence frequencies in the entire training set to cooperate with contextual features. Specifically, we propose a Semantic and Statistic-Joint Discriminative Network (SS-JDN) consisting of a semantic feature extractor, a statistical feature extractor, and a joint event discriminator. In experiments, SS-JDN effectively exceeds ten recent strong baselines on ACE2005 and KBP2015 datasets. Further, we perform extensive experiments to comprehensively probe SS-JDN.

## introduction event detection (ed) is an important information extraction task in the nlp field, which aims to identify event instances of specified types in given texts. associated with each event mention is a phrase, i.e., the event trigger 1 evoking that event. more precisely, the task involves identifying event triggers and classifying them into the specific types. for instance, according to ace2005 annotation guideline, in the sentence "a police officer was killed in new jersey today", an ed model should be able to recognize the word "killed" as a trigger for the event type "die". with the development of deep learning, ed has been formalized as a sequence labeling task and figure 1 : a case study of sota model dmbert @xcite and our proposed model. we train dmbert and our proposed model on ace2005, and select two texts (t1 and t2) with "killed" and "wounded" as triggers in the training set. for testing, we change the contexts of the two triggers in t1 and t2 to obtain v1 to v4, and let the two ed models predict their event types in the new contexts. we can see that dmbert is easy to get confused in the new contexts, while our model is obviously more stable. the detailed explanations are in section 4.3.5. theoretically, the contextual information in a given text is enough to determine the event type of each word. but the fact is an event can be described by various contexts, and the contexts in the test stage are usually not covered by the training set. therefore, a common shortcoming of existing neural-based ed models is that they are prone to get confused by the changeable contexts during testing. as shown in figure 1 , the variation of any surrounding words will lead to a different set of extracted contextual features and may misguide their final judgments. to alleviate this shortcoming, a natural idea is to seek an additional "stable" decision-making basis unchanged across varied contexts to cooperate with the contextual features. therefore, we envisage extracting a set of event features of words from the entire training set through global statistics-wordevent co-occurrence frequencies. global statistics were widely used by many early pattern-based and feature-based ed models @xcite . however, with the introduction of deep learning technologies @xcite , various powerful neural-based ed models directly extract contextual semantic features from each text via end-to-end architectures. the use of global statistics has gradually faded out of researchers' attention. in our work, we find that the word-event co-occurrence frequencies in ed datasets have potential to help neural-based ed models eliminate a large number of interference options in testing (section 3.1). thus, as the title suggests, the valuable features for neural-based ed models can be collected outside the contexts. specifically, we design a simple but novel semantic and statistic-joint discriminative network (s 2 -jdn) to implement the new decisionmaking pattern, which consists of three modules. (i) semantic feature extractor takes dynamic multi-pooling bert @xcite as the prototype and is used to obtain the token-level contextual features in each text. (ii) statistical feature extractor mines event features in both direct and indirect ways from word-event co-occurrence frequencies, and then fuses and rescales them as the final statistical event features. (iii) joint event discriminator adopts a layer normalization to unify the two types of event features and combines them for decision-making. in experiments, we compare with ten strong baselines on ace2005 and kbp2015 datasets, where s 2 -jdn exceeds the state-of-the-art (sota) models by 1.9% and 1.9% in trigger classification f1, respectively. further, we perform extensive experiments and draw multiple useful conclusions about s 2 -jdn. our contributions can be summarized as follows: ## related work as the key component of the event extraction system @xcite @xcite , the research of ed has experienced the periods of traditional methods and deep learning methods. during the period of traditional methods, global statistics collected from the training set were widely used as the knowledge sources or decisionmaking basis of different ed models @xcite @xcite @xcite . @xcite and @xcite used statistical information to train a maxent event classifier. @xcite constructed a frequency patternbased framework for ed. ji and @xcite obtained document-wide and cluster-wide statistics to correct the results of ed. @xcite proposed a classifier-based method to process statistical features for event filtering. @xcite recorded the frequency that each pattern is associated with an event type, and treated the frequencies as core features. with the introduction of deep learning technologies, many neural-based ed models extracted contextual semantic features from each text via end-toend architectures @xcite @xcite . @xcite acquired the contextual features via convolution and dynamic multi-pooling techniques. @xcite and @xcite employed the attention mechanisms during contextual feature extraction. nguyen and @xcite , @xcite , and @xcite introduced external knowledge to as-sist neural networks to better understand each given text. recently, many studies applied powerful pretrained language models to better comprehend contexts @xcite @xcite . neural-based ed models significantly outperform the traditional ones, and become the new research hotspot. accordingly, the use of global statistics has faded out of researchers' attention. as far as we know, there is no neural-based ed model explicitly using global statistics in the training sets. although neural networks possess powerful learning ability in theory, their training is based on individual sentences, and thus it's not easy for them to capture the global statistics. more importantly, we observe that the word-event co-occurrence frequencies for most words concentrate on only a few events, which are potential to help neural-based ed models eliminate a large number of interference options in testing (section 3.1). to this end, we propose s foot_0 -jdn, which takes into account both the (contextual) semantic features and statistical features of each word to make a decision. besides employing the collected global statistics as features directly like traditional ed models, s 2 -jdn also leverages the property of neural networks to indirectly extract more abundant features from wordevent co-occurrence frequencies (section 3.2.2). ## methodology in this section, we first elaborate the motivation for introducing word-event co-occurrence frequencies (section 3.1), then propose a concrete instantiation called semantic and statistic-joint discriminative network (s 2 -jdn) (section 3.2), and finally describe the training details of s 2 -jdn (section 3.3). ## conclusion & future work in this paper, we find that existing neural-based ed models are likely to get confused by changeable contexts during testing. to alleviate this problem, we propose s 2 -jdn model, which extracts a set of statistical event features from word-event cooccurrence frequencies as an additional decision basis besides contextual information. experimental results on two benchmark datasets ace2005 and kbp2015 against ten recent sota ed models demonstrate the effectiveness of s 2 -jdn and each proposed module. for future work, there are three intriguing directions: (i) extending the decision-making pattern of s 2 -jdn to other neural-based ed models and thus absorbing their advantages; (ii) incorporating the word-event co-occurrence frequencies into neuralbased ed models via prior on their output distributions in a bayesian framework; and (iii) combining s 2 -jdn with data augmentation methods, so as to collect more accurate global statistics and further promote the performance.