Extensively Matching for Few-shot Learning Event Detection (2020)

Abstract: Current event detection models under supervised learning settings fail to transfer to new event types. Few-shot learning has not been explored in event detection even though it allows a model to perform well with high generalization on new event types. In this work, we formulate event detection as a few-shot learning problem to enable to extend event detection to new event types. We propose two novel loss factors that matching examples in the support set to provide more training signals to the model. Moreover, these training signals can be applied in many metric-based few-shot learning models. Our extensive experiments on the ACE-2005 dataset (under a few-shot learning setting) show that the proposed method can improve the performance of few-shot learning.

## introduction event detection (ed) is an important task in information extraction (ie) in natural language processing (nlp). event detection is the task to detect event triggers from a given text (e.g. a sentence) and classify it into one of the event types of interest. the following sentence is an example of ed: in 1997, the company hired john d. idol to take over as chief executive. in this example, an ideal event detection system should detect the word hired as an event, and classify it to class of personnel:start-position, assuming that personnel:start-position is in the set of interested classes. the current works in ed typically employ traditional supervised learning based on feature engineering @xcite and neural networks @xcite @xcite . the main problem with supervised learning models is that they can not perform well on unseen classes (e.g. training a model to classify daily events, then run this model to classify laboratory operations). as a result, supervised learning ed can not extend to unseen event types. a trivial solution is to annotate more data for unseen event types, then retraining the model with newly annotated data. however, this method is usually impractical because of the extremely high cost of annotation @xcite . a human can learn about a new concept with limited supervision e.g. one can detect and classify events with 3-5 examples @xcite . this motivates the setting we aim for event detection: few-shot learning (fsl). in fsl, a trained model rapidly learns a new concept from a few examples while keeping great generalization from observed examples @xcite . hence, if we need to extend event detection into a new domain, a few examples are needed to activate the system in the new domain without retraining the model. by formulating ed as fsl, we can significantly reduce the annotation cost and training cost while maintaining highly accurate results. in a few shot learning iteration, the model is given a support set and a query instance. the support set consists of examples from a small set of classes. a model needs to predict the label of the query instance in accordance with the set of classes appeared in the support set. typical methods employ a neural network to embed the samples into a low-dimension vector space @xcite , then, classification is done by matching those vectors based on vector distances @xcite @xcite . one potential problem of prior fsl methods is that the model relies solely on training signals between query instance and the support set @xcite @xcite . thus, the matching information between samples in the support set has not been exploited yet. we believe that this is not an efficient use of training data because dataset in ed is very small @xcite . therefore, in this study, we propose to train an ed model using matching information (1) between query instance and the support set and (2) between the samples in the support themselves. this is implemented by adding two auxiliary factors into the loss function to constrain the learning process. we apply the proposed training signals to different fsl models on the benchmark event detection dataset @xcite . the experiments show that the training signal can improve the performance of the examined fsl models. to summarize, our contributions to this work include: ## related work early studies in event detection mainly address feature engineering for statistical models @xcite @xcite @xcite including semantic features and syntactic features. recently, due to the advances with deep learning, many neural network architectures have been presented for ed, e.g. convolutional neural networks (cnn) @xcite @xcite , recurrent neural networks (rnn) @xcite @xcite and graph convolutional neural networks (gcn) @xcite . these methods formulate ed as a supervised learning problem which usually fails to predict the labels of new event types. by transitioning the symbolic event types to descriptive event types in the form of bags of keywords @xcite @xcite , the adaptibility of event detection can be formed as a supervised-learning problem. however, these studies have not examined fsl as we do in this work. one can also address this problem in zero-shot learning with data generated from abstract meaning representation @xcite or two-stage pipeline ( trigger identification and few-shot event classification) based on dynamic memory network @xcite . a recent study has employed few-shot learning for event classification @xcite . our work is similar in terms of formulation, however, we consider it in a larger extent of event detection where the null event is also included. few-shot learning has been studied early in the literature @xcite . before the era of the deep neural network, fsl approaches focused on building generative models that can transfer priors across classes. however, these methods are hard to apply to real applications because they require a subject-dedicated design such as handwritten characters @xcite . as a result, they cannot capture the nature of the distribution @xcite . later studies, based on deep neural network, proposed metric learning to model the distribution of distance among classes, @xcite with many incremental improvements in distance functions such as cosine similarity @xcite , euclidean distance @xcite and learnable distance function @xcite . metric-based fsl presents its advantages in two dimensions. first, it is based on the well-studied theory in distance functions. second, the simplicity in architecture and training processes can encourage its application in practice. recently, meta-learning with parameter update strategy is also proposed to enable the models to learn quickly in few training iterations @xcite . ## methodology our goal in this work is to formulate ed as a fsl problem, which has not been done in prior work. in order to achieve this, this section is divided into three parts. in the section 3.1 we present the overall framework that formulate event detection as an few-shot learning problem. then, we present popular models for fsl in the prior work and common sentence encoders which have been widely used in ed in section 3.2. finally, we present two novel reguarlization technique to further improve the fsl model for ed in section 3.3. ## conclusion in this paper, we address the problem of extending event detection to unseen event types through fewshot learning. we investigate four metric-based few-shot learning models with different encoder types (cnn, lstm, and gcn). moreover, we propose two novel loss functions to provide more training signals to the model exploiting domainmatching information in the support set. our extensive experiments show that our method increases the efficiency of using training data, resulting in better classification performance. our ablation study shows that both intra-cluster matching and intercluster matching contributes to the improvement.