HCL - TAT : A Hybrid Contrastive Learning Method for Few-shot Event Detection with Task-Adaptive Threshold (2022)

Abstract: Event detection has been suffering from constantly emerging event types with lack of sufficient data. Existing works formulate the new problem as few-shot event detection (FSED), and employ two-stage or unified models based on meta-learning to address the problem. However, these methods fall far short of expectations due to: (i) insufficient learning of discriminative representations in low-resource scenarios, and (ii) representation overlap between triggers and non-triggers. To resolve the above issues, in this paper, we propose a novel Hybrid Contrastive Learning method with a Task-Adaptive Threshold (abbreviated as HCL-TAT), which enables discriminative representation learning with a two-view contrastive loss (support-support and prototype-query), and devises an easily-adapted threshold to alleviate misidentification of triggers. Extensive experiments on the benchmark dataset FewEvent demonstrate the superiority of our method to achieve better results compared to the state-of-the-arts. All the data and codes will be available to facilitate future research.

## introduction event detection (ed) is the subtask of information extraction (ie) @xcite , which aims at extracting events of task-specified types from an input text and is crucial for many downstream applications such as text summarization @xcite and machine reading comprehension @xcite . for example, in the sentence "kenyan police intensify manhunt for terror suspects", an ideal ed model is to identify "terror" as a trigger word and classify it into "conflict.attack" event type. previous works commonly formulate ed as a token-level classification problem and follow a supervised manner @xcite @xcite @xcite . despite the promising results, in real-world scenarios, new event types are constantly emerging with only a few samples, which gives rise to the problem of low generation on newly-emerged event types, in view of insufficient annotated data. therefore, it is reasonable to convert the traditional supervisedbased event detection task to the few-shot event detection (fsed) problem. most works in fsed exploit metric-based metalearning methods, in which the model needs to learn meta-knowledge from only a few instances in the support set and generalize to predict labels for instances in the query set. specifically, in each episode, these methods first build a prototype for each class over the support set, then predict the labels for each instance in query set by matching with the closest prototype representation in the metric space. indeed, many of these methods follow an identify-to-classify paradigm and are capable of keeping high generation on new event type based on a few observed samples @xcite . however, the performance of such two-stage model is limited to the problem of error propagation caused by trigger misidentification. thus, recent works formulate fsed as a few-shot sequence labeling task and apply conditional random field based (crf-based) methods to jointly identify and classify triggers. nevertheless, it is usually challenging for these crf-based methods to fully learn event type dependencies from a single sentence, since a sentence commonly contains only one trigger, or even worse with limited data in fsed. an example is presented in figure 1a , even if the state-of-the-art method pa-crf @xcite is powerful in exploring event type dependencies via gaussian distribution for approximation, it is still incapable of generating well discriminative representations for each event type in the whole embedding space. actually, it is non-trivial to solve the problem. typical metric-based models are mostly optimized with a query-anchored cross-entropy loss based on the similarity between each query instance and all prototypes. however, we find that the gradient directions of prototypes are not always optimized very well during training, and may not guarantee the learnt representations of prototypes are discriminative via optimizing the query-anchored loss, which in turn harms the learning of query instances. detailed proof can be found in appendix c. to address the problem, in this paper, we propose a hybrid contrastive learning (hcl) method to improve the representations for instances in both support set and query set. specifically, we first propose a support-support contrastive learning (sscl) method to makes it prone to generating more discriminative representations of prototypes, via encouraging instances of the same type closer and others of different types farther away within support set. however, the optimization direction of sscl is still unclear and only information within the support set is considered. therefore, inspired by @xcite , we further design a prototype-query contrastive learning (pqcl) method, which pulls query instances together with the prototype of the same type and pushes them apart from prototypes of different types. logically, pqcl is capable of guiding the learning process of sscl in a theoretically sound way, and such two contrastive learning losses are helpful for queryanchored loss to generate more discriminative representations for both the support set and the query set. additionally, another serious problem for typical fsed is that most of the words in ed sentences are non-triggers, i.e., they do not belong to any event type, called "o" type. the unbalanced nature influences the representation learning of triggers, and leads to representation overlap problem between triggers and non-triggers. to address this issue, we further design a task-adaptive threshold (tat). our key insight is that, if the similarity between a query instance and all event types are lower than the average similarity between query instances and "o" type, then it is most likely to be a non-trigger. thus, in each episode, we take the average similarity between all query instances and the prototype of "o" type as the threshold to filter misidentified triggers. this threshold can be easily adapted to any episode and ensure the generalization ability. as illustrated in figure 1b , by combining the above components, our hcl-tat could learn more discriminative representations and make better predictions, thus boosting the performance. the contributions of our work are summarized as follows: with advances in deep learning, many research efforts have been dedicated to enhancing ed with different neural network architectures like cnnbased model @xcite and rnn-based model @xcite . recently, pre-trained language models (plms) are adopted to leverage rich information from large-scale corpora @xcite @xcite . these methods achieve promising results for supervised ed, but they could not adapt well to fsed with limited data. few-shot event detection. in recent years, there have been several attempts to conduct metalearning based approaches @xcite for fsed. deng et al. (2020) proposed the benchmark fsed dataset fewevent, and designs a dynamic-memory-based prototypical network (dmbpn) to preserve contextual information of event mentions. lai et al. (2020a) and lai et al. (2020b) proposed intra and inter-cluster matching losses to provide more training signals. however, these works categorize events at sentence-level and are solely for the event classification task. recently, cong et al. (2021) proposed to treat ed as few-shot sequence labeling task and solved with a modified pa-crf model, but they still suffer from poor representation learning. in this paper, we propose to exploit contrastive learning to produce more discriminative representations. contrastive learning. contrastive learning has been widely used in various domains based on an idea to pull together similar instances and push apart distinct instances. self-supervised learning designs specific strategies to generate positive and negative pairs from unsupervised data @xcite @xcite @xcite . as an extension, supervised contrastive learning which leverages label information to generate positive and negative pairs is further proposed @xcite @xcite . recently, contrastive learning is also used in few-shot learning tasks to improve the performance @xcite , by pulling query samples closer to the prototype of the same class and further away from those of different classes. in this paper, we propose a hybrid contrastive learning (hcl) method, composed of two complementary contrastive losses, to generate more discriminative representations for both support and query set. with the help of a task-adaptive threshold (tat) method, the proposed hcl-tat can fully make use of limited data and boosts the performance. ## conclusion in this paper, we propose a contrastive learning method to make full use of limited data and produce more discriminative representations for fsed. specifically, we first propose a hybrid contrastive loss, composed of support-support contrastive loss and prototype-query loss, to conduct supervised contrastive learning within support set and between support and query set respectively. furthermore, we design a task-adaptive threshold method, to regularize the distance-based classifier in each episode. experimental results show that by improving the representation learning and classifier learning simultaneously, our method boost the performance under all four settings on fewevent dataset.