Zero-Shot Event Detection Based on Ordered Contrastive Learning and Prompt-Based Prediction (2022)

Abstract: Event detection is a classic natural language processing task. However, the constantly emerging new events make supervised methods not applicable to unseen types. Previous zero-shot event detection methods either require predefined event types as heuristic rules or resort to external semantic analyzing tools. To overcome this weakness, we propose an end-to-end framework named Zero-Shot Event Detection Based on Ordered Contrastive Learning and Prompt-Based Prediction (ZEOP). By creatively introducing multiple contrastive samples with ordered similarities, the encoder can learn event representations from both instance-level and class-level, which makes the distinctions between different unseen types more significant. Meanwhile, we utilize the prompt-based prediction to identify trigger words without relying on external resources. Experiments demonstrate that our model detects events more effectively and accurately than state-of-the-art methods.

## introduction as a classic nlp task, event detection aims to identify events from natural language text. most traditional supervised event detection methods @xcite @xcite rely on a great number of event-specific annotated texts. however, in practice, obtaining largescale and high-quality annotated data requires significant expertise and expensive resources. in the real-world scenarios shown in figure 1 , the constantly emerging of new events without annotated samples, making supervised event detection methods no longer applicable. to solve this challenge, the zero-shot event detection task is proposed to automatically discover and classify new events from unstructured texts in the absence of manual annotation. following previous works @xcite @xcite event mentions (unlabeled) huang and ji, 2020; @xcite , we denote the known types as seen types and the new types as unseen types. "unseen" means that the event labels of samples are not visible to the model. recently, multiple zero-shot event detection methods have been proposed and show better performance than supervised methods on zero-shot tasks. however, they @xcite @xcite all require predefined event types as heuristic rules or external semantic analyzing tools. for example, event names are used to query trigger words by semantical similarity, or the part-of-speech tagging tools are used to find nouns and verbs in the text as candidate trigger words. in these ways, human effort and external resources are still necessary when detecting new event types. there is also a problem of error accumulation across the tools and models. to overcome the above weakness, this paper proposes an end-to-end model named zero-shot event detection based on ordered contrastive learning and prompt-based prediction. the main idea is introducing contrastive learning to move away from the dependence on heuristic rules for unseen event detection. as shown in figure 2 , traditional original sample contrastive sample 𝐴 𝑆 1 𝑆 3 𝑆 2 𝑑 1 𝑑 2 𝑑 3 𝑑 1 < 𝑑 2 < 𝑑 3 𝑑 0 < (𝑑 1 , 𝑑 2 , 𝑑 3 ) 𝐴 𝑆 1 𝐵 𝑆 2 𝑆 3 𝑑 1 𝑑 2 𝑑 3 𝑑 0 contrastive learning simply divide samples into two opposite classes: positive or negative, while we construct four contrastive samples with different similarities to the original sample. then, the ordered contrastive learning can draw a stronger distinction between different unseen type events by learning the partial order relationship of different contrastive samples. meanwhile, in order to discover new event types without relying on heuristic rules, we utilize the prompt-based prediction @xcite @xcite for trigger words identification, which has been proved to be an efficient few-shot learner. the main work of this paper has the following three points: ## methodology the architecture overview of zero-shot event detection based on ordered contrastive learning and prompt-based prediction (zeop) is shown in figure 3 . given a set of seen events s and unseen events u , all samples are first input into the contrastive sample generator as the original sample x i , where a list of multiple contrastive samples is constructed as {s 1 , . . . , s 4 }. subsequently, the event encoder encodes event mentions as embedding vector e i , and the prototypical network predicts probability distribution over event types as p i . ordered contrastive loss is calculated for all samples, and supervised loss is calculated only for samples of seen events. model parameters in the event encoder and prototypical network could be updated by gradient backward. ## conclusion in order to solve the problem of zero-shot event detection, this paper proposes an end-to-end model named zero-shot event detection based on ordered contrastive learning and prompt-based prediction. by creatively introducing multiple contrastive samples with different similarities, the contrastive loss is extended from pairwise comparison to list-wise comparison. therefore, the model could learn a better representation across instancelevel and class-level. meanwhile, the prompt-based prediction is utilized to identify event trigger words without relying on heuristic rules. experiments demonstrate that our method can significantly improve the accuracy of identifying unseen event types while keeping the ability to classify seen event types. future research should consider the potential effects of the initialization of the prototype matrix in the prototypical network more carefully. a better initial value may reduce the need for large training samples and speed up model training.