Improving Event Definition Following For Zero-Shot Event Detection (2024)

Abstract: Existing approaches on zero-shot event detection usually train models on datasets annotated with known event types, and prompt them with unseen event definitions. These approaches yield sporadic successes, yet generally fall short of expectations.In this work, we aim to improve zero-shot event detection by training models to better follow event definitions. We hypothesize that a diverse set of event types and definitions are the key for models to learn to follow event definitions while existing event extraction datasets focus on annotating many high-quality examples for a few event types. To verify our hypothesis, we construct an automatically generated Diverse Event Definition (DivED) dataset and conduct comparative studies. Our experiments reveal that a large number of event types (200) and diverse event definitions can significantly boost event extraction performance; on the other hand, the performance does not scale with over ten examples per event type.Beyond scaling, we incorporate event ontology information and hard-negative samples during training, further boosting the performance. Based on these findings, we fine-tuned a LLaMA-2-7B model on our DivED dataset, yielding performance that surpasses SOTA large language models like GPT-3.5 across three open benchmarks on zero-shot event detection.

## introduction event detection (ed) focuses on identifying event triggers of specific event types in a given text with predefined event ontology. prior work has studied event detection largely in a fully-supervised fashion (wadden et al., 2019; lin et al., 2020; nguyen and grishman, 2015; nguyen et al., 2016; han et al., 2019; du and cardie, 2020; cheng * equal contribution. ## method in this section, we describe our data generation pipeline to generate the diverse event definition dataset (dived) and our systematic study on the impact of various components within event detection training data. we investigate how (1) the scaling of event types, event definition, and training samples and (2) incorporating ontology information and hard-negative samples can impact models' generalization to unseen event types. ## conclusion we investigate how incorporating diverse event types and definitions benefits zero-shot event detection models. the proposed dived dataset features a large number of diverse event types and definitions, which helps train the model to better generalize to unseen event definitions. by further incorporating event ontology and hard negative samples, we finetuned a llama-2 model on dived and geneva datasets, which consistently surpasses previous sota chatgpt prompting baselines in zero-shot ed on ace, m2e2, and mee datasets. overall, our findings provide insights to improve models' event definition following ability and provide an opportunity to further advance zero-shot ed on open-sourced models.