Zero- and Few-Shot Event Detection via Prompt-Based Meta Learning
Year: 2023

Abstract:
With emerging online topics as a source for numerous new events, detecting unseen / rare event types presents an elusive challenge for existing event detection methods, where only limited data access is provided for training. To address the data scarcity problem in event detection, we propose MetaEvent, a meta learningbased framework for zeroand few-shot event detection. Specifically, we sample training tasks from existing event types and perform meta training to search for optimal parameters that quickly adapt to unseen tasks. In our framework, we propose to use the clozebased prompt and a trigger-aware soft verbalizer to efficiently project output to unseen event types. Moreover, we design a contrastive meta objective based on maximum mean discrepancy (MMD) to learn class-separating features. As such, the proposed MetaEvent can perform zero-shot event detection by mapping features to event types without any prior knowledge. In our experiments, we demonstrate the effectiveness of MetaEvent in both zero-shot and few-shot scenarios, where the proposed method achieves state-of-the-art performance in extensive experiments on benchmark datasets FewEvent and MAVEN.

heading: 1 Introduction
text: Event detection tasks have experienced significant improvements thanks to the recent efforts in developing language-based methods (Lu et al., 2021; Pouran Ben Veyseh et al., 2021). One of such methods is pretrained large language models, which can be fine-tuned for detecting events upon input context (Liu et al., 2019; Cong et al., 2021). However, the detection of unseen or rare events remains a challenge for existing methods, as large amounts of annotated data are required for training in a supervised fashion (Shang et al., 2022; Zhang et al., 2022c). For instance, existing models often fail to detect unseen event types due to the lack of domain knowledge, as demonstrated in Figure 1.
To detect unseen event types (i.e., zero-shot learning), existing methods leverage external knowledge or contrastive learning to build classseparating features (Lyu et al., 2021; Zhang et al., 2022a,b). Similarly, it is possible to leverage limited annotated examples (i.e., few-shot learning) for event detection. For instance, prototypical features can be constructed to match event types upon inference (Deng et al., 2020; Cong et al., 2021). Promptbased methods align with the pretraining objective of language models to improve detection performance (Li et al., 2022a,b). Specifically, a clozebased prompt template (e.g., A <mask> event) is incorporated as part of the input, and the prediction can be obtained by decoding the <mask> prediction. As such, prompt-based methods exploit the masked language modeling (MLM) pretraining task by constructing similar input examples.
Nevertheless, no existing method is designed for both zero-shot and few-shot event detection, as it is a non-trivial problem to combine both settings under a unified framework. Previous efforts either address the zero-shot or the few-shot setting, yet detecting both unseen and rare events can be helpful in various scenarios (e.g., detecting emergency events online), which renders current approaches less effective in real-world applications. Additionally, event detection comprises of two subtasks:
7928
trigger identification and classification. Some methods provide an incomplete solution by solely performing the classification task, while many other approaches execute both tasks at the cost of reduced performance (Schick and Schütze, 2021; Li et al., 2022b). This is because existing methods heavily rely on trigger words for classification, which causes drastic performance drops in the case of trigger mismatch or ambiguity (Ding et al., 2019).
In this paper, we propose a meta learning framework MetaEvent for zero- and few-shot event detection. We consider both settings and optimize language models to identify unseen / rare events via a unified meta learning framework. That is, given zero / limited number of annotated examples per event type, our objective is to maximize the model performance in detecting such events. To this end, we develop a solution to integrate the trigger identification and classification subtasks for efficient forward passing in meta training. Moreover, we design a trigger-aware soft verbalizer to identify event types in our prompt-based event detection model. For optimization, we propose a meta objective function based on contrastive loss to learn generalizable and class-separating event features. In training, the model is first updated and evaluated upon the sampled zero- or few-shot tasks, then the meta loss is computed to derive gradients w.r.t. the initial parameters, such that the updated model learns to generalize to the target tasks even without labeled examples. In other words, MetaEvent learns from seen tasks, yet with the objective to generalize in rare and unseen scenarios. Therefore, the resulting model can optimally adapt to the target task upon deployment. We demonstrate the effectiveness of MetaEvent by evaluating zero- and few-shot event detection tasks on benchmark datasets, where MetaEvent consistently outperforms state-of-theart methods with considerable improvements.
We summarize our contributions as follows1:
1. To the best of our knowledge, we are the first to propose a unified meta learning framework for both zero- and few-shot event detection. MetaEvent is designed to exploit prompt tuning and contrastive learning for quick adaptation to unseen tasks.
2. We propose an integrated trigger-aware model in MetaEvent for efficient meta training. In
1We adopt publicly available datasets in the experiments and release our implementation at https://github.com/Yueeeeeeee/MetaEvent.
particular, our trigger-aware soft verbalizer leverages both the prompt output and attentive trigger features to identify event types.
3. We design a novel contrastive loss as the meta objective of MetaEvent. Our contrastive loss encourages class-separating and generalizable features to improve event matching in both zero- and few-shot event detection.
4. We demonstrate the effectiveness of MetaEvent with extensive experiments, where MetaEvent outperforms state-of-the-art baseline methods with considerable improvements in both zero- and few-shot event detection.

heading: 2 Related Work

heading: 2.1 Event Detection
text: Event detection refers to the task of classifying event types upon input text. While event detection has achieved progress under the supervised training paradigm (Ji and Grishman, 2008; Lin et al., 2020; Wadden et al., 2019; Liu et al., 2020; Du and Cardie, 2020; Lu et al., 2021; Liu et al., 2022), zeroand few-shot classification remains a challenge due to the lack of prior knowledge and annotated examples. For the zero-shot setting, relevant literature focuses on predefined event knowledge or heuristics to classify unseen events (Huang et al., 2018; Huang and Ji, 2020; Lyu et al., 2021; Zhang et al., 2021b; Yu et al., 2022; Yu and Ji, 2023; Zhan et al., 2023). Similarly, zero-shot contrastive learning requires unlabeled examples in training to learn class-separating features (Zhang et al., 2022b). Under the few-shot setting, prototypical networks and prompt-based tuning improve detection performance via prototype matching and alignment to language pretraining (Deng et al., 2020; Cong et al., 2021; Schick and Schütze, 2021; Lai et al., 2021; Li et al., 2022b). Overall, existing approaches focus on either zero- or few-shot event detection without considering a unified framework for both settings. Moreover, current zero-shot methods require additional resources for training, making such methods less realistic in real-world event detection. As such, we propose a meta learning framework MetaEvent for both zero- and few-shot event detection, where neither prior knowledge nor unlabeled examples are required to detect unseen events.

heading: 2.2 Prompt Learning
text: Prompt learning uses a predefined template with slots (i.e., A <mask> event) to instruct language models on the desired task, where the slot prediction is used to derive the final output (Brown et al., 2020; Liu et al., 2021). By leveraging the pretraining objective and designing prompt templates, pretrained large language models can be adapted for zero- or few-shot downstream tasks (Houlsby et al., 2019; Raffel et al., 2020). Soft and multitask prompts further improve the zero- and fewshot performance of language models on unseen tasks (Lester et al., 2021; Sanh et al., 2021). Clozebased prompts are proposed for the event detection task, where the predicted output can be mapped to the event types using verbalizer or mapping heuristics (Schick and Schütze, 2021; Li et al., 2022b; Zhang et al., 2022b). Nevertheless, previous prompt methods adopt the inefficient two-step paradigm (i.e., trigger identification and classification) and are not designed for both zero- and few-shot event detection. Therefore, we integrate both steps with a trigger-aware soft verbalizer for efficient forward passing in meta training. Moreover, we consider both zero- and few-shot scenarios with our prompt-based meta learning framework MetaEvent. By leveraging the proposed components, our approach demonstrates considerable improvements compared to existing methods.

heading: 2.3 Meta Learning
text: Meta learning (i.e., learning to learn) has demonstrated superiority in few-shot learning (Finn et al., 2017; Nichol et al., 2018; Rajeswaran et al., 2019). Existing methods learn class-wise features or prototypes in the metric space for quick adaptation to few-shot tasks (Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018). Model-agnostic meta learning (MAML) leverages the second-order optimization to find the optimal initial parameters for a new task (Finn et al., 2017). Approximation methods of second-order MAML demonstrates comparable performance while requiring significantly reduced computational resources (Finn et al., 2017; Nichol et al., 2018; Rajeswaran et al., 2019). Meta learning has also been applied to tasks like online learning, domain adaptation and multi-task learning (Finn et al., 2019; Li and Hospedales, 2020; Wang et al., 2021a). For event detection, meta learning is proposed to improve the performance on small-size data via memory-based prototypi-
cal networks and external knowledge (Deng et al., 2020; Shen et al., 2021).
To the best of our knowledge, meta learningbased methods for both zero- and few-shot event detection is not studied in current literature. However, detecting unseen and rare events is necessary in many applications. An example can be detecting emergency events on social media, where both unseen and rare events can be present (e.g., pandemic, safety alert etc.). Therefore, we propose MetaEvent: a meta learning framework for zero- and few-shot event detection. MetaEvent exploits seen events via trigger-aware prompting and a carefully designed meta objective, and thereby improving the zero- and few-shot performance with class-separating and generalizable features.

heading: 3 Preliminary
text: We consider the following event detection problem setup, whereN -wayK-shot examples are available for training each task (K is 0 for zero-shot setting). Our goal is to train a model f that maximizes the performance in unseen tasks. Problem: Our research focuses on zero- and fewshot event detection based on a collection of tasks {T i}Mi=1. For each task T i, a N -way K-shot training set and a held-out evaluation set are provided (i.e., Dtraini ,Dtesti ∈ T i). The training of MetaEvent is two-fold: (1) an initial model is updated using the training sets in each sampled task to achieve local convergence (i.e., inner-loop optimization); (2) the updated models are used to compute a meta loss on the corresponding evaluation sets, followed by deriving the gradients w.r.t. the initial model using our meta learning algorithm (i.e., outer-loop optimization). In particular, the input for each task T i include: • Training set: Dtraini contains K examples for
each of the N classes. An example comprises of context x(j)c , trigger x (j) c and label y(j) (i.e., Dtraini = {(x(j)c ,x(j)t , y(j))}N∗Kj=1 ). In the fewshot setting, Dtraini is used to tune the model, while Dtraini is an empty set for zero-shot setting. Training set is also known as support set.
• Evaluation set: Similarly, a held-out evaluation set Dtesti from the same input & label space is used to compute the meta loss in training. Upon deployment, the model can be updated with Dtraini and should perform well on Dtesti . Evaluation set is often referred to as query set.
Pipeline: We are interested in learning an encoder model f parameterized by θ. Conventional event detection methods compute trigger as an intermediate variable, followed by the event type classification. As such, we formulate fθ with contexts as input and event features as output, followed by some classifier CLF to map the features to the desired event type (i.e., y = CLF(fθ(xc))). Our goal is to find the optimal parameter set θ that quickly adapts to an unseen task T i using Dtraini and maximizes the performance on evaluation set Dtesti . Mathematically, this is formulated as optimization of θ over a collection of M evaluation tasks:
min θ
1
M
M∑
i=1
L(Alg(θ,Dtraini ),Dtesti ), (1)
where L represents the loss and Alg represents the gradient descent optimization algorithm.

heading: 4 Methodology

heading: 4.1 Model Design
text: To efficiently perform prompt-based meta training for event detection, we design a one-step model that integrates the trigger identification and classification stages. This is because a two-step approach (e.g., P4E (Li et al., 2022b)) requires extensive computational resources to obtain the gradients in the inner- and outer-loop optimization in MetaEvent. Consequently, we design an efficient model (as illustrated in Figure 2) that integrates attentive trigger features to avoid additional forward passes.
Different from existing trigger-aware methods (Ding et al., 2019), the proposed model innovatively uses both attentive trigger features and
prompt output to predict the event types. The attentive trigger features t can be computed using an integrated trigger classifier and attention weights from the pretrained language model. Specifically, a trigger classifier is trained upon each token features to perform binary classification (i.e., whether input token is trigger token). In inference, the classifier predicts the probabilities p of input tokens being classified as trigger words. To better estimate the importance of predicted trigger tokens, we design an attentive reweighting strategy to select informative trigger features from the input context. The idea behind our attentive reweighting strategy is to leverage attention scores from the model to select more relevant features. The attention scores reveal different importance weights of the context tokens and thus, can be used to compute ‘soft’ trigger features based on the semantics. Formally, our attentive reweighting strategy computes weights w ∈ RLc using trigger probabilities p and attention scores A ∈ RH×Lc×Lc of the context span from the last transformer layer. The weight of the i-th token wi in w is computed via
wi = σ
( p⊙ 1
H
H∑
j
( Lc∑
k
Aj,k
))
i
, (2)
where H is the number of attention heads, Lc is the context length, ⊙ and σ denote elementwise product and softmax functions. Base on input xc and w, the attentive trigger features t can be computed as the weighted sum of token features, i.e.,
t =
Lc∑
i=1
wifθ(xc)i. (3)
For the event classification, we design a promptbased paradigm using a predefined prompt and a trigger-aware soft verbalizer. Specifically, we preprocess the input context by prepending the prompt ‘A <mask> event’ to transform the prediction into a masked language modeling (MLM) task. The pretrained encoder model and MLM head fill the <mask> position with a probability distribution v over all tokens. Then, our trigger-aware soft verbalizer maps the predicted distribution v to an output event type. Unlike predefined verbalizer functions (Li et al., 2022b), we design a learnable verbalizer based on MLM predictions v and attentive trigger features t. For the N -way few-shot setting, the trigger-aware soft verbalizer with weights W ∈ R(|v|+|t|)×N and bias b ∈ RN predicts the output label with GELU activation via
ŷ = argmax(GELU([v; t])W + b). (4)
For zero-shot event detection, we use the concatenated features [v; t] to project input to unseen event types via the Hungarian algorithm.

heading: 4.2 Meta Training
text: Provided with the training and evaluation sets from sampled tasks, we present the formulation of our MetaEvent and our methods for the zero- and fewshot training. The designed framework leverages meta training to search for optimal parameters θ. Once trained, the event detection model quickly adapts to unseen tasks even without examples (Finn et al., 2017; Yue et al., 2023).
Given a set of tasks {T i}Mi=1 and model f parameterized by θ, MetaEvent aims at minimizing the overall evaluation loss of the tasks (as in Equation (1)). MetaEvent consists of an inner-loop optimization stage (i.e., Alg) and an outer-loop optimization staget that minimizes the overall loss w.r.t. θ. For the inner-loop update, Alg denotes gradient descent with learning rate α, i.e.:
Alg(θ,Dtrain) = θ − α∇θL(θ,Dtrain) = ϕ, (5) we denote the updated parameter set with ϕ. In the outer-level optimization, we are interested in learning an optimal set θ that minimizes the meta loss on the evaluation sets. The learning is achieved by differentiating through the inner-loop optimization (i.e., Alg) back to the initial parameter set θ, which requires the computation of second-order gradients or first-order approximation (as shown in Figure 2).
Specifically, we derive the gradients w.r.t. θ:
dL dθ = dϕ dθ ∇ϕL(Alg(θ,Dtrain),Dtest), (6)
notice that Alg(θ,Dtrain) is equivalent to ϕ. Component ∇ϕL(Alg(θ,Dtrain),Dtest) refers to firstorder gradients w.r.t. the task-specific parameter set ϕ (i.e., L → ϕ). The left component dϕdθ tracks parameter-to-parameter changes from ϕ to θ through Alg (i.e., ϕ → θ), which involves the computation of the Hessian matrix. As the estimation of the matrix dϕdθ requires extensive computational resources, we provide both first-order and secondorder implementations for MetaEvent. Zero-Shot MetaEvent: For the zero-shot evaluation, the learned initial parameter set should be directly evaluated on Dtest for an unseen task. Therefore, directly optimizing Equation (1) is not feasible for zero-shot event detection, as Dtrain is not provided. For optimization, however, training event types can be used for inner-loop optimization of the model. As such, we sample Dtrain and Dtest from different training tasks to improve the model generalization on unseen events. Specifically for training task T j , we optimize
min θ L(Alg(θ,Dtrain ∼ {T i}Mi=1,i ̸=j),Dtestj ), (7) where Dtrain is a disjoint training set sampled from {T i}Mi=1,i ̸=j . As a result, the model ‘learns to adapt’ to unseen events by optimizing on different training and evaluation sets. To improve the performance on unseen event types via Hungarian algorithm, we additionally design a contrastive loss term in the meta objective to learn class-separating features, which is introduced in Section 4.3. Few-Shot MetaEvent: In the few-shot event detection, we directly sample training tasks and optimize the model as in Equation (1). Similar to the zeroshot MetaEvent, the parameters are updated upon the tasks separately (i.e., ϕ) in each iteration based on the initial parameters θ. Then, the meta loss L and gradients w.r.t. θ are computed for each task using the updated ϕ and the evaluation sets. In our implementation, we adopt layer- and stepadaptive learning rates for inner-loop optimization and cosine annealing to improve the convergence of MetaEvent (Antoniou et al., 2018).

heading: 4.3 Meta Objective
text: We now introduce our training objective L for MetaEvent. Based on the proposed model in Sec-
tion 4.1, our loss function contains two classification losses: trigger classification loss Ltrigger and event classification loss Levent (i.e., negative log likelihood loss). To enlarge the inter-class event discrepancy for improved zero- and few-shot performance, we additionally propose a contrastive loss Lcon. based on the maximum mean discrepancy (MMD) (Gretton et al., 2012; Yue et al., 2021, 2022b,a). In particular, we measure the discrepancy between two different event types by estimating the MMD distance. MMD computes the distance between two event distributions using an arbitrary number of input features drawn from these event types. Mathematically, MMD distance between input features X and Y can be computed as:
D(X,Y ) = 1|X||X|
|X|∑
i=1
|X|∑
j=1
k(ψ(x(i)), ψ(x(j)))
+ 1
|Y ||Y |
|Y |∑
i=1
|Y |∑
j=1
k(ψ(y(i)), ψ(y(j)))
− 2|X||Y |
|X|∑
i=1
|Y |∑
j=1
k(ψ(x(i)), ψ(y(j))),
(8) where k represents the Gaussian kernel and ψ represents the feature mapping function defined by the transformer network (i.e., feature encoder).
Based on the MMD distance, we further compute the inter-class distances for all pairs of event types. Suppose Xi represents the set of triggeraware event features (i.e., concatenation of [v; t] as in Section 4.1) of the i-th class, the contrastive loss can be formulated for the N -way setting as:
Lcon. = − 1 N(N − 1) N∑
i=1
N∑
j=1,j ̸=i D(Xi,Xj),
(9) in which we compute N × (N − 1) pairs of interclass distances, such inter-class distances are maximized (by taking the negative values) in the meta objective function to encourage class-separating features in MetaEvent, and therefore improves both zero- and few-shot performance in event detection. Overall Objective: We now combine the mentioned terms into a single optimization objective for MetaEvent in Equation (10). In our objective function, Levent and Ltrigger represent the event and trigger classification loss (i.e., negative log likelihood loss), and Lcon. denotes the contrastive loss based on MMD. The overall objective con-
tains three terms and Lcon. is weighted by a scaling factor λc (to be chosen empirically):
L = Levent + Ltrigger + λcLcon.. (10)

heading: 4.4 Overall Framework
text: The overall framework of MetaEvent is presented in Figure 2. The right subfigure illustrates the proposed model that integrates attentive trigger features for prompt-based event detection. The left subfigure illustrates the meta training paradigm of MetaEvent, where the initial parameter θ is updated and evaluated upon sampled tasks using the proposed contrastive loss in Equation (10). Then the gradients w.r.t. θ can be computed (via second-order optimization or first-order approximation) to update the initial model. Unlike previous works (Schick and Schütze, 2021; Cong et al., 2021; Li et al., 2022b; Zhang et al., 2022b), we design a trigger-aware model for efficient training and inference on event detection tasks. Moreover, we propose a meta learning framework MetaEvent with a fine-grained contrastive objective function for zero- and few-shot event detection, which encourages generalizable and class-separating features across both seen and unseen event types.

heading: 5 Experiments

heading: 5.1 Settings
text: Model: Following previous work (Wang et al., 2021b; Li et al., 2022b), we select RoBERTa as the base model in MetaEvent (Liu et al., 2019). Evaluation: To validate the proposed method, we follow (Chen et al., 2021; Cong et al., 2021; Li et al., 2022b) to split the datasets into train, validation and test sets. For evaluation metrics, we adopt micro F1 score as main performance indicator. For the zero-shot setting, we additionally adopt adjusted mutual information (AMI) and Fowlkes Mallows score (FM) to evaluate clustering performance. See evaluation details in Appendix A. Datasets and Baselines: To examine MetaEvent performance, we adopt publicly available datasets FewEvent and MAVEN (Deng et al., 2020; Wang et al., 2020) and state-of-the-art baseline methods for comparison. For zero-shot baselines, we adopt SCCL (Zhang et al., 2021a), SS-VQ-VAE (Huang and Ji, 2020), BERT-OCL (Zhang et al., 2022b) and ZEOP (Zhang et al., 2022b). For the few-shot setting, we choose BERT-CRF (Devlin et al., 2019), PA-CRF (Cong et al., 2021), Prompt+QA (Li et al.,
2022b) and P4E (Li et al., 2022b) as baselines2. Dataset and baseline details are in Appendix A Implementation: We use the roberta-base variant in our implementation, our default model is trained with AdamW optimizer with zero weight decay and cosine annealing for meta learning rate of 1e − 5. For inner-loop optimization, we use layer- and step-adaptive learning rates with an intial learning rate of 1e − 3, where the model is updated 50 times in each task. We select the best model on the validation set for evaluation on the test set. For baseline methods, we follow the reported training procedure and hyperparameter settings from the original works unless otherwise suggested. More implementation and experiment details are provided in Appendix A.

heading: 5.2 Zero-Shot Results
text: We first report zero-shot results on all datasets in Table 1, which is divided into two parts by the used datasets. We perform 10-way event detection on unseen tasks from the disjoint test sets and report the evaluation results, the best results are marked bold, the second best results are underlined. We observe: (1) the zero-shot performance on FewEvent is comparatively higher than MAVEN for both baselines and MetaEvent, possibly due to the increased coverage of event domains in MAVEN. (2) MetaEvent performs the best on both datasets, in particular, MetaEvent achieves 35.9% average improvements on F1 compared to the second bestperforming method. (3) Despite lower performance on MAVEN, MetaEvent outperforms all baseline methods with up to 50.8% improvements on F1, suggesting the effectiveness of the proposed meta training algorithm in detecting unseen events.
We further study the effectiveness of the proposed contrastive loss quantitatively in zero-shot
2We additionally select the adapted ZEOP*, PA-CRF* and P4E* as improved variants of the baselines, see Appendix A.2.
experiments by varying scaling factor λc. Specifically, we choose λc from 0 to 5 and evaluate the performance changes on both datasets. The results are visually presented in Figure 3, from which we observe: (1) by applying the contrastive loss (i.e., λc ̸= 0) in the meta objective, the zero-shot performance consistently improves regardless of the choice of λc; (2) despite huge improvements, carefully chosen λc is required for the best zero-shot performance (up to ∼ 100% increases across all metrics). Overall, the contrastive objective proposed in MetaEvent is particularly effective in learning class-separating features, and thereby improves the zero-shot event detection performance.
We additionally present qualitative analysis of our zero-shot results by comparing the feature vi-
sualization (via T-SNE) with and without the proposed contrastive loss. We use color to represent different event types and present the visualization in Figure 4. With the contrastive loss (right subfigure), the model generates class-separating features on unseen events compared to MetaEvent trained without contrastive loss (left subfigure). Examples of the same event type are also more aggregated, showing improved clustering results, which can be combined with trigger predictions for identifying unseen event types. In sum, the proposed contrastive loss demonstrates effectiveness in zero-shot settings and consistently outperforms baselines.

heading: 5.3 Few-Shot Results
text: To examine the effectiveness of our method for both zero- and few-shot scenarios, we also perform few-shot experiments on both datasets. The 5-shot and 10-shot event detection experiment results are presented in Table 2, where all evaluation event types are used (10-way for FewEvent and 45-way for MAVEN3). We observe: (1) all baseline methods and MetaEvent achieve improved performance in few-shot event detection compared to the zero-shot results. For example, MetaEvent achieves 36.3% F1 improvement in 5-shot setting on FewEvent. (2) For MAVEN, the average performance of all baseline methods are comparatively
3Similar to (Chen et al., 2021), we perform binary classification for each of the event types in MAVEN as multiple event labels may exist on the same input context.
lower than FewEvent, indicating the challenge of few-shot classification with increased number of event types. (3) MetaEvent achieves the best results, outperforming the second best method in F1 by 3.0% (on FewEvent) and 38.1% (on MAVEN) on average across both few-shot settings. Overall, MetaEvent achieves state-of-the-art performance in event detection even only with few examples.
We now perform ablation studies in the few-shot setting to evaluate the effectiveness of the proposed component in MetaEvent. In particular, we remove the proposed attentive trigger features (i.e., trigger), trigger-aware soft verbalizer (i.e., verbalizer) and outer-loop optimization (i.e., meta learner) in sequence to observe the performance changes on FewEvent. The results are reported in Table 3. For all components, we observe performance drops when removed from MetaEvent. In the 5-shot setting, F1 score reduces by 1.6% and 12.9% respetively when removing the attentive trigger features and trigger-aware soft verbalizer. The results suggest that proposed components are effective for improving few-shot event detection.
Finally, we study the influence of prompt designs and report the results on FewEvent in Table 4. In particular, we select from prompt A: ‘A <mask> event’, B: ‘This text describes a <mask> event’, C: ‘This topic is about <mask>’ and D: ‘[Event: <mask>]’. From the results we observe: for 5-shot event detection, prompt A and B
perform the best while prompt B and D achieves better performance with 0.9644 and 0.9592 F1 in 10-shot setting. On average, prompt B outperforms all other prompt designs in the F1 metric, indicating that well-designed instructions may slightly improve few-shot event detection results.

heading: 6 Conclusion
text: In this paper, we design a meta learning framework MetaEvent for zero- and few-shot event detection. MetaEvent proposes to leverage attentive trigger features for efficient inference and predicts via a trigger-aware soft verbalizer. Moreover, the proposed MetaEvent trains the model to search for the optimal parameter set for quick adaptation to unseen event detection tasks. Extensive experiment results demonstrate the effectiveness of MetaEvent by consistently outperforming stateof-the-art methods on benchmark datasets in both zero- and few-shot event detection.

heading: 7 Limitations
text: While the proposed MetaEvent achieves significant improvements in both zero- and few-shot event detection, MetaEvent requires additional computational resources due to the layer- and step-adaptive learning rates and the outer-loop optimization, which may cause increased computational costs for training MetaEvent. Moreover, we have not investigated the benefits of task scheduling techniques and similarity-based meta learning in MetaEvent to fully explore the training event types. Consequently, we plan to study efficient meta learning with advanced training task scheduling for further improvements in zero- and few-shot event detection as future work.

heading: Acknowledgments
text: This research is supported in part by the National Science Foundation under Grant No. IIS-2202481, CHE-2105005, IIS-2008228, CNS-1845639, CNS1831669, U.S. DARPA KAIROS Program No. FA8750-19-2-1004 and U.S. DARPA AIDA Program No. FA8750-18-2-0014. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on.

heading: B Additional Results
text: In this section, we present additional results on zero-shot performance of MetaEvent. Specifically, we provide additional clustering metrics for
4For inner-loop optimization, the verbalizer weights are initialized with 10 times the base learning rate (i.e., inner-loop learning rate) for faster convergence.
MetaEvent on both datasets, with the results presented in Table 7. We adopt the following clustering metrics: (1) rand score (Rand Score); (2) adjusted rand score (Adjusted Rand); (3) normalized mutual information (Normalized MI) and (4) homegeneity score (Homogeneity Score). Surprisingly, the rand score and adjusted rand score demonstrates significant difference on MAVEN, suggesting disproportionate label distribution in MAVEN. For the rest metrics on both datasets, we observe similar magnitude of performance as reported in Section 5. Moreover, we present additional visualization results on FewEvent below with varying λc values. Compared to Figure 4, we observe that cluster aggregation worsens with reduced λc values.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: 7 Limitations

heading: 3 A2. Did you discuss any potential risks of your work?
text: 7 Limitations

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: 7 Limitations
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: 5 Experiments

heading: 3 B1. Did you cite the creators of artifacts you used?
text: 5 Experiments

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: 5 Experiments & A.1 Datasets

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 5 Experiments & A.1 Datasets

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? 5 Experiments & A.1 Datasets

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? 5 Experiments & A.1 Datasets
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. A.1 Datasets
C 3 Did you run computational experiments?

heading: 5 Experiments

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? 5 Experiments & A.3 Implementation Details
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 5 Experiments & A.3 Implementation Details

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 5 Experiments

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 5 Experiments & A.3 Implementation Details
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Consequently, we plan to study efficient meta learning with advanced training task scheduling for further improvements in zero- and few-shot event detection as future work