SlowGuess/NAACL_2022_OCR huggingface dataset:
To explore the effect of four contrastive samples, we conduct an ablation experiment by introducing them one by one to the ZEOP-woCL model, which takes no contrastive samples. The comparison of their performance is shown in Table 3. As all contrastive samples are added to the model, the performance improves on the unseen event detection task, but there is a performance decrease on the seen event detection task. This demonstrates that contrastive learning does help the model to
learn a better representation of unseen types, but its training objectives will conflict with the goal of supervised learning.
What's more, we also validate the effect of Wasserstein distance as the distance metric in the ordered contrastive loss. Table 4 indicates that the ZEOP model using Wasserstein distance performs better on unseen event detection task than ZEOP-Eu using Euclidean distance and ZEOP-Kl using Kullback-Leibler divergence. However, Euclidean distance shows better results on seen event detection task.
4.7 Qualitative analysis
For qualitative analysis, we visualize all unseen types samples in the dataset by t-SNE  ${}^{4}$  . As Figure 7 and Figure 8 show, the model with ordered contrastive learning (ZEOP and BERT-OCL) could learn a better representation of unseen types than other baselines. The larger the number of samples a type contains, the better cluster will be achieved. Overall, the classification of unknown event types is significantly more complex than known types, and only a small number of types with large sample sizes are correctly classified. Many clusters could not be mapped one-to-one to the actual event type with the absence of supervision learning. The model may use event elements such as subject and location as clustering features rather than the event type itself.
4.8 Hyperparameter sensitivity analysis
The number of clusters is an essential hyperparameter in the clustering problem. In order to explore
4https://scikit-learn.org/stable/modules/generated/ sklearn.manifold.TSNE.html
Figure 7: Visualization of unseen types on ACE-2005. Each color indicates a ground truth type.
Figure 8: Visualization of unseen types on FewShorED. Each color indicates a ground truth type.
the impact of parameter unseen types  $l$  on the performance of ZEOP, we conduct a hyperparameter experiment by setting  $l$  to the 1, 2, 3, 4, and 5 times of the actual number of unseen types. As shown in Figure 9, ZEOP suffers a slight performance drop when unseen type numbers increase. The model is more likely to classify the same event type into different clusters. In practice, properly estimating the approximate number range of unknown event types will help the model to achieve a better performance.
In order to solve the problem of zero-shot event detection, this paper proposes an end-to-end model named Zero-Shot Event Detection Based on Ordered Contrastive Learning and Prompt-Based Prediction. By creatively introducing multiple contrastive samples with different similarities, the contrastive loss is extended from pairwise comparison to list-wise comparison. Therefore, the model could learn a better representation across instance-level and class-level. Meanwhile, the prompt-based prediction is utilized to identify event trigger words without relying on heuristic rules. Experiments demonstrate that our method can significantly improve the accuracy of identifying unseen event types while keeping the ability to classify seen event types. Future research should consider the po
Figure 9: Effects of unseen types number.
tential effects of the initialization of the prototype matrix in the prototypical network more carefully. A better initial value may reduce the need for large training samples and speed up model training.
This work was supported by NSFC grants (No. 61972155), the Science and Technology Commission of Shanghai Municipality (20DZ1100300) and the Open Project Fund from Shenzhen Institute of Artificial Intelligence and Robotics for Society, under Grant No. AC01202005020, Shanghai Trusted Industry Internet Software Collaborative Innovation Center.
Tom B. Brown, Benjamin Mann, Nick Ryder, and etc. 2020. Language models are few-shot learners. In NeurIPS.
Shumin Deng, Ningyu Zhang, Jiaojian Kang, Yichi Zhang, Wei Zhang, and Huajun Chen. 2020. Meta-learning with dynamic-memory-based prototypical network for few-shot event detection. In WSDM, pages 151-159.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In *NAACL*, pages 4171–4186.
Marzieh Fadaee and Christof Monz. 2018. Backtranslation sampling by targeting difficult words in neural machine translation. In EMNLP, pages 436-446.
Tianyu Gao, Adam Fisch, and Danqi Chen. 2021a. Making pre-trained language models better few-shot learners. In ACL, pages 3816-3830.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021b. Simcse: Simple contrastive learning of sentence embeddings.
Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006. Dimensionality reduction by learning an invariant mapping. In CVPR, pages 1735-1742.
Lifu Huang and Heng Ji. 2020. Semi-supervised new event type induction and event detection. In EMNLP, pages 718-724.
Lifu Huang, Heng Ji, Kyunghyun Cho, Ido Dagan, Sebastian Riedel, and Clare R. Voss. 2018. Zero-shot transfer learning for event extraction. In ACL, pages 2160-2170.
Soheil Kolouri, Kimia Nadjahi, Umut Simsekli, Roland Badeau, and Gustavo K. Rohde. 2019. Generalized sliced Wasserstein distances. In NeurIPS, pages 261-272.
Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020. A joint neural model for information extraction with global features. In ACL, pages 7999-8009.
Lajanugen Logeswaran and Honglak Lee. 2018. An efficient framework for learning sentence representations. In ICLR.
Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In ICLR.
Qing Lyu, Hongming Zhang, Elior Sulem, and Dan Roth. 2021. Zero-shot event extraction via transfer learning: Challenges and insights. In ACL, pages 322-332.
Thien Huu Nguyen and Ralph Grishman. 2018. Graph convolutional networks with argument-aware pooling for event detection. In AAAI, pages 5900-5907.
Adam Paszke, Sam Gross, Francisco Massa, and etc. 2019. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, pages 8024-8035.
Timo Schick and Hinrich Schütze. 2021. It's not just size that matters: Small language models are also few-shot learners. In *NAACL*, pages 2339-2352.
Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical networks for few-shot learning. In NeurIPS, pages 4077-4087.
David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019. Entity, relation, and event extraction with contextualized span representations. In EMNLP, pages 5783-5788.
Yijun Wang, Changzhi Sun, Yuanbin Wu, Junchi Yan, Peng Gao, and Guotong Xie. 2020. Pre-training entity relation encoder with intra-span and inter-span information. In EMNLP, pages 1692-1705.
Ziqi Wang, Xiaozhi Wang, Xu Han, Yankai Lin, Lei Hou, Zhiyuan Liu, Peng Li, Juanzi Li, and Jie Zhou. 2021. CLEVE: contrastive pre-training for event extraction. In ACL, pages 6283-6297.
Thomas Wolf, Lysandre Debut, Victor Sanh, and etc. 2020. Transformers: State-of-the-art natural language processing. In EMNLP, pages 38-45. Association for Computational Linguistics.
Congle Zhang, Stephen Soderland, and Daniel S. Weld. 2015. Exploiting parallel news streams for unsupervised event extraction. Trans. Assoc. Comput. Linguistics, 3:117-129.
Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang-Wen Li, Henghui Zhu, Kathleen R. McKeown, Ramesh Nallapati, Andrew O. Arnold, and Bing Xiang. 2021a. Supporting clustering with contrastive learning. In NAACL, pages 5419-5430.
Hongming Zhang, Haoyu Wang, and Dan Roth. 2021b. Zero-shot label-aware event trigger and argument classification. In ACL, pages 1331-1340.