Few-Shot Cross-Lingual Learning for Event Detection (2022)

Abstract: Cross-Lingual Event Detection (CLED) models are capable of performing the Event Detection (ED) task in multiple languages. Such models are trained using data from a source language and then evaluated on data from a distinct target language. Training is usually performed in the standard supervised setting with labeled data available in the source language. The Few-Shot Learning (FSL) paradigm is yet to be explored for CLED despite its inherent advantage of allowing models to better generalize to unseen event types. As such, in this work, we study the CLED task under an FSL setting. Our contribution is threefold: first, we introduce a novel FSL classification method based on Optimal Transport (OT); second, we present a novel regularization term to incorporate the global distance between the support and query sets; and third, we adapt our approach to the cross-lingual setting by exploiting the alignment between source and target data. Our experiments on three, syntactically-different, target languages show the applicability of our approach and its effectiveness at improving the cross-lingual performance of few-shot models for event detection.

## introduction event detection (ed) is a significant sub-task within the larger task of information extraction (ie) in natural language processing (nlp). its core purpose is to identify the words, or phrases, that most clearly express the occurrence of an event, known as event triggers, and to correctly categorize them into a discrete set of classes. for instance, in the sentence: frank purchased his dream house yesterday. the word "purchased" should be identified by an ed system as the trigger of a transaction:transfer-ownership event type foot_0 . event detection is a highly active research area which has been lately dominated by deep-learning-based approaches @xcite @xcite @xcite @xcite . most of these works use the standard supervised learning paradigm in which lots of labeled data is required during training. however, a significant limitation of models trained in this manner is their inability to properly generalize to new event types that were unobserved during training @xcite . few-shot learning: in contrast to the supervised approach, few-shot learning (fsl) proposes a training setting in which a model must quickly learn new concepts from just a few examples, similar to how humans can learn to detect and identify new objects after having observed only a couple of instances. during an fsl training iteration, a model is given a support set and a query set, each of which contains only a handful of examples for a set of classes. then, the model is trained to predict the classes for the query samples based on the labeled support samples. under these constrained training settings, supervised training easily results in model overfitting due to the limited availability of training data. furthermore, in fsl, a model is evaluated on its ability to generalize to new, unobserved types. to achieve this, during testing an fsl model is provided with new support and query sets whose samples belong to entirely new classes never observed during training. typical fsl approaches consist of obtaining a vector representation for each sample and then performing classification based on the distance between such vectors, e.g., matching networks @xcite , relation networks @xcite , and prototypical networks @xcite . the key differences between these approaches often come down to the way the sample representations are generated, and how the distance between such representations is determined. fsl training allows a model to easily extend to new classes as it only needs to see a few labeled examples in order to successfully classify them. fsl has been applied successfully for many tasks. recently, there have been several efforts that explore event detection under a few-shot learning setting (fsled) @xcite @xcite @xcite . cross-lingual event detection: cross-lingual learning (cll) is a paradigm that aims at transferring the knowledge from one language to another @xcite . cll can help overcome the lack of data availability that plagues many languages and allow for the creation of nlp-based tools that can benefit their communities. as such, cross-lingual event detection (cled) aims at detecting and classifying event triggers with the added complexity of operating on two separate languages. these two languages are referred to as source and target, respectively. in standard zero-shot training, a cled model is trained using labeled data belonging to the source language exclusively. then, at testing time, data from the target language is used to evaluate the model's performance @xcite @xcite . contributions: a proper effort on cled under fsl conditions has yet to be explored despite the potential advantages it could contribute to crosslingual models. hence, we recognize this opportunity and propose the novel few-shot cross-lingual event detection (fscled) task to integrate these two settings. we consider the following as our main contributions: to provide foundation for future research, we first evaluate the performance of representative fsl methods @xcite @xcite in this task. the rest of the paper is organized as follows: section 2 provides a formal definition for fscled task, section 3 describes the details our proposed approach, section 4 presents the results of our experiments, and finally, we present our conclusions in section 6. ## model as done in prior fsl models for ed @xcite , our model for fscled involves two main components: an encoder e and a classifier c. ## related work event detection has been thoroughly studied over the years. early ed efforts were based on handcrafted features @xcite @xcite @xcite @xcite @xcite . more recently, deep learning techniques such as recurrent neural networks @xcite @xcite , convolutional neural networks @xcite @xcite , graph convolutional networks @xcite , adversarial networks @xcite , pre-trained language models @xcite @xcite @xcite , and generative models @xcite have been prevalent. nevertheless, these works study ed under a supervised or semisupervised setting. alternatively, ed was recently formulated as a few-shot task @xcite . in a short time, several methods have been proposed using a variety of techniques such as meta-learning @xcite , cross-task prototyping @xcite , dependency graphs @xcite , causal modeling @xcite , and label dependency via conditional random fields @xcite . previous works on cross-lingual ed generally make use of cross-lingual resources such as bilingual dictionaries or parallel corpora @xcite to address the differences between languages. more recent approaches exploit the language-invariant characteristics of pre-trained multilingual language models @xcite along with complementary features such as label dependency (m'hamdi et al., 2019), verb-class knowledge @xcite , and class-aware cross-lingual alignment @xcite . optimal transport has also been recently used in cross-lingual settings for information extraction tasks such as event co-reference resolution @xcite and event detection @xcite . however, the amalgamation of the few-shot and cross-lingual settings creates unique challenges that have not been tackled by any related work. consequently, our proposed use of ot differs from related works as it addresses the global alignment between the support and query sets for few-shot learning and between source and target languages for the cross-lingual setting. ## conclusion we explore a novel few-shot cross-lingual setting for event detection that combines the limited training-data conditions of fsl with zero-shot cross-lingual transfer learning. we provide the performance of typical fsl models as the foundations for future research. more importantly, we introduce a novel method for fscled that leverages the optimal alignment between query and support sets obtained via ot to perform fsl classification. our method is complemented by two additional regularization terms that aim at integrating the global distance between support and query sets and fostering language-invariant representations by leveraging unlabeled data in the target language. our experiments on three target languages demonstrate the advantages of our approach and its general applicability to traditional fsl models. as future work, we intend to extend our method to other related tasks in ie such as relation extraction.