Analysis of Zero-Shot Crosslingual Learning between E nglish and K orean for Named Entity Recognition (2021)

Abstract: This paper presents a English-Korean parallel dataset that collects 381K news articles where 1,400 of them, comprising 10K sentences, are manually labeled for crosslingual named entity recognition (NER). The annotation guidelines for the two languages are developed in parallel, that yield the inter-annotator agreement scores of 91 and 88% for English and Korean respectively, indicating sublime quality annotation in our dataset. Three types of crosslingual learning approaches, direct model transfer, embedding projection, and annotation projection, are used to develop zero-shot Korean NER models. Our best model gives the F1-score of 51% that is very encouraging, considering the extremely distinct natures of these two languages. This is pioneering work that explores zero-shot cross-lingual learning between English and Korean and provides rich parallel annotation for a core NLP task such as named entity recognition.

## introduction crosslingual representation learning aims to derive embeddings for words (or sentences) from multiple languages that can be projected into a shared vector space @xcite @xcite . one important application of crosslingual embeddings has been found for transferring models trained on a high-resource language to a low-resource one @xcite @xcite . the latest multilingual transformer encoders such as bert @xcite and xlm @xcite have made it possible to develop robust crosslingual models through zero-shot learning that requires no labeled training data on the target side @xcite @xcite . however, these approaches tend not to work as well for languages whose words cannot be easily aligned. our team is motivated to create a rich crosslingual resource between english and korean, which are largely different in nature as english is known to be rigid-order, morphologically-poor, and head-initial whereas korean is flexible-order, morphologicallyrich, and head-final @xcite @xcite . creation of a high quality parallel dataset to facilitate crosslingual research can reduce the gap between these two languages, and advance nlp techniques in both languages. this paper provides a comprehensive analysis of crosslingual zero-shot learning in english and korean. we first create a new dataset comprising a large number of parallel sentences and annotate them for named entity recognition (ner; sec. 3). we then adapt the crosslingual approaches and build ner models in korean through zeroshot learning (sec. 4). all models are experimented on our dataset and thoroughly compared to evaluate the feasibility of this work (sec. 5). our results are promising although depicting few challenges in zero-shot learning for english and korean (sec. 6). the contributions of this work can be summarized as follows: ## related work for crosslingual representation alignment, @xcite and @xcite suggested orthogonality constraints on the embedding transformation that led to better quality translation. @xcite derived a context-aware crosslingual mapping from a parallel corpus using word alignment. @xcite aligned for cross-lingual ner, @xcite presented weakly supervised crosslingual models using annotation and representation projection. @xcite made an empirical analysis of how sequential order and multilingual embeddings are used in crosslingual ner. @xcite presented multilingual transfer models that used few-shot learning adapting supervising bea, ranking and retraining for massive transfer. @xcite and @xcite directly transfers the ner model trained on the source language to the target language using crosslingual representations from multilingual encoders (direct model transfer). ## analysis 6.1 error distribution given the results of the best models for embedding and annotation projection approaches (section 5.6), a total of 105 parallel sentences (15 pairs per news section) are randomly selected for error analysis. 7 na ne wr wl wrl ep 7.64% 35.03% 28.02% 14.01% 15.28% ap 10.43% 28.83% 28.83% 23.31% 8.59% korean named entities describing cardinal are more prone to fall under wrong range than the other 7 detailed descriptions of the five error types are provided in appendix a.5. ## conclusion this paper presents a multilingual dataset that allows researchers to conduct crosslingual research between english and korean. our dataset contains high-quality annotation of named entities on parallel sentences from seven popular news sections. given this dataset, korean ner models are built by zero-shot learning using multilingual encoders. our data selection scheme for annotation projection significantly improves the ner performance although it is still suboptimal. our error analysis depicts unique characteristics in korean that make it hard for zero-shot learning, challenges that we need to overcome in the future work. • date: absolute or relative dates or periods. the period should last longer than 'time'. general expressions of dates are included too such as 'few months', 'that day', 'next season' and 'first quarter'.