CROP : Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation (2022)

Abstract: Named entity recognition (NER) suffers from the scarcity of annotated training data, especially for low-resource languages without labeled data. Cross-lingual NER has been proposed to alleviate this issue by transferring knowledge from high-resource languages to low-resource languages via aligned cross-lingual representations or machine translation results. However, the performance of cross-lingual NER methods is severely affected by the unsatisfactory quality of translation or label projection. To address these problems, we propose a Cross-lingual Entity Projection framework (CROP) to enable zero-shot cross-lingual NER with the help of a multilingual labeled sequence translation model. Specifically, the target sequence is first translated into the source language and then tagged by a source NER model. We further adopt a labeled sequence translation model to project the tagged sequence back to the target language and label the target raw sentence. Ultimately, the whole pipeline is integrated into an end-to-end model by the way of self-training. Experimental results on two benchmarks demonstrate that our method substantially outperforms the previous strong baseline by a large margin of +3 7 F1 scores and achieves state-of-the-art performance.

## introduction named entity recognition (ner) focuses on recognizing entities from raw text into predefined types @xcite @xcite @xcite , which is an essential component for downstream natural language processing (nlp) tasks, such as information retrieval @xcite and question answering @xcite . however, most of the existing approaches are highly dependent on the annotated training data and do not perform well in low-resource languages. zero-shot cross-lingual ner aims to address this challenging problem by transferring knowledge from the high-resource source language with lots amounts of annotated corpora to those languages without any labeled data @xcite . some methods leverage the cross-lingual representations @xcite , where the ner model is trained on the labeled corpus of the source language and then directly evaluated on target languages. due to the success of multilingual pretrained language models @xcite , these model-based transfer methods have shown a significant improvement in cross-lingual ner. another line of research is the data-based transfer @xcite , which adopts word-to-word translation to project the cross-lingual ner labels. for example, @xcite employ a multilingual translation model with placeholders for label projection. nevertheless, these methods are still limited by weak entity projection and do not leverage the unlabeled corpora in target languages. along the line of using the multilingual model to encourage knowledge transfer among different languages, we propose a cross-lingual entity projection (crop) framework to leverage the unlabeled corpora of target languages, which is supported by a strong multilingual labeled sequence translation model guided by multiple bilingual corpora and the corresponding phrase-level alignment information. in figure 1 , the unlabeled target sentence is forward translated to the source language and tagged by the source ner model. then, we use the labeled sequence translation model to backtranslate the annotated sentence to the target language. given the target annotated sentence, we project the entity labels of "gothenburg" to the target raw sentence through lexical matching. finally, we use self-training to integrate the pipeline into an end-to-end ner model. specifically, we construct multilingual corpora to train the labeled sequence translation model, where the aligned spans of the sentence pair are both surrounded by the boundary symbols. we conduct experiments on two benchmarks, including xtreme-40 of 40 languages and conll-5 of 5 languages. experimental results show that our method reaches new state-of-the-art results. furthermore, we also evaluate the performance of the multilingual labeled sequence translation model and visualize multilingual sentence representations. analytic results demonstrate that our method can transfer knowledge among even distant languages. ## zero-shot cross-lingual ner given the source ner model θ src ner only trained on the source ner dataset and the target raw sentence @xmath0, . . . , x m ) with m words, the zero-shot cross-lingual ner aims to identify each word of target language to predefined types and then obtains the labels @xmath1, . . . , t m ). the problem definition of zero-shot cross-lingual ner is described as: ## related work cross-lingual ner named entity recognition (ner) identifying the named entities into the predefined types has achieved huge progress in recent years @xcite @xcite @xcite @xcite . cross-lingual ner model supporting multiple languages is a key component for various downstream natural language processing (nlp) tasks, including information retrieval @xcite , question answering @xcite , and co-reference resolution @xcite . the previous works can be classified into two categories including modelbased transfer @xcite and data-based transfer methods @xcite @xcite . the model-based transfer methods benefit from the state-of-the-art cross-lingual pretrained model @xcite and the aligned cross-lingual word embeddings @xcite . @xcite emphasizes that the model-based transfer and data-based transfer methods are complementary to each other. ## conclusion in this work, we propose a novel zero-shot crosslingual ner framework with a multilingual labeled sequence translation model advised by multilingual corpora and phrase-level alignment pairs. the knowledge of the source ner model is effectively transferred to target languages by a round-trip translation and label projection. in this way, the multilingual translation model plays the role of the bridge to transfer knowledge from source languages to low-resource target languages. experimental results evaluated on the conll-5 and xtreme-40 benchmarks demonstrate the effectiveness of our method compared to the strong baselines.