D ual NER : A Dual-Teaching framework for Zero-shot Cross-lingual Named Entity Recognition (2022)

Abstract: We present DualNER, a simple and effective framework to make full use of both annotated source language corpus and unlabeled target language text for zero-shot cross-lingual named entity recognition (NER). In particular, we combine two complementary learning paradigms of NER, i.e., sequence labeling and span prediction, into a unified multi-task framework. After obtaining a sufficient NER model trained on the source data, we further train it on the target data in a dual-teaching manner, in which the pseudo-labels for one task are constructed from the prediction of the other task. Moreover, based on the span prediction, an entity-aware regularization is proposed to enhance the intrinsic cross-lingual alignment between the same entities in different languages. Experiments and analysis demonstrate the effectiveness of our DualNER.

## introduction aiming at classifying entities in un-structured text into pre-defined categories, named entity recognition (ner) is an indispensable component for various downstream neural language processing applications such as information retrieval @xcite and question answering @xcite . current supervised methods have achieved great success with sufficient manually labeled data, but the fact remains that most of the annotated data are constructed for high-resource languages like english and chinese, posing a big challenge to lowresource scenarios @xcite . to address this issue, zero-shot cross-lingual ner is proposed to transfer knowledge of ner from high-resource languages to low-resource languages. the knowledge can be acquired in either of the following two ways: 1) from aligned cross-lingual word representations or multilingual pretrained encoder fine-tuned on high-resource languages @xcite . 2) from translated target language data with label projection @xcite @xcite . these two kinds of methods can be unified into a knowledge distillation (kd) framework, to further improve the cross-lingual ner performance @xcite . though widely used, the transfer process still suffers from poor translation quality, label projection error and over-fitting of large-scale multilingual language models. in this paper, we present a simple and effective framework, named dualner, alleviating the above problems from a different angle. we combine two popular complementary learning paradigms of ner, sequence labeling and span prediction, into a single framework. specifically, we first train a teacher ner model by jointly exploiting sequence labeling and span prediction with the annotated source language corpus. unlike the previous kd-based methods that produce pseudo labels for the corresponding paradigms, we propose a dualteaching strategy to make the two paradigms complement each other. more concretely, the model prediction for sequence labeling is used to construct the pseudo-labels for span prediction and vice versa. furthermore, we propose a multilingual entity-aware regularization forcing same entities in different languages to have similar representations. by doing this, the trained model is able to leverage the intrinsic cross-lingual alignment across different languages to enhance the cross-lingual transfer ability. experiments and analysis conducted on xtreme for 40 target languages well validate the effectiveness of dualner foot_0 . ## framework recently, the dominate paradigm for ner shifts from sequence labeling @xcite @xcite @xcite to span-level prediction @xcite @xcite @xcite . we combine these two formulations into a unified multitask framework for complementarity. as shown in figure 1 , our dualner consists of three major modules: token representation layer, sequence labeling layer, and span prediction layer. ## conclusion in this paper, we propose a simple and effective dual-teaching framework, coined dualner, for zero-shot cross-lingual named entity recognition. in particular, dualner makes full use of the ex-changeability of the labels in span prediction and sequence labeling, and generates abundant pseudo data for available labeled and unlabeled data. experiments and analysis validate the effectiveness of our dualner.