Constructing U yghur Name Entity Recognition System using Neural Machine Translation Tag Projection (2020)

Abstract: Although named entity recognition achieved great success by introducing the neural networks, it is challenging to apply these models to low resource languages including Uyghur while it depends on a large amount of annotated training data. Constructing a well-annotated named entity corpus manually is very time-consuming and labor-intensive. Most existing methods based on the parallel corpus combined with the word alignment tools. However, word alignment methods introduce alignment errors inevitably. In this paper, we address this problem by a named entity tag transfer method based on the common neural machine translation. The proposed method marks the entity boundaries in Chinese sentence and translates the sentences to Uyghur by neural machine translation system, hope that neural machine translation will align the source and target entity by the self-attention mechanism. The experimental results show that the Uyghur named entity recognition system trained by the constructed corpus achieve good performance on the test set, with 73.80% F1 score(3.79% improvement by baseline)

## introduction named entity recognition (ner) is a task of identifying named entities (nes), especially person names (per), location names (loc), organization names (org), and classifying them into some pre-defined target entity classes @xcite . ner is essential to many natural language processing (nlp) tasks such as relation extraction @xcite , event detection @xcite , knowledge graph construction @xcite and so on. although the ner achieves great success by the introduction of the advanced neural networks @xcite @xcite @xcite @xcite @xcite , these methods are highly dependent on a large amount of annotated training data, and thus challenging to apply these models to low resource languages including uyghur. constructing a well-annotated ne corpus manually is very time-consuming and labor-intensive. instead, cross-lingual transfer is an effective solution, which addresses this challenge by transferring knowledge from a high-resource source language with abundant entity labels to a low-resource target language with few or no labels. according to the resource availability of the target language, different types of ner methods are proposed, such as bilingual parallel corpus based tag projection @xcite @xcite @xcite , crosslingual word embedding @xcite @xcite , cross-lingual wikification @xcite @xcite or multi-task learning @xcite . as a low resource language, uyghur has no well-annotated corpus available for ner, but it is easy to get uyghur-chinese bilingual parallel corpus as uyghur-chinese machine translation is an important task of china conference on machine translation (ccmt). a common way of constructing ner corpus for the language which has a bilingual parallel corpus is using off-the-shelf ner tool in the source language to get entity annotations and transfer them to target language combing with the automatic computational linguistics word alignment. although some researchers have also applied this method to transfer ne annotations from chinese to uyghur and achieved remarkable results @xcite , these pipeline methods inevitably introduce errors from the source language, including errors from ner tools and automatic word alignment. figure 1 in this paper, we address these challenges by a ne annotation transfer method based on neural machine translation (nmt). given an uyghur-chinese parallel corpus, first, we train a general-purpose chinese-uyghur nmt system using the parallel corpus. then, add the ne boundary information directly to the source chinese sentence by multiple off-the-shelf ner tools. finally, translate the chinese sentences with entity boundary to uyghur language using the pre-trained nmt system, we hope that nmt will align the source and target entity by the self-attention mechanism. our method can be illustrated by the following example provided in figure 2 . the main advantages of our method are used multi ner tools in the source language to minimize annotation errors and use general-purpose nmt without adding new tokens to indicate ne boundary in the parallel corpus, thus no need to annotate any training data manually. 2 related work named entity recognition: ner is typically framed as a task of sequence labeling which aims at automatic detection of nes in free text @xcite . crf, svm, and perceptron models with hand-crafted features are applied in early works @xcite @xcite c c l 2 0 2 0 2015). with the great advantages of deep neural networks, research focuses on the neural network-based methods that need less feature engineering and domain knowledge @xcite @xcite . @xcite proposed a feed-forward neural network with a fixedsized window for each word, which failed in considering useful relations between long-distance words. to overcome this limitation, @xcite presented a bidirectional lstm-cnns architecture that automatically detects word and character-level features. @xcite further extended it into bidirectional lstm-cnns-crf architecture, where the crf module was added to optimize the output label sequence. transfer learning for ner: low-resource languages often suffer from a lack of annotated corpora to estimate high-performing neural network models for many nlp tasks. transfer learning is an efficient way to bridge the gap across languages. transfer learning methods for ner can be divided into two types: parallel corpora based and shared representation based transfer. early works mainly focus on parallel corpora to projecting information from high-resource languages to low-resource languages @xcite @xcite @xcite . @xcite al. (2010) and wang et al. (2013) 2018 ) and @xcite shows that jointly training on multiple tasks or languages helps improve performance. different from transfer learning methods, multi-task learning aims at improving the performance of all the resources instead of low resource only. ## conclusion aiming at the lack of uyghur named entity recognition training corpus, this paper proposes a crosslanguage named entity tag transfer method based on general machine translation and entity boundary token. first obtains the named entity tags of chinese sentences in chinese-uyghur parallel corpus through a variety of chinese named entity recognition tools and uses tag fusion strategies to fuse multi-source c c l 2 0 2 0 tags, then select appropriate special symbols to surround the entities and uses chinese-uyghur neural machine translation system to translate the chinese sentences to uyghur. finally, the uyghur stems and affixes merge method is used to obtain a high-quality uyghur named entity recognition corpus. the uyghur ner system trained with this corpus achieved good performance, which was 3.79% points higher than the baseline system.