Two Languages Are Better than One: Bilingual Enhancement for C hinese Named Entity Recognition (2022)

Abstract: Chinese Named Entity Recognition (NER) has continued to attract research attention. However, most existing studies only explore the internal features of the Chinese language but neglect other lingual modal features. Actually, as another modal knowledge of the Chinese language, English contains rich prompts about entities that can potentially be applied to improve the performance of Chinese NER. Therefore, in this study, we explore the bilingual enhancement for Chinese NER and propose a unified bilingual interaction module called the Adapted Cross-Transformers with Global Sparse Attention (ACT-S) to capture the interaction of bilingual information. We utilize a model built upon several different ACT-Ss to integrate the rich English information into the Chinese representation. Moreover, our model can learn the interaction of information between bilinguals (inter-features) and the dependency information within Chinese (intra-features). Compared with existing Chinese NER methods, our proposed model can better handle entities with complex structures. The English text that enhances the model is automatically generated by machine translation, avoiding high labour costs. Experimental results on four well-known benchmark datasets demonstrate the effectiveness and robustness of our proposed model.

## introduction one way to utilize external resources is to perform chinese ner with bilingual constraints. previous works @xcite have demonstrated that the joint use of bilingual information in chinese and english can significantly improve performance in the chinese ner task. in addition, incorporating vocabulary knowledge has also become a promising solution @xcite @xcite ). what's more, several studies were proposed to exploit information from other modalities to supplement the representation of chinese text @xcite @xcite . however, despite the success of the abovementioned methods in chinese ner by introducing external information, these methods still have the following limitations. first, the external resources utilized by these methods are mainly obtained manually, which increases the cost significantly. second, existing methods for boosting ner with bilingual constraints @xcite rely on bilingual word alignment information and both chinese and english sentences need to be manually annotated, which limits its usage. third, as the state-of-the-art approaches based on deep neural networks for chinese ner, lexical enhancement methods and multimodal methods still fail to effectively handle entities with complex composition structures, which, however, are frequently observed in chinese ner tasks. the example in figure 1 illustrates one of such dilemmas in chinese ner. in this example, due to the complex component structure, the org entity "美 国'亚特兰蒂斯'号航天飞机(the us space shuttle atlantis)" tends to be incorrectly labeled by the ner model as a loc entity "美国(usa)" and an org entity "'亚特兰蒂斯'号航天飞机(space shuttle atlantis)". however, it is encouraging that the clues of the english expression, such as the preposition "on", will potentially alleviate this type of incorrect labeling that often occurs in chinese ner tasks. to address the above issues, in this work, we propose to boost the performance of chinese ner with the unlabelled english text translated from the corresponding chinese text. english texts are automatically generated through the publicly available neural machine translation api without any extra human labor all the way through. besides, considering the chinese texts and corresponding english translations as two different modalities, we perform bilingual enhancement of chinese ner with multimodal ner approach. furthermore, based on the fact that a word in the text will only be strongly correlated with a small fraction of words in the translated text, we propose a bilingual interaction enhancement model based on adapted cross-transformers with global sparse attention (short for act-s). the interaction of bilingual information as well as intra-linguistic interaction are taken into account in the our model. the primary contributions of this work can be summarized as follows: (1) we improve the perfor-mance of chinese ner by bilingual enhancement, based on the unlabeled translated english text automatically generated using the neural machine translation api. to the best of our knowledge, this is the first end-to-end ner method that effectively exploits bilingual information. (2) we further propose the neural module called adapted cross-transformers with global sparse attention to simultaneously model bilingual interactions and inter-lingual interactions. so far as we know, it is the first attempt to use global sparse attention mechanisms for multimodal information interaction. (3) experimental results on four chinese ner datasets show that our proposed model achieves superior performance to other strong baseline models. ## conclusion in this paper, we propose an adapted cross-transformers with global sparse attention (act-s) module to explore bilingual interaction information to improve the performance of the chinese ner task. several parameter independent act-ss are employed in our work to capture the rich information in both english and chinese. we evaluate the proposed model on four chinese ner datasets and the experimental results illustrate that our method achieves significant and consistent improvement compared to other baselines. in the fu-ture, we will explore how to improve chinese ner with features from languages other than english and extend our model to other sequence labelling tasks.