Abstract: In recent years, great success has been achieved in the field of natural language processing (NLP), thanks in part to the considerable amount of annotated resources. For named entity recognition (NER), most languages do not have such an abundance of labeled data, so the performances of those languages are comparatively lower. To improve the performance, we propose a general approach called Back Attention Network (BAN). BAN uses translation system to translate other language sentences into English and utilizes the pre-trained English NER model to get task-specific information. After that, BAN applies a new mechanism named back attention knowledge transfer to improve the semantic representation, which aids in generation of the result. Experiments on three different language datasets indicate that our approach outperforms other state-of-the-art methods.

Introduction: Named entity recognition (NER) is a sequence tagging task that extracts the continuous tokens into specified classes, such as person names, organizations and locations. Current state-of-the-art approaches for NER usually base themselves on long short-term memory recurrent neural networks (LSTM RNNs) and a subsequent conditional random field (CRF) to predict the sequence labels BIBREF0 . Performances of neural NER methods are compromised if the training data are not enough BIBREF1 . This problem is severe for many languages due to a lack of labeled datasets, e.g., German and Spanish. In comparison, NER on English is well developed and there exist abundant labeled data for training purpose. Therefore, in this work, we regard English as a high-resource language, while other languages, even Chinese, as low-resource languages. There is an intractable problem when leveraging English NER system for other languages. The sentences with the same meaning in different languages may have different lengths and the positions of words in these sentences usually do not correspond. Previous work such as BIBREF2 used each single word translation information to enrich the monolingual word embedding. To our knowledge, there is no approach that employs the whole translation information to improve the performance of the monolingual NER system. To address above problem, we introduce an extension to the BiLSTM-CRF model, which could obtain transferred knowledge from a pre-trained English NER system. First, we translate other languages into English. Since the proposed models of BIBREF3 and BIBREF4 , the performance of attention-based machine translation systems is close to the human level. The attention mechanism can make the translation results more accurate. Furthermore, this mechanism has another useful property: the attention weights can represent the alignment information. After translating the low-resource language into English, we utilize the pre-trained English NER model to predict the sentences and record the output states of BiLSTM in this model. The states contain the semantic and task-specific information of the sentences. By using soft alignment attention weights as a transformation matrix, we manage to transfer the knowledge of high resource language — English to other languages. Finally, using both word vectors and the transfer knowledge, we obtain new state-of-the-art results on four datasets.

Question: Which translation system do they use to translate to English?
Answer: Attention-based translation model with convolution sequence to sequence model