Named Entity-Factored Transformer for Proper Noun Translation (2021)

Abstract: Subword-based neural machine translation decreases the number of out-of-vocabulary (OOV) words and also keeps the translation quality if input sentences include OOV words. The subword-based NMT decomposes a word into shorter units to solve the OOV problem, but it does not work well for non-compositional proper nouns due to the construction of the shorter unit from words. Furthermore, the lack of translation also occurs in proper noun translation. The proposed method applies the Named Entity (NE) fea-ture vector to Factored Transformer for accurate proper noun translation. The proposed method uses two features which are input sentences in subwords unit and the feature obtained from Named Entity Recognition (NER). The pro-posed method improves the problem of non-compositional proper nouns translation included a low-frequency word. According to the experiments, the proposed method using the best NE feature vector outperformed the baseline sub-word-based transformer model by more than 9.6 points in proper noun accuracy and 2.5 points in the BLEU score.

## introduction recent advances in neural machine translation (nmt) have made machine translation (mt) systems useful in practical applications. however, translation of proper nouns still remains difficult in spite of its importance in practice. proper nouns are sometimes processed as out-of-vocabulary (oov) words in mt systems due to the limitation of the vocabulary size and data sparseness. approaches for proper noun translation can be divided roughly into two approaches: the use of hand-crafted bilingual lexicon as the external knowledge and the use of subwords. the former approach uses a bilingual proper noun dictionary to translate proper nouns. @xcite proposed replacement-based proper noun translation. their method uses a bilingual dictionary whose entries are associated with proper noun classes to replace a proper noun with another surrogate proper noun that frequently appears in the training corpus. another method called lexically constrained decoder (lcd) @xcite guarantees that proper nouns are translated into the target language sentence constrained by a bilingual dictionary @xcite . it extends the beam search algorithm to find the hypothesis that contains all of the proper nouns @xcite . the dictionary-based approach works well only if the proper nouns to be translated are included in the bilingual dictionary and requires efforts for developing the dictionary. in nmt, the subword-based approach is widely used. @xcite proposed the use of subwords to decompose a word into shorter units. the method decreases the number of oov words and keeps the translation quality if input sentences include oov words. however, the subword-based nmt does not always work on a proper noun translation due to wrong compositions of subword translations. in this paper, we propose a method for nmt focusing on the proper noun translation using factored transformer with named entity (ne) features. the proposed method only uses a parallel corpus and an ne recognition (ner) model as external knowledge. ## proposed method we propose the use of ne features as linguistic factors of factored transformer for accurate proper noun translation. ## experimental settings we conducted japanese-to-english and english-to-japanese mt experiments to compare the performance of the proposed method with a standard nmt method. ## results with respect to proper noun translation, the lack of a specific treatment of proper noun translation in the baseline resulted in worse performance than the proposed method. translation examples are shown in as shown in ## conclusions we proposed a method to enhance accurate proper noun translation using subword-based nmt by factored-transformer and ne features. the ne feature vectors are injected into factored transformer model as factors together with subwords. in the japanese-to-english experiments using a small bilingual training corpus, the proposed method using the best ne feature vector outperformed the baseline sub-word-based transformer model by more than 9.6 points in proper noun accuracy and 2.5 points in the bleu score. it also showed some improvements in the english-to-japanese experiments using a largescale bilingual corpus. in future work, we will work on automatic clustering of proper nouns instead of given ne categories. in the mountains, gifu castle is located above the top. 2-encoders there is gifu castle on the top of the mountain.