# Named Entity Correction in Neural Machine Translation Using the Attention Alignment Map
###### Abstract  
Neural machine translation (NMT) methods based on various artificial neural network models have shown remarkable performance in diverse tasks and have become mainstream for machine translation currently. Despite the recent successes of NMT applications, a predefined vocabulary is still required, meaning that it cannot cope with out-of-vocabulary (OOV) or rarely occurring words. In this paper, we propose a postprocessing method for correcting machine translation outputs using a named entity recognition (NER) model to overcome the problem of OOV words in NMT tasks. We use attention alignment mapping (AAM) between the named entities of input and output sentences, and mistranslated named entities are corrected using word look-up tables. The proposed method corrects named entities only, so it does not require retraining of existing NMT models. We carried out translation experiments on a Chinese-to-Korean translation task for Korean historical documents, and the evaluation results demonstrated that the proposed method improved the bilingual evaluation understudy (BLEU) score by 3.70 from the baseline. neural networks; recurrent neural networks; natural language processing; neural machine translation; named entity recognition +
Footnote †: journal: Medical  
of the translation results were skewed to higher scores. Especially for old literature translation where the machine translation is of great importance, obtaining reliable training data is much more difficult. The types of errors in the machine translation were extensively analyzed by [PERSON] [20]. They were wrong word mapping, omitted or surplus words, morphological and lexical errors, and syntactic errors such as word order and punctuation errors. There have been several methods to successfully solve these problems using transfer learning [21], contrastive learning [22], and open vocabularies [23]. Another major problem in NMT are out-of-vocabulary (OOV) words [24; 25]. This is often called the rare word problem as well [26; 27]. The words in the training dataset are converted into indices to the word dictionary or a predefined set of vectors, and a sequence of the converted numbers or vectors is used as an input to NMT systems. When a new word that is not in the dictionary is observed, the behavior of the trained network is unpredictable because there are no training sentences with the OOV words. It is almost impossible to include all of them in the dictionary because of the complexity limit for efficient translation. One of the solutions to this problem is subword tokenization using byte pair encoding (BPE) [27]. In this work, the unknown words are broken into reasonable subunits. Another solution is the unsupervised learning of the OOVs [28]. However, most of the OOV words are for named entities: human names, city names, and newly coined academic terms, and subword tokenization [27] and unsupervised learning [28] are not able to handle the named entities because they do not contain any meaningful information in them. As a solution, conventional systems use special labels for such OOV words (often as \"UNK\") and include them in the training data [24; 25; 26], so that the NMT model would distinguish them from ordinary words. Table 1 shows examples of translation outputs with an \"UNK\" symbol. The first named entity in the first example, \"\"\" \" is mistranslated into \"\(\circ\)\(|\)\(\exists\),\" although the expected output is \"\(\circ\)\(|\)\(\exists\),\".\" The second named entity, \"\(\exists\)\(\exists\),\", is not translated, but replaced with an \"UNK\" symbol because the true translation \"\(\exists\)\(\exists\)\(\exists\)\" is an OOV or rarely occurring word for the trained NMT model. Moreover, there are many similar cases in the subsequent named entities. There have been several attempts to build open-vocabulary NMT models to deal with OOV words. [PERSON] et al. [29] used a sub-LSTM layer that takes a sequence of characters to produce a word embedding vector.