Zero-shot N orth K orean to E nglish Neural Machine Translation by Character Tokenization and Phoneme Decomposition (2020)

Abstract: The primary limitation of North Korean to English translation is the lack of a parallel corpus; therefore, high translation accuracy cannot be achieved. To address this problem, we propose a zero-shot approach using South Korean data, which are remarkably similar to North Korean data. We train a neural machine translation model after tokenizing a South Korean text at the character level and decomposing characters into phonemes. We demonstrate that our method can effectively learn North Korean to English translation and improve the BLEU scores by +1.01 points in comparison with the baseline.

## introduction neural machine translation (nmt) has been adapted to many languages; however, machine translation of the north korean language 1 has seldom been performed. one of the reasons is the lack of large-scale bilingual data for training north korean neural models. it is known that large-scale bilingual data are required to improve the translation accuracy of an nmt model. for example, one of the previous works suggests that an nmt system is less accurate than a phrase-based statistical machine translation system if there are no more than 100 million words in the bilingual training data @xcite . there are three approaches to solve low language resource bottleneck. first, @xcite proposed a method to train a translation model using a pivot language as an intermediate language. this approach translates from the source language to 1 korean is a language mainly used in the korean peninsula; however, there are some grammatical differences between the republic of korea and the democratic people's republic of korea. in this study, we refer to the korean language used in the republic of korea as "south korean," and the korean language used in the democratic people's republic of korea as "north korean." the pivot language and from the pivot language to the target language. however, there is no good pivot language between north korean and english. second, @xcite proposed a many-tomany translation model, where multiple languages are translated into other languages using a single shared encoder and decoder. they demonstrated that this model can translate a language pair that is unseen in training data. however, north korean does not have any bilingual data between any languages. third, @xcite proposed a rulebased method to convert similar languages into a target language, such as brazilian portuguese to european portuguese, and extended the target language resources. north korean is a language remarkably similar to south korean, but conversion from south korean to north korean needs to be determined considering the context, which makes rule-based conversion difficult. therefore, in this study, we propose a method to tokenize south korean input sentences at the character level and decompose them into phonemes to mitigate the grammatical differences between south korean and north korean, and demonstrate that the translation model from north korean to english can be effectively learned using bilingual south korean-english data. the main contributions of this study are as follows. ## related work the pivot language approach increases the translation error between the source language and the target language, because the translation model of each language is independently trained. @xcite addressed this problem by allowing interaction during the translation model training. moreover, @xcite proposed a method to train a source-to-target model using a pretrained teacher model as its guide. @xcite proposed a rule-based method to convert similar languages into a target language to extend the language resources of the target side. @xcite presented a method to extract the conversion rules between similar languages. @xcite proposed a many-to-many translation model with several encoders and decoders. however, the accuracy of a many-to-many translation model with a single shared encoder and decoder was found to be higher @xcite . finally, the translation accuracy was improved by preprocessing of the bilingual data. @xcite demonstrated that higher translation accuracy can be obtained by decomposing kanji into ideographic characters and strokes in japanese-chinese nmt. @xcite proposed a speech-parsing model for south korean with character-level tokenization and decomposition into phonemes, demonstrating an improvement in the speech-parsing accuracy. ## korean neural machine translation using character tokenization and phoneme decomposition we propose a method to tokenize input sentences into characters or decompose them into phonemes. using this method, it is possible to reduce the influence of grammatical differences between south korean and north korean to train a machine translation model in north korean using bilingual south korean data. in the following south korean or north korean sentences, we indicate the word boundary as ⬚ for better understanding. ## discussion we extract two subsets that have differences in the ws or isr in the test data to test the hypothesis that each preprocessing step can absorb the grammatical differences. ## conclusions and future work in this study, to solve the language resource bottleneck in north korean translation, we proposed a method to tokenize input sentences in south korean and north korean at the character level and decompose them into phonemes. this method is simple and mitigates the grammatical differences between south korean and north korean; moreover, the method demonstrates improvement in translation accuracy for north korean to english translation. however, the differences that exist between south korean and north korean are not only grammatical ones. there are some words that have the same pronunciation and notation but different meanings. for example, the meaning of "낙지" is "squid" in south korean, but "octopus" in north korean. therefore, the differences in word meanings are a major challenge. in the future, we intend to use the english translation data of north korean news articles to create an evaluation dataset that considers differences in words, and attempt to develop a translation method using a language model with context, such as bert @xcite .