Inference-only sub-character decomposition improves translation of unseen logographic characters (2020)

Abstract: Neural Machine Translation (NMT) on logographic source languages struggles when translating ‘unseen’ characters, which never appear in the training data. One possible approach to this problem uses sub-character decomposition for training and test sentences. However, this approach involves complete retraining, and its effectiveness for unseen character translation to non-logographic languages has not been fully explored. We investigate existing ideograph-based sub-character decomposition approaches for Chinese-to-English and Japanese-to-English NMT, for both high-resource and low-resource domains. For each language pair and domain we construct a test set where all source sentences contain at least one unseen logographic character. We find that complete sub-character decomposition often harms unseen character translation, and gives inconsistent results generally. We offer a simple alternative based on decomposition before inference for unseen characters only. Our approach allows flexible application, achieving translation adequacy improvements and requiring no additional models or training.

## introduction while neural machine translation (nmt) has evolved rapidly in recent years, not all of its successful techniques are equally applicable to all language pairs. a particular example is the representation and translation of unseen tokens, which do not appear in the training data. with techniques like subword decomposition @xcite , an unseen word in an alphabetic language can in the worst case be represented as a sequence of characters. since alphabetic languages usually have few unique characters, it is reasonable to assume that * now at amazon, work performed while at sdl all of these 'backoff' characters will be present in the limited model vocabulary. we focus instead on the translation of unseen chinese and japanese logographic characters into alphabetic languages, a task that remains a challenge for nmt. logographic writing systems may have many thousands of logograms, each representing at least one word, morpheme or concept as well as conveying phonetic and prosodic information. inevitably some characters will either not be present in the training data, or will be present but too rare to be included in the vocabulary. if the model is required to translate a previously unseen character, it will usually be replaced with an unk (unknown word) token. the most likely outcome is that it will be ignored by the translation model, which will instead rely on the context of the unseen character to produce the translation. in the worst case, the presence of a previously-unseen character at inference time may harm the translation quality. this is a particular concern for nmt in low-resource domains, when a model is less able to rely on lexical context. many logographic characters share subcharacter components foot_0 , which can carry semantic or phonetic meaning ( we identify two motivations for using subcharacters in logographic nmt: our hypothesis is that, while complete subcharacter decomposition for all characters might be useful in case 1, only some characters benefit from only semantic elements of the decomposition in case 2. the focus of this work is case 2. our contributions are as follows: ## sub-character decomposition over 80% of chinese characters can be broken down into both a semantic and a phonetic component @xcite . the semantic meaning of a chinese character often corresponds to the subcharacter occupying its top or left position @xcite . these may be -but are not always -radicals: sub-characters that cannot be broken down any further. however, radicals in these positions are not necessarily directly meaningful. for example, radical 魚 ('fish') has a clear semantic relationship with the character 鰯 ('sardine'), but the semantic connection of radical 木 ('tree') to character 校 ('school') is more abstract. example decompositions are given in ## conclusions we explore the effect of sub-character decomposition on nmt from logographic languages into english. during training decomposition may hurt general translation performance without necessarily helping unseen character translation. we propose a flexible inference-time sub-character decomposition procedure which targets unseen characters, and show that it aids adequacy and reduces misleading overtranslation in unseen character translation. the scheme is straightforward, requires no additional models or training, and has no negative impact on sentences without unseen characters.