# Multi-Source Neural Model for Machine Translation of Agglutinative Language
## 1 Introduction  
With the rapid development of artificial intelligence and deep learning, neural networks are widely applied to various fields ranging from computer vision [1; 2], speech recognition [3; 4], and natural language processing (NLP) [5; 6; 7; 8]. The standard neural machine translation (NMT) model [9; 10; 11; 12] employs the encoder to map the source sentence into a continuous representation vector, then it feeds the resulting vector to the decoder to generate the target sentence, which directly learns the translation relationship between two distinct languages from the bilingual parallel sentence pairs. Recently, by exploiting advanced neural networks, such as long short-term memory (LSTM) [13], gate recurrent unit (GRU) [14], and attention mechanism [15], NMT has become the current dominant machine translation approach, and it achieves impressive performance on many high-resource language pairs, such as Chinese-English translation and English-German translation. However, existing NMT models still struggle in the translation task of agglutinative languages with complex morphology and limited resources, such as Turkish to English and Uyghur to Chinese. The morpheme structure of the word in agglutinative language is formed by a stem followed by a sequence of suffixes (since the words only have a few prefixes, we simply combine the prefixes with astem into the stem unit), which can be denoted as: word = stem + suffix1 + suffix2 +... + suffixN [16]. For example, in the Turkish phrase \"kucuk fagernes kasabasundaym\" (I'm in a small town of fagernes), the morpheme structure of the word \"kasabasundaym\" (I'm in town) is: kasaba + si + nda + yim. Due to the fact that the suffixes have many inflected and morphological variants depending on the case, tense, number, gender, etc., the vocabulary size of an agglutinative language is considerable even in small-scale training data. Moreover, a word can express the meaning of a phrase or sentence. Thus, there are many rare and out-of-vocabulary (OOV) words in the training process, which leads to many inaccurate translation results [17] and increases the NMT model complexity. Recently, researchers attempted to explicitly use the source-side linguistic knowledge to further improve the NMT performance. [PERSON] and [PERSON] generalized the word-embedding layer of the encoder to accommodate for additional linguistic input features including lemma, sub-word tag, part-of-speech (POS) tag, dependency label, and morphological tag for German-English translation [18]. [PERSON] et al. proposed a tree-to-sequence-based model for English-Japanese translation, which encodes each phrase in the source parse tree and used the attention mechanism to align both the input words and phrases with the output words [19]. [PERSON] et al. improved the above work by encoding each node in the source parse tree with the local and global context information, and they utilized a weighted variant of the attention mechanism to adjust the proportion of the conditional information for English-German translation [20]. [PERSON] et al. combined the source-side sentence with its linearized syntactic structure, which makes the NMT model automatically learn useful language information for Chinese-English translation [21]. [PERSON] and [PERSON] modified the multi-source technique [22] for English-German translation, and they exploited the syntax structure of the source-side sentence by employing an additional encoder to encode the linearized parse tree [23]. [PERSON] et al. presented a linguistic knowledge-aware neural model for both English-Chinese translation and English-German translation, which uses a knowledge gate and an attention gate to control the information from the source words and the linguistic features of POS tag, named entity (NE) tag, chunk tag and dependency label [24].