# Multi-Source Neural Model for Machine Translation of Agglutinative Language
## 4 Experiment
### Model Parameter Setting  
We modified the Transformer model implemented in the OpenNMT-tf toolkit [56]. Both the encoder and decoder had \(N=6\) identical layers. The number of head was set to \(h=8\), and the number of the hidden units in the fully connected feed-forward network was set to 1024. Both the source-side  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Language** & **\# Sentences** & **\# Tokens** & **\# Word** & **\# Lemma** \\ \hline Turkish & 355,251 & 6,712,018 & 283,858 & 96,047 \\ English & 355,251 & 8,376,414 & 110,522 & - \\ Uyghur & 330,192 & 6,043,461 & 261,918 & 128,786 \\ Chinese & 330,192 & 5,947,903 & 163,265 & - \\ \hline \hline \end{tabular}
\end{table}
Table 1: The statistics of the training dataset for Turkish–English and Uyghur–Chinese translation.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Encoder** & **Linguistic Feature** & \multicolumn{6}{c}{**Training Sentence Example**} \\ \hline word-based encoder & word & \multicolumn{6}{c}{Ve bunlar sinek@@ kap@ an@ em@@ o\(\oplus\) o\(\mathrm{\mathrm{\mathrm{\mathrm{\mathrm{\mathrm{\mathrm{\mathrm{\mathrm{\mathrm{\mathrm{\mathrm{\mathrm{ \mathrm{ \mathrm{ }}}}}}}}}}}}}\) \\  & lemma & ve & bu & sinek@@ kapan & ane@@ & mon \\ \cline{2-10}  & (Part-Of-Speech) & \multirow{2}{*}{Conj} & \multirow{2}{*}{Pron} & \multirow{2}{*}{Noun} & \multirow{2}{*}{Noun} & \multirow{2}{*}{Noun} & \multirow{2}{*}{Noun} \\  & POS tag & & & & & & \\  & morph tag & \textless{}null\textgreater{} & \textless{}null\textgreater{} & \textless{}null\textgreater{} & \textless{}null\textgreater{} & A3 pl & A3 pl \\ \hline \hline \end{tabular}
\end{table}
Table 2: The training sentence examples for Turkish–English machine translation task.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Encoder** & **Linguistic Feature** & \multicolumn{6}{c}{**Training Sentence Example**} \\ \hline word-based encoder & word & \multicolumn{6}{c}{**[PERSON] sentence Example**} \\ \hline  & lemma & \multicolumn{6}{c}{**[PERSON] sentence Example**} \\ \cline{2-10} knowledge-based encoder & POS tag & \textless{}null\textgreater{} & \textless{}null\textgreater{} & \textless{}null\textgreater{} & \textless{}null\textgreater{} & \textless{}null\textgreater{} & A3 pl & A3 pl \\ \hline \hline \end{tabular}
\end{table}
Table 3: The training sentence examples for Uyghur–Chinese machine translation task.  
word and the target-side word embedding size was set to 512. We used a mini-batch size of 48 training sentences, and a maximum sentence length of 100 tokens with 0.1 label smoothing. The dropout rate in the Transformer was set to 0.1, the length penalty was set to 0.6, and the clip gradient [57] was set to 5.0. The parameters were uniformly initialized in [\(-\)0.1, 0.1]. The model was trained for 100,000 steps by using the Adam optimizer [58] with an initial learning rate of 0.0002. We reported the result of averaging the last five saved model checkpoints (saved every 5000 steps). Decoding was performed by using the beam search with a beam size of five.  
We employed both the(Bilingual Evaluation Understudy) BLEU [59] and (Character n-gram F3-score) ChrF3 [60] scores to evaluate the translation performance. The vocabulary size and embedding size for the source-side Turkish and Uyghur translation tasks are shown in Table 4.