# Phrase Table Combination Based on Symmetrization of Word Alignment for Low-Resource Languages
## 3 An Interpolation System Approach
### Datasets and Pre-Processing  
In this study, we used the news-commentary [20] and ALT datasets [16] for Kazakh-English (Kk-En) and Japanese-Indonesian (Ja-Id), respectively. We considered the two news domain datasets because of the use of formal language. Some languages have flexible word order, such as Russian [31]. Flexible word order means sentences can use free word order of subject, verb, and object. Since a news domain uses formal language, we expected that the sentences would use one dominant word order. Thus, we could analyze the effects of different word orders in the pivot approaches. A news domain uses relatively long sentences, i.e., about more than 22 words on average, as shown in Tables 1 and 2. We expected that longer sentences would produce various phrase pairs from the symmetrization of word alignment. Additionally, news-commentary and ALT were used as training datasets for low-resource translation tasks in the Workshop on Machine Translation (WMT) 2019 and Workshop on Asian Translation (WAT) 2019.  
We performed several pre-processing steps, i.e., tokenizing, normalizing punctuation, re-casing, and filtering sentences for both datasets. The tokenization step separates words and punctuation. We employed [PERSON] [32] to tokenize the Kk, En, and Id, and MeCab [16] was used to tokenize Ja. We then normalized the punctuation so that the decoder system could recognize it. Next, the re-casing step reduced the data sparsity by converting the initial word in each sentence to its most probable casing. Lastly, we removed sentences that had a length of more than 80 words in the filtering step. We show the dataset statistics for Kk-En and Ja-Id in Tables 1 and 2, respectively.  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Dataset** & **\#Sentences** & **Average Sentence Length** & **Vocabulary** \\ \hline \multicolumn{4}{c}{Baseline system} \\ \hline Train & & & \\ news-commentary-v14.en-kk.kk & 9619 & 18.09 & 29,142 \\ news-commentary-v14.en-kk.en & 9619 & 22.15 & 16,742 \\ Dev & & & \\ newsdev2019-enkk.kk & 2068 & 18.02 & 11,389 \\ newsdev2019-enkk.en & 2068 & 22.23 & 7726 \\ Language Model & & & \\ news-commentary-v14.en & 532,560 & 21.58 & - \\ \hline \multicolumn{4}{c}{Interpolation system} \\ \hline Train & & & \\ news-commentary-v14.kk-ru.ru & 7230 & 23.68 & 27,819 \\ news-commentary-v14.kk-ru.kk & 7230 & 20.12 & 24,627 \\ news-commentary-v14.en-ru.en & 97,652 & 23.04 & 51,566 \\ news-commentary-v14.en-ru.ru & 97,652 & 21.35 & 126,476 \\ Dev & & & \\ news-commentary-v14.kk-ru.ru & 2000 & 20.88 & 11,841 \\ news-commentary-v14.kk-ru.kk & 2000 & 18.05 & 10,561 \\ newstest2018-ruen.dev.en & 3000 & 20.98 & 10,108 \\ newstest2018-ruen.dev.ru & 3000 & 17.33 & 17,091 \\ Language Model & & & \\ news-commentary-v14.en-ru.ru & 114,375 & 21.27 & - \\ news-commentary-v14.en-ru.en & 114,375 & 22.98 & - \\ \hline \hline \end{tabular}
\end{table}
Table 1: Dataset statistics of Kazakh–English (Kk–En).  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Dataset** & **\#Sentences** & **Average Sentence Length** & **Vocabulary** \\ \hline \multicolumn{4}{c}{Baseline system} \\ \hline Train & & \\ DataALT.01.jp-id.SP.true.jp & 8500 & 34.90 & 19,086 \\ DataALT.01.jp-id.SP.true.id & 8500 & 24.75 & 28,014 \\ Dev & & & \\ DataALT.02.jp-id.true.jp & 2000 & 33.88 & 6680 \\ DataALT.02.jp-id.true.id & 2000 & 24.26 & 10,201 \\ Language Model & & & \\ DataALT.01.jp-id.SP.true.id & 8500 & 24.75 & - \\ \hline \multicolumn{4}{c}{Interpolation system} \\ \hline Train & & \\ DataALT.01.jp-id.SP.true.jp & 8500 & 34.90 & 19,086 \\ DataALT.01.jp-id.SP.true.ms & 8500 & 25.11 & 26,835 \\ DataALT.01.jp-id.PT.true.ms & 8500 & 25.04 & 26,922 \\ Dev & & & \\ DataALT.02.jp-id-true.jp & 2000 & 33.88 & 6680 \\ DataALT.02.jp-id-true.ms & 2000 & 24.51 & 9888 \\ DataALT.02.jp-id-true.id & 2000 & 24.26 & 10,201 \\ Language Model & & & \\ DataALT.01.jp-id.SP.true.ms & 8500 & 25.11 & - \\ DataALT.01.jp-id.SP.true.id & 8500 & 24.75 & - \\ \hline \hline \end{tabular}
\end{table}
Table 2: Dataset statistics of Japanese–Indonesian (Ja–Id).