# Improved Arabic-Chinese Machine Translation with Linguistic Input Features
## 6 Results and Analysis  
Table 3 shows the results for Arabic-to-Chinese translation on the extracted Multi-UN corpus. In PBSMT, we observe that tokenization strategies (ATB and D3*) have a major effect in alleviating the data sparseness problem, improve the translation quality and achieve higher BLEU scores than the baseline with an advantage to the D3* scheme. The differences in terms of BLEU scores between baseline and both tokenization schemes are statistically significant (\(p\)-value < 0.05). This result confirms that Arabic preprocessing helps address data sparseness. All the factored systems considerably outperform the baseline and achieve better or comparable performance with tokenized PBSMT. The differences in BLEU scores between tokenized phrase-based MT and the factored systems are not statistically significant, whereas the differences between the baseline and all the factored systems are statistically significant (\(p\)-value < 0.05). Moreover, using a single feature provides a better result than a combination of models with different features, as we notice that the POS features outperform all the systems. The results confirm that the translation model is benefited from the existence of additional linguistic information. MDP Lemma/morph as well as lemma Backoff models exhibit the best results in the reduction of OOV rates compared to all models. Although the tokenized models show better results in terms of OOV rates compared to some of the factored models, the effects of OOV reduction are not always well reflected by BLEU score [32]. Another important observation is that most OOV words are related to proper nouns that were not found in the training corpus. In this case, transliteration of named entities would be helpful to improve the translation quality. According to these results, factored models are the clear winners in all the scenarios we have presented. To perform a manual analysis, we randomly selected sentences from the system output over the baseline, tokenized models and best-factored models as shown in Figure 5, and observed the following: \begin{table}
\begin{tabular}{c c c c c c c c} \hline \multirow{2}{*}{**System**} & \multirow{2}{*}{**Model**} & \multirow{2}{*}{**BLEU**} & \multicolumn{4}{c}{**OOV**} & \multicolumn{2}{c}{**Sign.**} \\ \cline{3-8}  & & & & **Tokens (\%)** & **Types (\%)** & **\(p\) \textless{}0.05** & **\(p\) \textless{}0.01** \\ \hline \multirow{4}{*}{**Synthetic**} & Baseline & 16.13 & – & 5.74 & 22.80 & – & – \\  & Tok-ATB & 17.32 & (1.19) & 1.52 & 12.53 & ✓ & \\  & Tok-D3* & 17.67 & (1.54) & 1.40 & 11.20 & ✓ & \\ \hline \multirow{4}{*}{**Synthetic**} & Surface+Lemma & 17.96 & (1.83) & 1.72 & 14.51 & ✓ & ✓ \\  & Surface+POS & **18.30** & **(2.17)** & 1.70 & 14.33 & ✓ & ✓ \\  & Surface+Morph & 17.90 & (1.77) & 1.71 & 14.42 & ✓ & ✓ \\  & Surface+Lemma+POS & 17.48 & (1.35) & 1.80 & 13.92 & ✓ & \\  & Surface+Lemma+Morph & 17.79 & (1.66) & 1.72 & 14.16 & ✓ & \\  & Surface+POS+Morph & 17.43 & (1.30) & 1.75 & 14.20 & ✓ & \\  & Surface+All Features & 17.82 & (1.69) & 1.69 & 13.19 & ✓ & ✓ \\  & Multiple Decoding & 17.66 & (1.53) & 0.81 & 6.72 & ✓ & \\  & Paths & & & & & \\  & Lemma Backoff & 17.59 & (1.46) & **0.70** & **6.71** & ✓ & \\ \hline \end{tabular}
\end{table} 
Table 3: BLEU scores with the improvement over the baseline model, along with the effects of pre-processing schemes and linguistic factors on the test set according to OOV rates. Besides, results of statistical significance test where ✓indicates a significant improvement over the baseline with the specified \(p\)-value. The best result in terms of each metric (the highest BLEU and lowest OOVs) is highlighted in bold. 1) Alleviate issues of dropping translations  
The baseline and both tokenized models ignore or drop the translation of some words in the dataset, whereas this condition occurs at a lower rate in the factored models. This inclusion is due to the poor performance of SMT system for unobserved words in the training corpus, and worse performance (dropping to 48.6%) on words that were seen once [15].