Statistical machine translation is quite robust when it comes to the choice of input representation. It only requires consistency between training and testing. As a result, there is a wide range of possible preprocessing choices for data used in statistical machine translation. This is even more so for morphologically rich languages such as Arabic. In this paper, we study the effect of different word-level preprocessing schemes for Arabic on the quality of phrase-based statistical machine translation. We also present and evaluate different methods for combining preprocessing schemes resulting in improved translation quality.
REF experimented with a wide range of tokenization schemes for Arabic-English translation.
Combination Of Arabic Preprocessing Schemes For Statistical Machine Translation
International Conference On Computational Linguistics And Annual Meeting Of The Association For Computational Linguistics
Computer Science