We present a systematic study of the effect of crowdsourced translations on Machine Translation performance. We compare Machine Translation systems trained on the same data but with translations obtained using Amazon's Mechanical Turk vs. professional translations, and show that the same performance is obtained from Mechanical Turk translations at 1/5th the cost. We also show that adding a Mechanical Turk reference translation of the development set improves parameter tuning and output evaluation.
REF conducted a systematic study that showed that training an MT system on crowdsourced translations resulted in the same performance as training on professional translations, at 1 5 the cost.
Systematic Comparison of Professional and Crowdsourced Reference Translations for Machine Translation
NAACL
Computer Science