The V olc T rans System for WMT 22 Multilingual Machine Translation Task (2022)

Abstract: This report describes our VolcTrans system for the WMT22 shared task on large-scale multilingual machine translation. We participated in the unconstrained track which allows the use of external resources. Our system is a transformer-based multilingual model trained on data from multiple sources including the public training set from the data track, NLLB data provided by Meta AI, self-collected parallel corpora, and pseudo bitext from back-translation. Both bilingual and monolingual texts are cleaned by a series of heuristic rules. On the official test set, our system achieves 17.3 BLEU, 21.9 spBLEU, and 41.9 chrF2++ on average over all language pairs. Averaged inference speed is 11.5 sentences per second using a single Nvidia Tesla V100 GPU.

## introduction multilingual machine translation attracts much attention in recent years due to its advantages in sharing cross-lingual knowledge for low-resource languages. it also dramatically reduces training and serving costs. training a multilingual model is much faster and simpler than training many bilingual ones. serving multiple low-traffic languages using one model could drastically improve gpu utilization. the wmt22 shared task on large-scale multilingual machine translation includes 24 african languages @xcite . inspired by previous research works, we train a deep transformer model to translate all languages since large models have been demonstrated effective for multilingual translation @xcite @xcite . we participated in the unconstrained track that allows the use of external data. besides the official dataset for the constrained track, and the nllb corpus provided by metaai @xcite , we also collect parallel and monolingual texts from public websites and sources. these raw data are cleaned by a series of commonly used heuristic rules, and a minimum description length (mdl) based approach to remove samples with repeat patterns. monolingual texts are used for back translation. for some very low-resource languages such as wolof, iterative back-translation is adopted for higher accuracy. we compare different training strategies to balance efficiency and quality, such as streaming data shuffling, and dynamic vocabulary for new languages. furthermore, we used the open-sourced lightseq toolkit foot_0 to accelerate training and inference. on the official test set, our system achieves 17.3 bleu, 21.9 spbleu, and 41.9 chrf2++ on average over all language pairs. averaged inference speed is 11.5 sentences per second using a single nvidia tesla v100 gpu. ## conclusion this paper presents our system for the wmt22 shared task on multilingual machine translation for african languages. we focus on data collection, augmentation, and cleaning. due to the limited time, we did not try modeling tricks such as reranking and ensemble. our finding is that the amount of data is crucial for translation quality, especially monolingual data in low-resource languages.