GTCOM Neural Machine Translation Systems for WMT 21 (2021)

Abstract: This paper describes the Global Tone Communication Co., Ltd.’s submission of the WMT21 shared news translation task. We participate in six directions: English to/from Hausa, Hindi to/from Bengali and Zulu to/from Xhosa. Our submitted systems are unconstrained and focus on multilingual translation odel, backtranslation and forward-translation. We also apply rules and language model to filter monolingual, parallel sentences and synthetic sentences.

## introduction we applied fairseq @xcite as our develop tool and use transformer @xcite as the main architecture. the primary ranking index for submitted systems is bleu @xcite , therefore we apply bleu as the evaluation matrix for our translation system. for data preprocessing, punctuation normalization, tokenization and bpe(byte pair encoding) @xcite are applied for all language. further, we apply truecase model for english, hausa, zulu and xhosa according to the character of each language. regarding to the tokenization, we use polyglot foot_0 as the tokenizer for hausa, hindi, bengali, zulu and xhosa. besides, knowledge based rules and language model are also involved to clean parallel data, monolingual data and synthetic data. due to the quantity limitation of parallel corpus in low-resource language pair, we use forwardtranslation with monolingual data to generate more synthetic data instead of knowledge distillation @xcite . here forward-translation refers to translate the source language sentences to the target language, and then clean this synthetic data with the above described method. in order to enrich the low-resource language corpus, we add english to x corpus to construct a multilingual translation model. this multilingual model is expected to obtain the inner deep information among all languages and give us a better translation. this paper is arranged as follows. we firstly describe the task and show the data information, then introduce our multilingual translation model. after that, we describe the techniques on low-resource condition and show the conducted experiments in detail of all directions, including data preprocessing, model architecture, back-translation, forwardtranslation and multilingual translation model. at last, we analyze the results of experiments and draw the conclusion. ## task description the task focuses on bilingual text translation in news domain and the provided data is show in ## multilingual translation model in low-resource condition, data augmentation and pretrained model are the most effective approaches to improve translation quality. according google's multilingual neural machine translation system @xcite pairs parallel data along with the provided bilingual data to training a multilingual translation model, the low-resource language pair is expected to get the benefits from other language pair's parallel data, especially in similar language. for the multilingual model preprocessing, we add a language tag at beginning of each source sentence, and use joint bpe for all languages in one multilingual translation model. ## summary this work mainly focus data augmentation and pay less attention on modeling. because optimizing translation by data augmentation is the most elegant way for a commercial system. it can avoid many unexpected translation result generated by a newly proposed model which may give our customers worse translating experience. this paper describes gtcom's neural machine translation systems for the wmt21 shared news translation task. for all translation directions, we build systems mainly base on multilingual translation model and enrich information by back-translation and forward-translation. the effect of increasing information is also dependent on data filtering.