The U niversity of E dinburgh’s E nglish- G erman and E nglish- H ausa Submissions to the WMT 21 News Translation Task (2021)

Abstract: This paper presents the University of Edinburgh’s constrained submissions of English-German and English-Hausa systems to the WMT 2021 shared task on news translation. We build En-De systems in three stages: corpus filtering, back-translation, and fine-tuning. For En-Ha we use an iterative back-translation approach on top of pre-trained En-De models and investigate vocabulary embedding mapping.

## introduction we describe the university of edinburgh's participation in english↔german (en↔de) and english↔hausa (en↔ha) at the wmt 2021 news translation task. we apply distinct sets of techniques to the two language pairs separately, as the two pairs are very different in terms of language proximity and the availability of resources. we follow the constrained condition where we only use the provided data available to all participants. for en↔de we first employ rule-based and dual conditional cross-entropy filtering to clean the datasets. then we add to training back-translations generated in a few ways: tagged, greedy, beam search and sampling. we fine-tune our models on past years' test sets, and finally tune a few configurations: length normalization, test sentence splitting, and german post-processing. for en↔ha we adopt iterative back-translation, where at each iteration we initialize the model parameters from an en-de model in the corresponding direction (@xmath0. these en-de models are trained in the same way as those submitted to the en-de task, except that their vocabulary includes subwords from the hausa language. besides, we experiment with vocabulary mapping at the embedding level. some configurations are kept consistent across language pairs and systems. sentences are tok-enized using sentencepiece @xcite with a 32k shared vocabulary, except that we added a few extra tokens for tagged backtranslation. all models are trained following marian's transformer-big task preset @xcite unless otherwise specified: 6 encoder and decoder layers, 16 heads, 1024 hidden embedding size, tied embeddings (press and wolf, 2017), etc. 1 section 2 and section 3 describe the detailed model building process for en↔de and en↔ha respectively. while awaiting human evaluation results, we summarize our automatic metric scores on the wmt 2021 test sets computed by the task organizers in ## conclusion we describe our english-german and english-hausa submissions to the news translation task at wmt 2021. for the en↔de task, fine-tuning and splitting test instances significantly boosts bleu while back-translation alone does not help. in the en↔ha task, we experiment with interesting low resource nmt techniques, but unfortunately, our submission contains translations from overfitted models.