OPPO ’s Machine Translation Systems for WMT 20 (2020)

Abstract: In this paper we demonstrate our (OPPO’s) machine translation systems for the WMT20 Shared Task on News Translation for all the 22 language pairs. We will give an overview of the common aspects across all the systems firstly, including two parts: the data preprocessing part will show how the data are preprocessed and filtered, and the system part will show our models architecture and the techniques we followed. Detailed information, such as training hyperparameters and the results generated by each technique will be depicted in the corresponding subsections. Our final submissions ranked top in 6 directions (English \leftrightarrow Czech, English \leftrightarrow Russian, French \rightarrow German and Tamil \rightarrow English), third in 2 directions (English \rightarrow German, English \rightarrow Japanese), and fourth in 2 directions (English \rightarrow Pashto and and English \rightarrow Tamil).

## introduction this paper describes the oppo's submission to the fifth conference on machine translation (wmt20) news translation shared task. we built transformer @xcite based systems for all the directions, and applied several well known, widelyused techniques, such as large scale backtranslation @xcite ) and forwardtranslation, model ensemble and rerank ing. since all the systems share a roughly similar data preprocessing and training methods, to avoid duplication words, we will demonstrate the com mon knowledge in section 2 firstly, which will be divided into two parts: the preprocessing part shows the data preprocessing pipeline and data fil tering pipeline, the latter is generally composed by rulebased filtering and alignmentbased filtering; the training part depicts the techniques we applied. detailed information, including training hyperpa rameters, the results generated by each technique, and some other explorations will be listed in each corresponding direction in section 3. finally, we will summarize the report and indicate our final works. we used marian @xcite to implement our systems for english ↔ {khmer, russian, tamil} and french ↔ german task pairs, and fairseq @xcite for the rest foot_0 . ## system overview we preprocess corpora in two stages. in the pre processing stage, data is converted but not filtered. the common pipeline of preprocessing including the following steps: the last three steps are all processed by moses scripts. this pipeline is both applied for the paral lel corpora and monolingual corpora, and true case models are generally trained on the mixture of par allel and monolingual datasets. after preprocessing we filter the parallel cor pora according to statistical information and align ment information, set the thresholds according to our previous experiences. for the statistic perspec tive, we mainly focus on some heuristic rules, con tain but not limited in • pairs of which the source side and the target side are the same. for the alignment perspective, we use fast_align @xcite to acquire the alignment scores from source to target and vice versa, then we aver age the scores for each pair to calculate a data pair's sentencelevel alignment score. if a sentence pair's sentencelevel alignment score is lower than 15, it will be expelled from the final dataset. having purified the corpus, we generally try to boost our systems using the following techniques, step by step: we also followed @xcite to it eratively backtranslate and forwardtranslate the corpus for several times. ## experiments details in this section we demonstrate our experiments de tails for each direction. for brevity we will ig nore the same preprocessing and techniques we in troduced in the previous section, mainly focus on how the techniques boosted the systems, and some other unique observations we found during the ex periments. in the text we will sometimes use iso6391 twoletter codes for each language for short. map ping between the abbreviations and full names can be found in ## conclusion this report described oppo's submissions to the wmt20 news translation task. we use the sim ilar data preprocess and filtering strategy for all the tasks, contains statistical information based rules and alignment information based rules. we trained transformerbig models for all the direc tions and applied some mature techniques, like backtranslation, ensemble model, finetune and reranking, they generally all brought gains for the final results. our final submissions ranked top in 6 directions (english ↔ czech, english ↔ rus sian, @xmath1, third in 2 directions (@xmath2, @xmath3, and fourth in 2 directions (@xmath4.