IOL Research Machine Translation Systems for WMT 23 General Machine Translation Shared Task (2023)

Abstract: This paper describes the IOL Research team’s submission systems for the WMT23 general machine translation shared task. We participated in two language translation directions, including English-to-Chinese and Chinese-to-English. Our final primary submissions belong to constrained systems, which means for both translation directions we only use officially provided monolingual and bilingual data to train the translation systems. Our systems are based on Transformer architecture with pre-norm or deep-norm, which has been proven to be helpful for training deeper models. We employ methods such as back-translation, data diversification, domain fine-tuning and model ensemble to build our translation systems. An important aspect worth mentioning is our careful data cleaning process and the utilization of a substantial amount of monolingual data for data augmentation. Compared with the baseline system, our submissions have a large improvement in BLEU score.

## introduction this paper describes our submissions to the wmt23 general machine translation shared task. we participated in two language translations: english-to-chinese and chinese-to-english. for both tasks, we built our system in a constrained scenario, using only official training data. our systems are based on transformer @xcite architecture with pre-norm or deep-norm @xcite , which has been proven to be helpful for training deeper models. we used rule-based methods, language models, and alignment models to clean bilingual and monolingual data, and then used back-translation @xcite , data diversification @xcite , and model ensemble @xcite to leverage large-scale monolingual data to construct our translation systems. we also tried domain fine-tuning and found that this approach still helped in improv-ing the bleu @xcite scores on the wmt23 test set. the design of the subsequent paper is as follows. we introduce the data source and processing strategy in section2; section 3 describes the details of our training procedure; section 4 presents the experimental settings and results. ## system overview we chose transformer @xcite as our base translation model and used both pre-norm and deep-norm @xcite variants to help us train deeper models. to improve the quality of translation models, we first pre-trained the translation models from scratch on the synthesized datasets generated by back-translation, then continue training on the datasets generated by data diversification, and finally used domain data for finetuning. we also iteratively performed two rounds of data augmentation to improve the quality of the synthetic data. the final synthetic data is generated by the model after training on data diversification data of the first round. we only used domain finetuning in the final submission. this method we adopt is a commonly used method in the field of machine translation and has been proven to be effective. in the following sections, we show the specifics of how we use these methods. ## conclusion in this paper, we described iol research's submissions to the wmt2023 general translation shared task. we participated in the english from and to chinese translation. our system aims to leverage as much monolingual data as possible to improve the quality of machine translation