DUTNLP Machine Translation System for WMT 22 General MT Task (2022)

Abstract: This paper describes DUTNLP Lab’s submission to the WMT22 General MT Task on four translation directions: English to/from Chinese and English to/from Japanese under the constrained condition. Our primary system are built on several Transformer variants which employ wider FFN layer or deeper encoder layer. The bilingual data are filtered by detailed data pre-processing strategies and four data augmentation methods are combined to enlarge the training data with the provided monolingual data. Several common methods are also employed to further improve the model performance, such as fine-tuning, model ensemble and post-editing. As a result, our constrained systems achieve 29.01, 63.87, 41.84, and 24.82 BLEU scores on Chinese-to-English, English-to-Chinese, English-to-Japanese, and Japanese-to-English, respectively.

## introduction dutnlp lab participates in the wmt22 general mt task on four translation directions: english ↔ chinese and english ↔ japanese. our translation system is trained on the officially provided bilingual and monolingual data under the constrained condition. several strategies such as fine-grained data pre-processing, large-scale synthetic data augmentation, diverse model architectures and domain fine-tuning are utilized to enhance the performance of the final ensemble model. since the quality of the training data is crucial to the translation performance, all the training sets are filtered by the off-the-shelf toolkits and some manual rules. details will be discussed in section 2. those data pre-processing strategies are also employed to filter out the synthetic data generated by different data augmentation methods. to generate synthetic parallel data, four data augmentation methods including back-translation * corresponding author @xcite , forward-translation @xcite , knowledge distillation @xcite and r2l training @xcite are employed in our experiments. specifically, we leverage source-side monolingual data by exploring forward-translation, knowledge distillation and r2l training, while target-side monolingual data by back-translation. these strategies increase the data size to a large extent. the generated data and the original parallel data are combined to train nmt models. for model architectures, starting from transformer-big @xcite settings, several transformer variants are used to improve the model capacity and diversity. previous studies @xcite have shown that the translation performance can be significantly improved by increasing the model capacity. therefore, we build different model architectures with either wider ffn layers @xcite or deeper transformer encoder @xcite . moreover, the pre-norm @xcite is also adopted in all our experiments as its performance and training stability are better than the post-norm counterpart. domain fine-tuning is the most effective method in our experiments, which greatly improves the translation performance. we first employ previous wmt test sets as the domain data to fine-tune several models with different architectures. then we ensemble those fine-tuned models and translate the test sets to construct pseudo parallel data. finally, the original and the pseudo test sets are merged for further domain fine-tuning. this paper is structured as follows: section 2 describes the data pre-processing strategies. we present the details of our systems in section 3 and show the experiment settings and results in section 4. we draw the conclusion in section 5. ## data pre-processing for each language pair, we follow the constrained data requirements and make full use of the provided bilingual and monolingual data. as the quality of the parallel training data is crucial to the final translation performance, we perform fine-grained data filtering with the off-theshelf toolkits and some manual rules. for both language pairs, the pre-processing strategies are as follows: ## conclusion this paper presents the dutnlp translation systems for wmt22 general mt task. our main exploration is to improve the translation performance with the fine-grained data filtering, diverse model architectures, large-scale data augmentation and domain fine-tuning. the effectiveness of each method is demonstrated in our experiments. model