Netmarble AI Center’s WMT 21 Automatic Post-Editing Shared Task Submission (2021)

Abstract: This paper describes Netmarble’s submission to WMT21 Automatic Post-Editing (APE) Shared Task for the English-German language pair. First, we propose a Curriculum Training Strategy in training stages. Facebook Fair’s WMT19 news translation model was chosen to engage the large and powerful pre-trained neural networks. Then, we post-train the translation model with different levels of data at each training stages. As the training stages go on, we make the system learn to solve multiple tasks by adding extra information at different training stages gradually. We also show a way to utilize the additional data in large volume for APE tasks. For further improvement, we apply Multi-Task Learning Strategy with the Dynamic Weight Average during the fine-tuning stage. To fine-tune the APE corpus with limited data, we add some related subtasks to learn a unified representation. Finally, for better performance, we leverage external translations as augmented machine translation (MT) during the post-training and fine-tuning. As experimental results show, our APE system significantly improves the translations of provided MT results by -2.848 and +3.74 on the development dataset in terms of TER and BLEU, respectively. It also demonstrates its effectiveness on the test dataset with higher quality than the development dataset.

## introduction automatic post-editing (ape) aims to improve the quality of an existing machine translation (mt) system by learning from human-edited samples @xcite . with the continuous performance improvements of neural machine translation (nmt) systems along with deep learning advancements, developing ape systems has faced a big challenge. simple translation errors are hard to find in machine translation outputs, and * these authors equally contributed to this work. the remaining errors are still hard to solve. in recent years, transfer learning and data augmentation techniques have shown their efficiency when training models on datasets with limited size @xcite . therefore, such approaches are also adopted in ape tasks @xcite . participants in wmt21 ape shared tasks are required to develop systems to automatically postedit the translation outputs from an unknown mt system. in this year, the same data has been repost-edited to improve the quality. as a result of performing statistics on the development set, the evaluation scores are 19.057 and 68.79 in terms of ter and bleu, which are much higher than the scores of last year, 31.374 and 50.37, respectively. the central distribution of ter has shifted to the left compared to last year. we find that the section in the range of 5 to 10 has the most examples, which indicates that over-correction problems should be considered during the ape tasks. in addition, the dataset has been changed in terms of the domain (from it to wikipedia), which results in the change in data distribution. therefore, directly using previous datasets or officially provided synthetic corpus @xcite to enlarge the training set of ape tasks might not be appropriate under such circumstances. in work by @xcite , considering the change of data distribution, they select to use additional mt candidates as the data augmentation method to improve feature diversity in their ape systems, which significantly improves the ape performance. inspired by this idea, we decided to solve the ape task as nmt alike task and utilize the external mt at the fine-tuning stage. however, because of the limited size of the ape corpus and the improvement of mt quality, fine-tune the model only on the ape data, easily reach the performance ceiling in spite of using external translation. to solve the aforementioned issues, existing works for other natural language processing (nlp) tasks have adopted several multi-task learning (mtl) methods with the auxiliary task @xcite . we wondered whether it is possible to apply mtl mechanism with ape task to the fine-tuning stage since mtl trains the model to encourage representation sharing and improve generalization performance. furthermore it aims to alleviate the data sparsity problem with a limited number of data in each task @xcite . therefore, we add some related nlp tasks along with the ape task. our experiment results demonstrate that such approaches can further improve performance. as mentioned above, large-volume data, such as news translation data and artificial synthetic data, can not be used to enlarge the ape corpus directly during the fine-tuning because of the large gap in data distribution. we wondered if there is a way to apply any learning method to the post-training so that we can utilize more data to train a more robust and powerful model. in work by @xcite , they applied curriculum learning according to the difficulty of each example on a single training stage. inspired by the research, we try to apply curriculum learning across multiple training stages. as the training stage increases, we make the system learn to solve the different tasks by gradually providing extra information, described in section 3 in detail. extensive experiments show the effectiveness of applying the curriculum learning strategy during the training phase. finally, we combined these two approaches to make our final ape system, which significantly improves the performance of the ape task. our ape system is built based on transformer @xcite and is post-trained on wmt21 news-translation data @xcite @xcite @xcite and artificial synthetic data @xcite provided by ape task with curriculum learning strategy. for finetuning, mtl is applied with related nlp subtasks such as part-of-speech (pos), named entity recognition (ner), masked language model (mlm), and keep/translate are added to the model to reduce the over-fitting as well as achieve better performance, described in section 4 in detail. for better training efficiency, the dynamic weight average (dwa) mechanism @xcite is applied during the mtl to keep the correct balance between these subtasks. here we summarize our contributions as follows: ## base system our system is based on facebook fair's wmt19 news translation model @xcite , which used the big transformer @xcite and provided the pre-trained weights. we use both of them as our base system. in addition, we utilize data augmentation with external mt, which has been proposed by @xcite to generate the external translated sentence (mt_ext) and help generate the post-editing sentence (pe). an input sentence x that contains a source sentence (src), a translated sentence by the machine translation system (mt), and an external translated sentence (mt_ext) is defined as, and output a sequence, n+m+l+2) , where d h represents a dimension of the encoder, and n, m, l represents the number of tokens for src, mt, mt_ext, respectively. we represent the parameters of the encoder as θ s . then, h is fed into the decoder, and the decoder target is defined as @xmath0. ## curriculum training strategy (cts) cts has been inspired by curriculum learning @xcite ) that is applied according to the difficulty of each example on a single training stage, which has already been applied to our baseline architecture by @xcite . in addition, we propose cts, which applied curriculum learning across multiple training stages. cts aims at stepby-step learning. in an early stage, the system learns to solve easy problems or something that needs to know beforehand and complex problems or target tasks in the later stages. ## multi-task learning strategy (mls) existing works for mtl propose jointly learning methods among related tasks. mtl aims to improve the generalization performance of the whole tasks by sharing knowledge representations of other tasks and can also alleviate the data sparsity problem where each task has limited labeled data @xcite . therefore, we utilize mls for our system because wmt21 ape shared task provides only 7,000 train sentences. in nmt, existing works for mtl applied pos, ner, or mlm as subtasks and provided improved results @xcite . despite the impressive results, they applied only a few subtasks, such as one or two. since we defined the ape task as nmt alike problem in our work, it would be helpful to leverage these subtasks into our work to achieve better performance. we find out that all these subtasks are cooperative with each other and benefit our system. inspired by the word-level quality estimation task, we also add the keep/translate classification tasks for encoder and decoder to handle the high-quality ape task, which is described in section 4.2 in detail. since utilizing multiple subtasks, we have to consider the loss ratio between these subtasks. in our work, we apply the dynamic weight average method described in @xcite , and more details are described in section 4.4. our final system based on the model post-trained using cts with fine-tuning the ape data with mls. ## conclusion in this paper, we propose an ape system based on cts and mls. cts allows understanding between machine translation and automatic post-editing, and shows a way using additional data in large volume in ape task. mls learns a shared unified representation from related subtasks to improve the performance. we submitted the system, which fine-tunes with mls, as our primary version and the ensembled cts as our contrastive version. the experimental results show that our system is able to effectively detect and correct the errors made by a high-quality nmt system, improving the score by -2.848 and +3.74 on the development dataset in terms of ter and bleu, respectively. our proposed methods also achieved performance improvement on the test dataset with higher quality.