LUL ’s WMT 22 Automatic Post-Editing Shared Task Submission (2022)

Abstract: By learning the human post-edits, the automatic post-editing (APE) models are often used to modify the output of the machine translation (MT) system to make it as close as possible to human translation. We introduce the system used in our submission of WMT’22 Automatic Post-Editing (APE) English-Marathi (En-Mr) shared task. In this task, we first train the MT system of En-Mr to generate additional machine-translation sentences. Then we use the additional triple to bulid our APE model and use APE dataset to further fine-tuning. Inspired by the mixture of experts (MoE), we use GMM algorithm to roughly divide the text of APE dataset into three categories. After that, the experts are added to the APE model and different domain data are sent to different experts. Finally, we ensemble the models to get better performance. Our APE system significantly improves the translations of provided MT results by -2.848 and +3.74 on the development dataset in terms of TER and BLEU, respectively. Finally, the TER and BLEU scores are improved by -1.22 and +2.41 respectively on the blind test set.

## introduction automatic post-editing (ape) is the task of automatically editing the translations of mt system. by using ape models, we can transfer mt system from general domain to specific domain and then reduce the workload of human post-edits. wmt has been holding ape task competitions in different languages and fields since 2015. now the ape models are often based on transformer and improved on this basis. wmt 2022's automatic post-editing task focused on english-marathi language pairs. the difference from the previous competition is that the target language is changed to marathi. besides, two new fields, medical care and tourism, 1 work performed during internship in samsung research china -beijing are added. participants are provided a training set with 18000 instances, a development set and a test set with 1000 instances respectively. each dataset consists of source, machine-translation and postedit triplets. the source sentences in english come from the healthcare, tourism, and general/news domains. the mt outputs are automatic translations to marathi. the post-edits are human revisions of the target elements. this synthetic training data is prepared as a part of the 2022 ape shared task. the data is created by taking a parallel corpus, where the source data is translated using an mt system, and the references are considered as post-edits. participants are also allowed to use any additional data for systems training. last year's research mainly focused on transfer learning and data augmentation. @xcite utilizes the most advanced en-de machine translation model and further fine tune the ape dataset on this basis. we adopted the same strategy to train our baseline model with transfer learning and data augmentation. due to the lack of a ready-made machine translation model of en-mr as the basis of the ape model, we trained an ape model by using synthetic data and additional data. the ape model is then further fine-tuned with the ape dataset, which is data enhanced. in order to make use of the domain information in the training dataset, we use the mixture of experts structure and add adapter modules in the transformer, so that different adapters can learn the distribution of different domain information, thus improving the translation performance. the contributions of this work are as follows. (1) data augmentation. we trained an external mt to obtain more data sets consistent with ape tasks. at the same time, we use google translation to back translate the postedits in the training set. the dataset is composed as follows: back @xmath0. on the other hand, we take @xmath1. (2) mixture of adaptors. we implement the mixture of experts structure to deal with inputs from different domains, in which we use lightweight adapters as experts and introduce a classifier for expert routing. considering the effect of directly initializing the adapter for training is not good enough, so we first set an adapter in the model and pre-train the model to obtain the adapter weight w 0 . since the training set comes from three different fields, we add two additional adapters, which will read w 0 as a parameter for initialization. we freeze all model parameters when training the model, and only fine tune the weights of the three adapters. considering the experience of previous competitions, we used the existing data to train an en-mr translation model as a data augmentation method due to the lack of advanced en-mr translation model. inspired by the moe, the built-in adapter module enables the model to learn three data distributions at the same time to improve the performance of translation. ## model we describe our baseline model followed by the details of domain and task adaptation in this section. ## conclusion in this paper, we first use the data augmentation method to build the src' <@xmath2. we suppose that the enhanced datasets can effectively improve the performance of the ape model. the experimental results show that the data augmentation method we used is effective. at the same time, it also shows that adding pe can make the model automatically learn the rules of human post-edits. after that, we draw lessons from mixture of experts. we add adapters in the ape baseline model. and we let the training data be sent to different adapters through the trained classifier so that the model can further learn the post-editing rules in different translations. the experimental results confirm that our system can modify the output of mt system with high efficiency and quality. compared with baseline, the ter and bleu scores are improved by -1.22 and + 2.41 respectively.