HW - TSC ’s Participation in the WMT 2023 Automatic Post Editing Shared Task (2023)

Abstract: The paper presents the submission by HW-TSC in the WMT 2023 Automatic Post Editing (APE) shared task for the English-Marathi (En-Mr) language pair. Our method encompasses several key steps. First, we pre-train an APE model by utilizing synthetic APE data provided by the official task organizers. Then, we fine-tune the model by employing real APE data. For data augmentation, we incorporate candidate translations obtained from an external Machine Translation (MT) system. Furthermore, we integrate the En-Mr parallel corpus from the Flores-200 dataset into our training data. To address the overfitting issue, we employ R-Drop during the training phase. Given that APE systems tend to exhibit a tendency of ‘over-correction’, we employ a sentence-level Quality Estimation (QE) system to select the final output, deciding between the original translation and the corresponding output generated by the APE model. Our experiments demonstrate that pre-trained APE models are effective when being fine-tuned with the APE corpus of a limited size, and the performance can be further improved with external MT augmentation. Our approach improves the TER and BLEU scores on the development set by -2.42 and +3.76 points, respectively.

## introduction automatic post-editing (ape) is a post-processing task in a machine translation (mt) workflow, aiming to automatically identify and correct errors in mt outputs @xcite . wmt has been holding ape task competitions in different languages and fields since 2015. similar to wmt 2022, wmt 2023's ape task still focuses on the en-mr language pair. participants are provided with a training set comprising 18,000 instances, a development set, and a test set, with each containing 1,000 instances. each dataset consists of * work done during internship at huawei triplets -the source (src) sentences, the corresponding machine-translation (mt) outputs, and the human post-edited versions (pe) of the translations. in this task, the source sentences have been translated into the target language by using a stateof-the-art neural mt system to get the machinetranslation data. the provided data encompasses diverse domains, such as healthcare, tourism, and general/news. in addition, the synthetic training data is offered to participants, which is created by taking a parallel corpus, where the source data is translated using an mt system, and the references are considered as post-edits. furthermore, participants are permitted to utilize any additional data for systems training. typically, training an ape model requires large amount of training data. however, obtaining pe is an expensive task in terms of time and money. as a result, there exists a scarcity of large-scale ape datasets. to address this challenge, numerous data augmentation techniques have been proposed @xcite @xcite @xcite . @xcite augment the ape training data with translations generated using a different mt system. @xcite train an external mt to obtain more datasets consistent with ape tasks. they also use google translation to back translate the post-edits in the training set. deoghare and bhattacharyya (2022) augment the ape data by generating phrase-level ape triplets using smt phrase tables. to ensure the quality of the synthetic data, they employe the labse technique @xcite to filter low-quality triplets. in our method, we use google translation to back translate the post-edits in the training set. subsequently, our dataset is structured as follows: the concatenation of source sentence, back translation and machine translation as the input, while the post-edits serve as the reference output. additionally, we incorporate en-mr parallel sentences from @xcite dev and test data to our training set. given that we have an en-mr parallel corpus only and lack machine translation data, we directly utilize english sentences as the source input and marathi sentences as the postedits. furthermore, we use r-drop @xcite , which regularizes the training inconsistency induced by dropout and has been proven beneficial for different kinds of models. @xcite have proven that ape systems often make unnecessary edits to translation output. to mitigate this issue of over-correction, we employ a sentence-level qe system to determine the final output, selecting between the ape system's output and the original machine-translated (mt) version. when being evaluated on the development set, our approach improves the ter @xcite by -2.42 points and the bleu score @xcite by +3.76 points. the contributions of our work are as follows: deoghare and bhattacharyya (2022) use two separate encoders to generate representations for src and mt. they also employ a pre-trained language model to initialize the weights for both our encoders. for data augmentation, they leverage external mt candidates and generate phrase-level ape triplets using smt phrase tables. furthermore, they filter low-quality ape triplets from the synthetic data using labse-based filtering. they also use a sentence-level qe system to select the final output between the ape-generated output and the original translation. with experience in previous competitions, we also utilize an external mt system to generate backtranslations. additionally, we adopt a sentencelevel qe system for selecting the final output. ## model figure 1 shows the architecture of our ape model. in this section, we provide the details of our approach. ## conclusion this paper presents our ape system submitted to the wmt 2023 ape english-marathi shared task. in our approach, we initially employ the data augmentation method to build the [src, 
, src', 
, mt] additional training datasets. we augment our training data by incorporating the en-mr parallel sentences from flores-200 dataset. we mitigate overfitting by employing r-drop during the training phase. moreover, we explore the sentence-level qe system to discard low-quality ape outputs. evaluation of our ape system shows that our approach achieves significant gains on the wmt-22 ape development sets.