Improved Data Augmentation for Translation Suggestion (2022)

Abstract: Translation suggestion (TS) models are used to automatically provide alternative suggestions for incorrect spans in sentences generated by machine translation. This paper introduces the system used in our submission to the WMT’22 Translation Suggestion shared task. Our system is based on the ensemble of different translation architectures, including Transformer, SA-Transformer, and DynamicConv. We use three strategies to construct synthetic data from parallel corpora to compensate for the lack of supervised data. In addition, we introduce a multi-phase pre-training strategy, adding an additional pre-training phase with in-domain data. We rank second and third on the English-German and English-Chinese bidirectional tasks, respectively.

## introduction translation suggestion (ts) is a scheme to simplify post-editing (pe) by automatically providing alternative suggestions for incorrect spans in machine translation outputs. @xcite formally define ts and build a high-quality dataset with human annotation, establishing a benchmark for ts. based on the machine translation framework, the ts system takes the spliced source sentence x and the translation sentence m as the input, where the incorrect span of m is masked, and its output is the correct alternative y of the incorrect span. the ts task is still in the primary research stage, to spur the research on this task, wmt released the translation suggestion shared task. this wmt'22 shared task consists of two subtasks: naive translation suggestion and translation suggestion with hints. we participate in the former, which publishes the bidirectional translation suggestion task for two language pairs, english-chinese and english-german, and we participate in all language pairs. * yufeng chen is the corresponding author. our ts systems are built based on several machine translation models, including transformer @xcite , sa-transformer @xcite , and dynamicconv @xcite . to make up for the lack of training data, we use parallel corpora to construct synthetic data, based on three strategies. firstly, we randomly sample a sub-segment in each target sentence of the golden parallel data, mask the sampled sub-segment to simulate an incorrect span, and use the sub-segment as an alternative suggestion. secondly, the same strategy as above is used for pseudo-parallel data with the target side substituted by machine translation results. finally, we use a quality estimation (qe) model @xcite to estimate the translation quality of words in each translation output sentence and select the span with low confidence for masking, and then, we utilize an alignment tool to find the sub-segment corresponding to the span in the reference sentence and use it as the alternative suggestion for the span. considering that there is a domain difference between the synthetic corpus and the humanannotated corpus, we add an additional pre-training phase. specifically, we train a discriminator and use it to filter sentences from the synthetic corpus that are close to the golden corpus, which we deem as in-domain data. after pre-training with large-scale synthetic data, we perform an additional pre-training with in-domain data, thereby reducing the domain gap. we will describe our system in detail in section 3. ## related work the translation suggestion (ts) task is an important part of post-editing (pe), which combines machine translation (mt) and human translation (ht), and improves the quality of translation by correcting incorrect spans in machine translation outputs by human translators. to simplify pe, some early scholars have studied translation prediction @xcite al. (2014), knowles and koehn (2016) ), which provides predictions for the next word (or phrase) when given a prefix. and some scholars have also studied prediction with the hints of translators @xcite . in recent years, some scholars have devoted themselves to researching methods to provide suggestions to human translators. @xcite present a proof-of-concept interactive translation system that provides human translators with instant hints and suggestions. @xcite utilize two quality estimation models and a translation suggestion model to provide alternatives for specific words or phrases for correction. @xcite propose a transformer model based on segment-aware self-attention, provide strategies for constructing synthetic corpora, and released the human-annotated golden corpus of ts, which became a benchmark for ts tasks. ## method in this section, we describe the translation suggestion system, followed by our strategies for building synthetic corpora, and finally the details of the additional pre-training phase. ## conclusion we describe our contribution to the translation suggestion shared task of wmt'22. we propose a strategy to construct synthetic data with the quality estimation model to make the constructed data closer to the real scenarios. furthermore, we introduce an additional phase of pre-training with in-domain data to reduce the gap between synthetic corpus and golden corpus. experimental results demonstrate the effectiveness of our strategy. considering the heavy labor of annotating ts data, we think data augmentation is the most important strategy that should be addressed. in the future, we will put more effort into the data generation method, to make the most of openly-accessible parallel data.