P e P e: Personalized Post-editing Model utilizing User-generated Post-edits (2023)

Abstract: Incorporating personal preference is crucial in advanced machine translation tasks. Despite the recent advancement of machine translation, it remains a demanding task to properly reflect personal style. In this paper, we introduce a personalized automatic post-editing framework to address this challenge, which effectively generates sentences considering distinct personal behaviors. To build this framework, we first collect post-editing data that connotes the user preference from a live machine translation system. Specifically, real-world users enter source sentences for translation and edit the machine-translated outputs according to the user’s preferred style. We then propose a model that combines a discriminator module and user-specific parameters on the APE framework. Experimental results show that the proposed method outperforms other baseline models on four different metrics (i.e., BLEU, TER, YiSi-1, and human evaluation).

## introduction language usage is strongly influenced by the state of the individual, which can be considered by multiple attributes such as age, gender, socioeconomic status, and occupation @xcite . taking these aspects into account in the machine translation task, we need personalized translations to reflect individual characteristics that vary from person to person; thus, the translation system should consider not only fluency and content preservation, but also personal style. however, most existing neural machine translation (nmt) models ignore personal style @xcite . previous studies attempt to address this problem by personalizing the nmt models, but in these studies the definition of personal style figure 1 : example of a personal post-editing triplet (i.e., source (src), machine translation (mt), and post-edit (pe)) given the source text in english and the translated text in korean. a post-edited sentence does not only contain error correction of an initial machine translation result but also reflects individual preference. for instance, a human post-editor modifies the word "primarily" to "primary," but also change "공헌" to its synonym "기여" while keeping the rest as it is (e.g., "research"). in contrast with previous studies, we propose a method based on an ape framework and newly utilize post-editing data to capture diverse personal traits in translation. originally, the need for postediting data is to improve the quality of machinetranslated sentences in an ape task @xcite @xcite . however, we suggest that the post-editing data can also be adequate references for personalized translation if various users post-edit sentences according to their preferences. in this respect, we collect a user-generated post-editing dataset called usp through a live translation system. after the system translates a source sentence (src) to a target sentence, i.e., machine-translated sentence (mt), each user edits the translated result according to their purpose or preferences, i.e., post-edited sentence (pe). we collect (src, mt, pe) triplets called personalized post-editing triplets for each user and an example is depicted in fig 1 . along with the personalized post-editing data, we develop a model which utilizes user parameter and a discriminator module. the user-specific parameters allow the model to adapt to each user in that the model can consider inter-personal variations. these parameters are aggregated with the output word probability such that the generation word probability distribution differs by each particular user. moreover, since the prevalence of pretrained language models encourages significant performance improvements on various natural language generation tasks @xcite @xcite , we exploit the pre-trained language model (lm) but do not fully lean on it. we assume that not all the features from the pre-trained lm contribute to capturing the distinct taste of users that are departing from the neutral and standardized patterns. thus, our discriminator module, inspired by adversarial training @xcite , attempts to dismantle the unnecessary features from a pre-trained lm, while tuning the model to incorporate a personal style. the details will discuss in section 3. experiments on our dataset and speaker annotated ted talks dataset @xcite (sated) demonstrate that the proposed approach generates diverse translations for different users. in summary, our contributions include the following: our work is closely related to the recent work on personalized neural machine translation and auto-matic post-editing. ## proposed method overview: it is challenging to generate appropriate translations that impose personal variations. to address such a demanding problem, we take a detour by applying an ape framework. we propose pepe, a personalized post-editing model utilizing user-generated post-edits. pepe includes a discriminator module to allow the model to dismantle the pre-trained lm features. specifically, we maximize the discriminator loss to encourage the encoder to throw away irrelevant pre-trained lm features, while minimizing the ape loss to guide the model to utilize the pre-trained lm features that are useful for personalization. in addition, pepe utilizes userspecific parameters to capture the personal style. user-specific parameters are combined at the end of the decoder layer to adjust the prediction of the word probability, i.e., the word choice based on a user preference. our strategy does not require expensive supervision on the personal style, such as explicit attribute labeling or an attribute-tailored model architecture. the overall architecture of pepe is illustrated in fig. 2 . the two following subsections will describe the modules shown in fig. 2-(a ), (b), (c), (d), and (e), respectively. ## experiments in this section, we qualitatively and quantitatively demonstrate the effectiveness of our proposed method. we validate pepe, described in section 3, against other baseline methods using a real-world user dataset usp. we also provide a detailed explanation for the dataset. moreover, through extensive experiments and analyses, we show that pepe can incorporate inter-personal variations into a target sentence. we provide training details in appendix a. ## conclusion in this work, we propose a personalized postediting method, pepe, utilizing user-generated postedits. based on the ape framework, pepe leverages two modules, 1) user-specific parameters and 2) a semantic cluster-based discriminator module. these modules lead to reflect the multifarious interpersonal variations, where the former allows the model to learn user-dependent probabilities for each word while the latter unlearns the detrimental features in a pre-trained language model and maintains advantageous effects of the transfer learning. we empirically demonstrate that pepe reflects finegrained user preference in a variety of settings. to the best of our knowledge, this work is the first attempt to utilize the ape framework with the usergenerated post-edits for personalized translation. we believe that our work can draw more attention toward personalized translation, which is the ul-timate direction that the neural translation model should go forward.