Prompt-Driven Neural Machine Translation (2022)

Abstract: Neural machine translation (NMT) has obtained significant performance improvement over the recent years. However, NMT models still face various challenges including fragility and lack of style flexibility. Moreover, current methods for instance-level constraints are limited in that they are either constraint-specific or model-specific. To this end, we propose prompt-driven neural machine translation to incorporate prompts for enhancing translation control and enriching flexibility. Empirical results demonstrate the effectiveness of our method in both prompt responding and translation quality. Through human evaluation, we further show the flexibility of prompt control and the efficiency in human-in-the-loop translation.

## introduction neural machine translation (nmt) has achieved much performance improvement over the recent years @xcite @xcite ), yet still faces various challenges such as low cross-domain robustness @xcite , fragility @xcite and lack of style flexibility @xcite . to address these issues, a line of work considers introducing constraints to the translation outputs, typically in the form of lexical constraints @xcite and style control @xcite @xcite . for example, @xcite ensure that polysemous words are translated to their domain-specific senses in ecommerce. such instance-level constraint has been shown useful for improving both the translation adequacy and readability in practical applications @xcite @xcite @xcite . however, they are limited in being (1) model-specific and (2) prompt: the translation should include "on the desk" prompt: " " should be translated before " " translation: the apple pie on the table was eaten by me yes -terday. translation: yesterday, i ate the apple pie on the desk. prompt: " " should be translated into "apple pie" translation: yesterday, i ate the apple pie on the table. prompt: the translation should begin with "i" translation: i ate the apple pie on the table yesterday. ) based on the given prompts. one can specify phrase translations, guarantee translation positions or alter word order by feeding the system with different prompts. without losing generality, we consider the forms of constraints in figure 1 in this work. building on a standard transformer @xcite baseline, we consider the following research questions. first, what is the most effective system architecture for encoding both the source sentence and the prompt? to this end, we compare various methods including concatenating source sentences with prompts, encoding prompts using a dedicated module, and incorporating prompt representations with an attention layer. the model performance is also compared with previous work on lexical constraints, a form of constraints in figure 1 that has been much studied in the literature. second, can different types of constraints be effectively trained within the same model? to this end, we design an algorithm to automatically construct different types of prompts from a standard mt training corpus, training a model with mixed prompts. third, can a prompt-driven nmt system accept different number of prompts, while maintaining the same level of performance compared to a transformer baseline without constraints? to this question, we consider a sampling-based training strategy, where the model receives random combinations of arbitrary number of prompts or no prompt at all for each sample during training. fourth, can the set of flexible constraints we use serve to improve the efficiency of human-in-the-loop translation? we deploy our prompt-drive system in a real application scenario where professional translators conduct machine translation post editing (mtpe) by using prompts. empirical results show that the prompt-driven transformer (prompt-transformer) responds to different prompts effectively, while giving competitive performance when used as a unconstrained nmt model. in addition, prompt-driven model outperforms previous lexical constraints methods @xcite ) by a large margin. human experiments further demonstrate the control flexibility and effectiveness of our method. through system deployment in a practical scenario, we show that the prompt-driven nmt system achieves a trade-off between translation quality and human efficiency, as compared with full nmt or nmt with human post editing. our code is released on https://github.com/yafuly/promptnmt . ## related work lexical constraint has received much attention for machine translation. some researchers incorporate the constraints into the beam search algorithm @xcite , and recently @xcite investigate alignmentbased constrained decoding methods using attention weights. another approach focuses on data augmentation. song et al. (2019) and dinu et al. (2019 ) create a synthetic code-switching corpus. @xcite augment the input sentences with lemmatized constraints to correct inflection. @xcite propose a lexical constraint-aware transformer model (leca) by concatenating constraints and source sentence. lexical constraints is one of the application scenarios of our method. prompt-driven model gives strong results, while also simultaneously enables structural and style constraints with the versatility of prompts. there has been study on controlling the global output style in mt @xcite @xcite @xcite . @xcite analyze the impact of dialogue specific aspects in smt for fictional dialogues. @xcite employ personalized smt models for better preservation of gender traits, and @xcite propose to adapt the bias of the output softmax to different users of an nmt system. @xcite use target-constraint t-v annotation in nmt training to control the level of politeness. @xcite propose a formality-sensitive nmt model taking formality levels as an extra input. our work is similar in that the output of our model can be adaptive at inference time, but different in that the control is more fine-grained and not limited to certain styles. human in the loop for nmt @xcite has been proved effective to domain adaptation. @xcite propose an interactive framework which takes two human actions: picking a critical translation error and revising the translation. @xcite propose a simple sentence-level weighting method to integrate partial chunk-based feedback into nmt. kreutzer et al. (2018) improve nmt with explicit and implicit user feedback collected on the ecom-prompt-encoder encoder training deployment output y: the apple pie on the desk was eaten by me yesterday. (x,y) translation prompts prompt candidate pool target-constraint prompts ordering prompts generate prompt p sample translator type prompts after reading the input x prompt-driven transformer input x: </@xmath0: the overall framework of prompt-transformer. during training, the prompts are sampled from the prompt candidate pool, which contains all possible prompts for each sentence pair. in deployment, the translators give arbitrary prompts to control output translations according to their needs. merce platform. domingo et al. (2019) leverage data generated during the post-editing process. the above methods improve the performance of nmt by leveraging extra training signals from human feedback. different from them, our method allows human to control the nmt output by training a model with mixed prompts, without the requirement of human in training. ## problem definition in neural machine translation, a set of parallel sentence pairs @xmath1, y )} is given where @xmath2, ..., x tx ) and @xmath3, ..., y ty ), and the nmt systems model the conditional probability: the prompts can be general and flexible. in this paper, we consider the following three types of common prompts: ## approach the overall architecture of our system is shown in figure 2 . in particular, we take a transformer baseline (section 4.1), discussing different ways to additionally encode prompt constraints (section 4.2). we propose a sampling-based training framework (section 4.4), with automatic methods for generating rich constraints from standard mt training instances (section 4.3). ## experiments on the model design we evaluate models under two test scenarios using iwslt'14 de-en: inference without prompt and inference with prompt. the former is the same as the vanilla machine translation setting and is evaluated using bleu score. for the latter, we also evaluate the model's effectiveness on responding to prompts by calculating resr. we apply sampling strategy same to training and run on the test set once to build a deterministic prompt sets. 0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00 90.00 100.00 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 bleu score/resr ratio bleu w/o prompts bleu w/ prompts resr figure 4: resr and bleu scores with respect to the bernoulli ratio during training. number of prompts during decoding. we investigate how prompts improve translation performance by feeding different number of prompts during decoding. specifically, we randomly select certain number of prompts from the prompt candidate pool and construct test prompts accordingly. the results are shown in figure 3 . prompt-transformer further achieves higher bleu scores when there are more prompts. given as many as 10 prompts, the bleu reaches 60.59. we also investigate how the sampling ratio affects decoding performance, which is discussed in appendix c. robustness to different prompts. we explore how the model behaves under different prompt sets, by fixing the sampling ratios but varying the seed for prompt sampling. we conduct experiments with 10 seeds, under which the model receives different prompts for translation, calculating the mean and standard deviation of bleu scores and resr over each seed. for each sentence, the model is provided with 1 to 8 sampled prompts. the model achieves a average bleu score of 54.79 with a standard deviation of 0.17, and an average of 92.82 with a standard deviation of 0.14 for resr, demonstrating that the model is stable for flexible types of prompt combinations. influences of model architecture. based on the modules in section 4.2, we compare different model architectures to incorporate prompts using a fixed prompt seed. as shown in number of prompts during training. the sampling strategy in section 4.4 can affect the performance. we investigate how varying the bernoulli ratio during training affects the model performance. the bernoulli ratio indicates how many of samples in the train set are driven by prompts. for example, a bernoulli ratio of 0.3 denotes 65.7% of the training samples are provided with prompts. the result is shown in figure 4 . we can observe that resr grows steadily with the increasing ratio during training. the model gives a low resr with a bernoulli ratio of 0.1, as there are limited samples for the model to capture prompt patterns. despite the increasing resr, there is a sharp decline on bleu scores when the ratio exceeds 0.5. this is because high bernoulli ratios indicate almost all training samples are prompted (e.g., a ratio of 0.7 denotes 97.3% of training samples are provided with prompts). therefore, the model learns to output translations by over reliance on prompts, but fails to build correspondence between source and target languages. thus it is important to balance the learning of translation and receiving prompts. comparison with existing work on lexical constraints. among the types of prompts we accommodate, lexical constraints have been investigated by existing work. we compare our method with two typical methods, i.e., codeswitch @xcite and leca @xcite . following song et al. (2019) and chen et al. (2021b), the copy success rate (csr) is also calculated, which is the percentage of successfully generated tokens in constraints, differing from resr which is the ratio of correctly responded prompts (i.e., phrases for lexical constraints). compared with codeswitch, prompt-transformer maintains better performance without prompts, while also achieves a higher score of csr and resr. although leca is slightly better in terms of bleu without prompts, prompt-transformer outperforms leca by a large margin in terms of csr and resr. performance in lexical constraints further demonstrates the effectiveness of our method for controlling translation and meanwhile maintaining performance without prompts. experiments on wmt. for a large scale test, we apply prompt-transformer on the wmt'17 @xmath4. based on the preliminary experiments, we choose the param-share prompt encoder architecture. as shown in ## experiments on prompts we evaluate how model responds to prompts in practical scenarios, where no "gold-standard" references are given. we sample 100 source sentences from the wmt'17 en-zh test set and ask 2 professional translators to assign each sentence with two different prompt groups, each of which includes at least one type of prompts. in particular, for constructing translation prompts, the translators are asked to give a source segment two different valid translations (e.g., "translationsegment-1" or "translation-segment-2"); for constructing target-constraint prompts, the translators should choose two different ways to prompt the model; for constructing ordering prompts, the translators provide two opposite orderings (e.g., "source-segment1" should be translated before and after "source-segment2", respectively). the model is expected to output two different and correct trans-lations corresponding to the two prompt groups, respectively. we ask 3 professional translators to evaluate the resr and translation quality based on the adequacy and fluency metrics in section 5. the system achieves resr scores of 89.80, 94.74, 90.20 for translation prompts, targetconstraint prompts, and ordering prompts, respectively, showing the effectiveness of our proposed model on responding to human prompts. the system obtains a competitive performance compared to the unprompted baseline in tuns of both adequacy (3.49 vs. 3.40) and fluency (3.24 vs. 3.31), demonstrating that our system can enable flexible translation style and maintain translation quality at the same time. ## human-in-the-loop translation machine translation post-editing (mtpe) is widely used by translation companies to improve efficiency as well as ensure translation quality. studies show that conducting post-editing over highquality mt can increase the productivity of professional translators compared to manual translation 'from scratch' @xcite . however, mtpe still can be expensive in heavy involvement of human efforts in editing. to alleviate human labor, prompt-driven methods can be used for a better trade-off between translation quality and efficiency. to verify our hypothesis, we ask professional translators to compare two methods for editing on mt translations: the traditional mtpe or giving prompts based on mt translations (mtprompt). we compare mt, mtpe and mtprompt based on time efficiency and translation quality. mt refers to use machine translations without editing. for 0.5 1.0 1.5 2.0 2.5 p1 p2 p3 p4 time translator1 translator2 figure 6: time cost (hours) for mtprompt with respect to the round of mtprompt. mtpe, translators are required to edit translations output by the wmt-trained transformer baseline in section 6. for mtprompt, translators are required to observe output translation errors and give prompts to correct them. more details are presented in appendix e. the translation quality is presented in figure 5 . mtpe achieves full marks on both adequacy and fluency, whereas the scores for mt translations are on average around 2.5. translations with prompt obtain substantial improvement over mt translations, with both the adequacy and fluency scores being close to 4 (i.e., the translations cover most meaning and also have good fluency). in terms of speed, the average time spent on mtpe is 3.75 hours, which is stable for more batches since the translators have strong experience in mtpe. in contrast, the time cost can be lower as they conduct more mtprompt actions. we ask two translators to conduct multiple rounds of mtprompt edit, with each round containing 50 translations. the time cost for each round is shown in figure 6 . we can observe that as the translators get familiar with the mtprompt mode, they become more efficient in giving prompts. the fastest batch costs an average of 1.1 hours for mtprompt, which is 2.4 times more efficient than mtpe, and meanwhile translation quality is maintained (adequacy: 3.87 vs. 3.84 and fluency: 3.71 vs. 3.63). ## conclusion we proposed a prompt-driven transformer model to incorporate flexible constraints on translation. under a sampling-based training framework, the model learned prompt responding effectively and achieved competitive performance compared with both the unconstrained baseline and existing work on lexical constraints. human experiments further demonstrated that prompt-transformer was able to respond to various combinations of prompts accurately, and generate versatile translations. through deployment in an application scenario, we showed that our system could serve to improve the efficiency of human-in-the-loop translation.