Controlling Styles in Neural Machine Translation with Activation Prompt (2023)

Abstract: Controlling styles in neural machine translation (NMT) has attracted wide attention, as it is crucial for enhancing user experience. Earlier studies on this topic typically concentrate on regulating the level of formality and achieve some progress in this area. However, they still encounter two major challenges. The first is the difficulty in style evaluation. The style comprises various aspects such as lexis, syntax, and others that provide abundant information. Nevertheless, only formality has been thoroughly investigated. The second challenge involves excessive dependence on incremental adjustments, particularly when new styles are necessary. To address both challenges, this paper presents a new benchmark and approach. A multiway stylized machine translation (MSMT) benchmark is introduced, incorporating diverse categories of styles across four linguistic domains. Then, we propose a method named style activation prompt (StyleAP) by retrieving prompts from stylized monolingual corpus, which does not require extra fine-tuning. Experiments show that StyleAP could effectively control the style of translation and achieve remarkable performance.

## introduction natural language texts can be written in various styles while preserving the content, such as politeness, formal, classical, and many others @xcite @xcite . styles are crucial for communication since every sentence should fit a specific scenario and the appropriate style makes it more user-centric. a speaker needs to switch the styles of words to adapt to different conditions. using the inappropriate style can be impolite or ridiculous in some societies and result in serious cultural conflicts @xcite . in the andes, this glacier is the source of drinking water for this city. nos andes, todo o gelo é a principal fonte de água potável para toda cidade. nos andes, essa geleira é a fonte de água potável para toda cidade. what's up guys, i'm cole. recently, controlling style in machine translation has also drawn much attention in the academic community @xcite @xcite . formally, stylized machine translation refers to translating the source sentence into different styles with certain attributes while the translation quality remains satisfactory, as the cases showed in figure 1 . many previous studies have explored the task and gained promising results @xcite @xcite . however, challenge still remains in two aspects. the first challenge is about the benchmark. the style of natural languages consists of many aspects like word preference and grammar structure. however, the well-studied benchmark datasets mainly focus on the formality and politeness of european languages. due to this limitation, previous work re-stricts styles to a relative narrow scene. in addition, most of the test sets of the previous work have only one reference rather than multiple stylized references, which hinders the automatic evaluation for different styles. as such, a benchmark involving more diverse styles, multiple stylized references and beyond european languages is greatly needed. the second challenge is about the iterative training framework. most related work heavily relies on fine-tuning with new stylized data @xcite . basically, they collect stylized bilingual texts and append tags before the sentence, then conduct fine-tuning to adapt the model to the given style @xcite . however, parallel data in specific styles is pretty sparse and costly to gather. furthermore, in this way, we have to re-tune the model every time we want to add new styles, which is inconvenient. correspondingly, this paper contributes in terms of both benchmark and approach: for the benchmark, we re-visit this task and push the boundary of styles to a wider range of language phenomena. we propose a dataset msmt, including four directions with diverse language styles. we collect related public corpus as training sets and provide newly labeled sentences as test sets. each source sentence has two references in different styles, which is convenient for automatic evaluation. by broadening the category and providing standard datasets, we hope to effectively push the development of this field. for the approach, we propose style activation prompt (styleap), a method to avoid re-tuning time after time. the main idea is to extract one sentence of the target style as a prompt to guide the main sentence translation style. the intuition is straightforward. we assume that once the model has been trained on all kinds of data with various styles, it has the potential to generate any style as far as correctly activated. we can activate the ability by language model since it tends to maintain the sequence consistency @xcite . and the prompt can be easily retrieved in a specific stylized monolingual corpus. in a word, we can obtain a "plug-and-play" model for any new generation style with mere stylized monolingual data instead of iterative fine-tuning. the experiments show that our approach achieves explicit style transformation while well maintaining the text semantics. ## task definition & msmt: a multiway stylized translation benchmark stylized machine translation refers to the translations with certain language characteristics or styles on the basis of ensuring the quality of translation. based upon the definition, we construct a stylized machine translation benchmark including four language directions. in each language direction, we give the illustration of various styles and provide corresponding training and test sets. different from traditional stylized machine translation studies, each group of our test sets is consist of one single source and multiple references in parallel. for example, for english-to-chinese direction, for each english source sentence, we have two parallel chinese references: classical style and modern style. in this way, we can automatically evaluate the style transformation by measuring the similarity between the stylized hypothesis and stylized references. all the data has been publicly released and the detailed number is in ## style activation prompt prior work usually uses the fixed tag to control the generation with expected attributes @xcite . however, tag-based methods rely on a large amount of labeled parallel data, requiring relabeling and retraining of models when new styles need to be generated. we go back to a standard nmt model. during the generation process, the nmt model tries to maximize the probability of the generation sentence. when predicting the i-th token, the model searches from the vocabulary to maximize: specifically, we retrieve an instance as a prompt from the stylized corpus and use it to instruct the nmt model to generate the sentence with the same attributes. to adapt to the prompt training, we extract every sentence in the basic parallel data and retrieve one similar sentence as the prompt. the whole framework is in figure 2 . we introduce the details of our proposed method as follows. ## experiments in this section, we will introduce the details of our experiments. ## analysis 6.1 retrieval strategy matters there are many retrieval methods to select a similar sentence from stylized monolingual sentences. we conduct a detailed comparison in english-to-chinese inference with the following strategies: the results are shown in ## conclusions in cross-lingual generation fields, most studies focus on the translation quality but ignore the style issue, which happens to be important in the realworld communication. however, the previous studies face two major challenges including the bench-mark as well as the approach. for those purposes, we re-visit this task and propose a standard stylized nmt benchmark msmt with four well-defined tasks to push the boundary of this field. we also propose a new translation style controlling method with activation prompt. with stylized prompts that are retrieved from the stylized monolingual corpus, we successfully guide the translation generation style without iterative fine-tuning. through automatic evaluation and human evaluation, our method achieves a remarkable improvement over baselines and other methods. a series of analysis also show the advantages of our method.