Improving Formality-Sensitive Machine Translation Using Data-Centric Approaches and Prompt Engineering (2023)

Abstract: In this paper, we present the KU x Upstage team’s submission for the Special Task on Formality Control on Spoken Language Translation, which involves translating English into four languages with diverse grammatical formality markers. Our methodology comprises two primary components: 1) a language-specific data-driven approach, and 2) the generation of synthetic data through the employment of large-scale language models and empirically-grounded prompt engineering. By adapting methodologies and models to accommodate the unique linguistic properties of each language, we observe a notable enhancement in performance relative to the baseline, substantiating the heightened efficacy of data-driven approaches. Moreover, our devised prompt engineering strategy yields superior synthetic translation instances.

## introduction neural machine translation (nmt) models have achieved remarkable progress in recent years, as evidenced by their high bleu scores @xcite . nonetheless, these models generally rely on generic parallel corpora and assume a single target translation for a given source sentence, often overlooking the significance of style and pragmatic aspects in translation, such as formality or politeness @xcite . to address this issue, formality-sensitive machine translation (fsmt) has emerged as a research area, aiming to control grammatical formality in translated text across languages @xcite . the special task on formality control on spoken language translation introduces a new benchmark with high-quality training datasets for diverse languages, encompassing both supervised and zero-shot language pairs. despite these new datasets @xcite , controlling formality in mt remains a challenging problem due to the source: it did, many people liked his show so yeah, do you like chris pratt? korean formal: 그랬어요, 많은 사람들이 그의 쇼를 좋아했죠. 그래서 당신 크리스 프랫 좋아해요? korean informal: 그랬어, 많은 사람들이 그의 쇼를 좋아했지. 그래서 너 크리스 프랫 좋아해? in the 2023 shared task, an english source segment is paired with two references that are minimally contrastive in grammatical formality, representing both formal and informal levels as shown in in this paper, we present our approach to fsmt, focusing on the supervised setting for the english-korean (en-ko) and english-vietnamese (en-vi) language pairs and evaluating our methods on the zero-shot english-portuguese (en-pt) and english-russian (en-ru) pairs. our method consists of two main strategies: 1) a language-specific data-driven approach, and 2) synthetic data gener-ation using large-scale language models and empirical prompt engineering. we apply techniques and models tailored to the linguistic features of each language. for korean, we utilize a morphemecentric subword tokenization method, while for vietnamese, we employ a pre-trained envit5 model with high-quality vietnamese parallel corpora. additionally, we generate synthetic translation datasets for portuguese and russian using prompt engineering and refine these datasets using formality classifiers for fine-tuning our models. furthermore, we founded significant performance improvements in en-ko and en-vi and conducted an ablation study to utilize high-quality synthetic examples. ## background in this work, we focus on data-centric approaches to improve neural machine translation (nmt) performance. several studies have investigated different strategies to address the challenges of lowresource languages and enhance translation quality. @xcite proposed subword regularization to improve nmt models using multiple subword candidates, effectively increasing data diversity and robustness. @xcite introduced a universal nmt model for extremely low-resource languages, leveraging multilingual knowledge from high-resource languages to assist in translation. @xcite explored transfer learning for low-resource nmt, utilizing pre-trained models on related high-resource languages to improve the performance on the target low-resource language. additionally, @xcite proposed a method of improving nmt models by generating synthetic parallel data through back-translation, which has proven successful in various translation tasks. these studies highlight the diverse data-centric approaches in nmt, aiming to improve translation quality and overcome the limitations of low-resource languages. ## conclusion in this paper, we presented the ku x upstage team's submission for four languages, employing two main strategies: 1) a language-specific data-driven approach, and 2) synthetic data generation using large-scale language models and empirical prompt engineering. while our data-driven approach excelled, particularly in en-ko and en-vi, the quality of synthetic data generation was called into question. in light of this feedback, we propose to enhance the quality of synthetic data by integrating quality estimation (qe) techniques as an additional filter in the generation process. this step aims to further refine our synthetic examples, potentially improving the overall system performance. we also plan to explore the use of translation models with larger parameters and conduct a thorough analysis through more shot examples and linguistically-grounded data augmentation techniques. finally, we aim to extend our understanding of factors influencing fsmt performance, such as the impact of formal register versus grammatical formality in training data and a detailed examination of zero-shot transfer. figure 3 : prompt template for supervised setting based on @xcite . we utilize n randomly selected shots from the english training set of other language pairs in the iwslt 23 formality track as input for our model, with few-shot examples derived from the target language's training set.