Translating away Translationese without Parallel Data (2023)

Abstract: Translated texts exhibit systematic linguistic differences compared to original texts in the same language, and these differences are referred to as translationese. Translationese has effects on various cross-lingual natural language processing tasks, potentially leading to biased results. In this paper, we explore a novel approach to reduce translationese in translated texts: translation-based style transfer. As there are no parallel human-translated and original data in the same language, we use a self-supervised approach that can learn from comparable (rather than parallel) mono-lingual original and translated data. However, even this self-supervised approach requires some parallel data for validation. We show how we can eliminate the need for parallel validation data by combining the self-supervised loss with an unsupervised loss. This unsupervised loss leverages the original language model loss over the style-transferred output and a semantic similarity loss between the input and style-transferred output. We evaluate our approach in terms of original vs. translationese binary classification in addition to measuring content preservation and target-style fluency. The results show that our approach is able to reduce translationese classifier accuracy to a level of a random classifier after style transfer while adequately preserving the content and fluency in the target original style.

## introduction translated texts often exhibit distinct linguistic features compared to original texts in the same language, resulting in what is known as translationese. translationese has a tangible impact on various cross-lingual and multilingual natural language processing (nlp) tasks, potentially leading to biased results. for instance in machine translation (mt), during training, when the translation direction of parallel training data matches the direction of the translation task, (i.e. when the source is original and the target is translated), mt systems perform better @xcite . similarly, @xcite , @xcite and @xcite show that translating already translated texts results in increased bleu scores. more recently, @xcite observed that cross-lingual models when evaluated on translated test sets show false improvements simply due to induced translation artifacts. when investigating the effects of translationese in cross-lingual summarization, @xcite found that models trained on translated training data suffer in real-world scenarios. these examples show the importance of investigating and mitigating translationese. despite this, removing translationese signals in already generated output of translations is an underexplored research topic. dutta @xcite remove translationese implicitly encoded in vector embeddings, and demonstrate the impact of eliminating translationese signals on natural language inference performance. @xcite leverage abstract meaning representation (amr) as an intermediate representation to abstract away from surface-level features of the text, thereby reducing translationese. neither of these works explicitly analyzes surface forms of the resulting "debiased text" and its resemblance to original texts. in this work, we aim to reduce the presence of translationese in human-translated texts and make them closely resemble original texts, notably, without using any parallel data as parallel human original -translated data in the same language do not exist. to this end, we explore a self-supervised neural machine translation (nmt) system @xcite and its application to style transfer (st) @xcite . in both works, validation is performed on a parallel dataset, either bilingual (mt data) or monolingual (st data). however, parallel human original-translationese data in the same language are unavailable. to overcome this challenge, we define an unsupervised validation criterion by combining a language model loss and a semantic similarity loss, inspired by @xcite . our baseline is the self-supervised approach (ssnmt) from @xcite . however, we go a step further and propose a novel joint training objective that combines both selfsupervised and unsupervised criteria, eliminating the need for parallel data during both training and validation. the contributions of this work are as follows: ## translationese mitigation via style transfer our goal is to eliminate translationese signals from translated texts by transforming them into an original-like version. we define two style attributes, og and tr, representing original style and translated style, respectively. given a text sample x belonging to tr, our aim is to convert this x tr to x og , where x og belongs to style og but retains the same semantic content as x tr . we denote the corpus with original sentences og and the corpus with translated sentences t r. we illustrate the process in figure 1 . ## evaluation we perform evaluation on the outputs ( ôg) of the style transfer models, given the humantranslationese (htr) (without an original reference) half of the test data as input. ## conclusion in this work, we reduce the presence of translationese in human-translated texts and make them more closely resemble originally authored texts. we approach the challenge as a monolingual translation-based style transfer task. due to the absence of parallel translated and original data in the same language, we employ a self-supervised style transfer approach that leverages comparable monolingual data from both original and translated sources. however, even this self-supervised approach necessitates the use of parallel data for the validation set to facilitate the learning of style transfer. therefore, we develop a novel joint training objective that combines both self-supervised and unsupervised criteria, eliminating the need for parallel data during both training and validation. our unsupervised criterion is defined by combining two components: the original language model loss over the style-transferred output and the semantic similarity loss between the input and style-transferred output. with this, we show that the generated outputs not only resemble the style of the original texts but also maintain semantic content. we evaluate our approach on the binary classification task between original and translations. our findings show that training with joint loss significantly reduces the original and translation classification accuracy to a level comparable to that of a random classifier, indicating the efficacy of our approach in mitigating translation-induced artifacts from translated data. as future work, we intend to explore more sophisticated approaches to improve content preservation and evaluation. it would be also interesting to apply a version of our style transfer approach to the output of machine translation to alleviate translationese in cross-lingual scenarios. allel data, for our approach to work it is important to ensure that the data in both modalities (original and translationese) are comparable. manually evaluating the quality of style transfer and the reduction of translationese is inherently subjective. while we conduct a preliminary analysis to evaluate the outputs, more nuanced linguistic expertise is required for in-depth analysis. though we propose evaluation metrics, there is no universally accepted gold standard for measuring the effectiveness of translationese reduction. these factors may introduce biases and challenges in comparing the performance of different approaches. at the individual text level, even human experts struggle to distinguish between translationese and originals. detecting translationese reliably involves analyzing large quantities of data or training classifiers on original and translated text.while amponsah- @xcite show evidence that highperformance translationese classifiers may in some cases rely on spurious data correlations like topic information, recent work by @xcite indicates that the effect arising from spurious topic information accounts only for a small part of the strong classification results. a decrease in classifier accuracy strongly suggests reduced translationese signals. that said, further research on addressing spurious correlations in translationese classification is an important research not explored in our paper. finally, it is worth noting that the proposed systems may attempt to improve translations already of high quality that should not really be touched. our results include potential quality degradation due to system overcorrections. future work aims to address this phenomenon using an oracle-based style transfer approach. attain the best checkpoint earlier than their counterparts and hence, we report the number of parallel pairs from this epoch. as seen in the quantitative analysis for en-all, the large number of accepted pairs due to the application of a threshold in the self-supervised approach, helps improve style transfer. however, for the joint training, due to the weaker unsupervised signals, the threshold does not have the same impact. interestingly, for de-all, the additional use of a threshold in both the self-supervised and joint training approaches does not increase the number of parallel pairs. furthermore, across both the data setups, joint training with no threshold achieves a greater number of parallel pairs than its counterpart with threshold. this explains why the quantitative metrics perform slightly better for this model variant.