How Human is Machine Translationese? Comparing Human and Machine Translations of Text and Speech (2020)

Abstract: Translationese is a phenomenon present in human translations, simultaneous interpreting, and even machine translations. Some translationese features tend to appear in simultaneous interpreting with higher frequency than in human text translation, but the reasons for this are unclear. This study analyzes translationese patterns in translation, interpreting, and machine translation outputs in order to explore possible reasons. In our analysis we – (i) detail two non-invasive ways of detecting translationese and (ii) compare translationese across human and machine translations from text and speech. We find that machine translation shows traces of translationese, but does not reproduce the patterns found in human translation, offering support to the hypothesis that such patterns are due to the model (human vs machine) rather than to the data (written vs spoken).

## introduction in recent years, a growing body of work has pointed to the presence, across different genres and domains, of a set of relevant linguistic patterns with respect to syntax, semantics and discourse of human translations. such patterns make translations more similar to each other than to texts in the same genre and style originally authored in the target language. these patterns are together called "translationese" @xcite . linguists classify translationese in two main categories: (i) source's interference, or shining-though as put forward by @xcite . for example, a translation replicating a syntactic pattern which is typical of the source language, and rare in the target language, displays a typical form of shiningthrough; (ii) aherence or over-adherence to the target language's standards, that is normalisation. for example, translating a sentence displaying marked order in the source with a sentence displaying standard order in the target it a typical example of overnormalization. nonetheless, translationese's main causes remain unclear @xcite . translationese displays different patterns depending on the translation's mode and register: a typical example is simultaneous interpreting, which shows translationese patterns distinct from those observed in written translations @xcite . we can interpret differences as either (i) an effect of the limitations of the human language apparatus that constrain translations, (ii) an inevitable effect of the structural and semantic differences between languages; or (iii) a combination of the two. to test these hypotheses, it is common to compare human translations (ht) produced under different circumstances, e.g. written translation versus simultaneous interpreting, following the assumption that, if translationese is a product of human cognitive limitations, translations produced under higher cognitive constrains should present more evident translationese symptoms. machine translation (mt) does not have any human cognitive limitation, but current state-of-the-art systems learn to translate using human data, which is affected by translationese. on the other hand, unlike human translators, most standard modern mt systems still work at the level of individual sentences (rather than document or dialog) unaware of the surrounding co-text and context. taking these observations into account, this paper explores the following research questions: q1 . does machine translation replicate the translationese differences observed between text and speech in human translation? q2 . do different machine translation architectures learn differently enough to display distinctive translationese patterns? we study translationese in speech and writ-ten text by comparing human and machine translations to original documents. specifically, we present a comparison between three mt architectures and the human translation of spoken and written language in german and english, exploring the question of whether mt replicates the differences observed in human data between text-based and interpreting-based translationese. while some aspects of translationese tend to appear in simultaneous translation of speech with higher frequency than in translation of text, it is unclear whether such patterns are data-or modeldependent. since machine translation does not have the cognitive limitations humans have (in terms of memory, online processing, etc.), mt systems should fail to replicate the differences between text and speech translationese observed in human translations, if such differences are due to the limits of human cognitive capacities. assuming that mt engines are trained only on translated texts and not on simultaneous interpreting, so that they cannot simply learn to mimic interpreting's translationese signal, the differences between text and speech translationese observed in human translation vs. interpreting are not expected to the same extent in mt if they are an effect of human cognitive limitations. if, on the other hand, translationese patterns in text and speech are due to characteristics inherent in the two modes of expression, mt systems' translationese patterns should mimic human translationese patterns. the paper is organized as follows. section 2 presents related work. section 3 introduces the translationese measures and section 4 the data and translation systems used in our study. section 5 presents our results, comparing human with machine translations and comparing mt architectures among themselves. section 6 concludes the paper. ## related work translationese seems to affect the semantic as well as the structural level of text, but much of its effects can be seen in syntax and grammar @xcite . an interesting aspect of translationese is that, while it is somewhat difficult to detect for the human eye @xcite , it can be machine learned with high accuracy @xcite . many ways to automatically detect translationese have been devised, both with respect to textual translations and simultaneous interpreting @xcite @xcite . simultaneous interpreting has shown specific forms of translationese distinct from those of textual translation @xcite , with a tendency of going in the direction of simplification and explicitation @xcite . due to the particularly harsh constraints imposed on human interpreters, such particularities can be useful to better understand the nature and causes of translationese in general @xcite . the features of machine translated text depend on the nature of both training and test data; and possibly also on the approach to machine translation, i.e. statistical, neural or rule-based. the best translation quality is achieved when the training parallel corpus is in the same direction as the test translation, i.e. original-to-translationese when one wants to translate originals @xcite . in this case, more human translationese features are expected, as systems tend to learn to reproduce human features. the effects of translationese in machine translation test sets is also studied in @xcite . in fact, texts displaying translationese features seems to be much easier to translate than originals, and recent studies advise to use only translations from original texts in order to (automatically) evaluate translation quality @xcite . @xcite show a slight preference of human evaluators to outputs closer to originals; in this case the translation is done from translationese input, because, as noted by @xcite , the best of the two worlds is not possible: one cannot create original-to-original corpora to train bias-free systems. aranberri (2020) analyzed translationese characteristics on translations obtained by five state-of-the-art spanish-to-basque translation systems, neural and rule-based. the author quantifies translationese by measuring lexical variety, lexical density, length ratio and perplexity with part of speech (pos) language models and finds no clear correlation with automatic translation quality across different test sets. the results are not conclusive but translation quality seems not to correlate with translationese. similar results are obtained by @xcite when using 45 morpho-syntactic features to analyze english-to-russian translations. @xcite noted that statistical systems reproduce human lexical diversity better than neural systems, for english-spanish and english-french even if transformer models, i.e. neural, are those with the highest bleu score. however, their transformer model did not use subword segmentation putting a limit on the plausible lexical diversity it can achieve. in our study, we focus on english (en) and german (de) texts and compare the presence of translationese in human translations and interpreting, and in machine translations obtained by in-house mt engines. differently to aranberri (2020), we develop the mt engines to have control on the training data, since the aim of this work is not to study how translationese features correlate with translation quality, but the presence of translationese features themselves. to measure them, we use two metrics: part-of-speech (pos) perplexity and dependency parsing distances. we use them as complementary measures, perplexity to model pos sequences of linguistics objects in sentences (linear dimension), and dependency distance to model their syntactic depth (hierarchical dimension). ## experimental results below we present the translationese characteristics in the different modalities found in our study. ## conclusions crossing pos perplexity scores and syntactic dependency lengths, we draw some conclusions about the difference between human and machine structural translationese in text and speech. we summarize our findings as preliminary answers to the questions posed in the introduction.