Measuring Translationese across Levels of Expertise: Are Professionals more Surprising than Students? (2021)

Abstract: The present paper deals with a computational analysis of translationese in professional and student English-to-German translations belonging to different registers. Building upon an information-theoretical approach, we test translation conformity to source and target language in terms of a neural language model’s perplexity over Part of Speech (PoS) sequences. Our primary focus is on register diversification vs. convergence, reflected in the use of constructions eliciting a higher vs. lower perplexity score. Our results show that, against our expectations, professional translations elicit higher perplexity scores from a target language model than students’ translations. An analysis of the distribution of PoS patterns across registers shows that this apparent paradox is the effect of higher stylistic diversification and register sensitivity in professional translations. Our results contribute to the understanding of human translationese and shed light on the variation in texts generated by different translators, which is valuable for translation studies, multilingual language processing, and machine translation.

## introduction translationese is a set of linguistic patterns that tell translations apart from texts originally written in the same language and that make translations stylistically more similar to each other than original texts tend to be. while translationese was extensively discussed in the area of corpus-based translation studies and machine translation (mt) @xcite , there are relatively few computational studies that focus on the varying amount of translationese characterizing different kinds of written translation (see section 2.2 below). this study focuses on the relation between translators' level of expertise and translationese throughout different registers. if we can connect translationese at least partly to the translator's experience, we can expect to find different degrees of translationese between student and professional translations. as translationese is probabilistic in nature @xcite , we use a framework that enables a probabilistic design of language use in the form of a language model. we model language conventions in terms of grammatical structures represented by pos sequences through long short-term memory (lstm), a recurrent neural network architecture, using monolingual corpora of non-translations in both source and target language as a training set. we then test how students' and professionals' translations conform to linguistic conventions using our models' perplexity scores. through this approach, we aim at testing two related hypotheses: hypothesis 1 overall, we can expect professional translators to be more efficient at reproducing the patterns of their target language. if this is the case, we would expect professional translations to elicit lower perplexity scores from the target language model than from the model of the source language. hypothesis 2 on the other hand, it is possible that students converge more on standard patterns: due to their lack of expertise, they might have lower register sensitivity, and thus they could be less bold and more repetitive in their use of grammatical constructions. a higher value of perplexity for a register means a less usual (hence, more perplexing) order of pos with respect to a reference corpus, and hence a more distinct register. so a higher level of convergence would result in more homogeneous surprisal values across registers. we then compare the results of our perplexity measures with the distribution of different pos patterns across registers to qualitatively analyze translation divergence in the data. we organized the remainder of the paper as follows: section 2 introduces the main concepts we are developing our research on and provides an overview of the related work, section 3 includes details on the data and methods used in the analyses. we show the results of the analyses in section 4 and 5, interpret them, and conclude in section 6. ## analysis of pattern diversity in (1-a), the student translator uses a complex nominal structure and adds some information not available in the source. the translation by a professional in (1-b) contains the same information as in the source (1-c) and a more lexically dense structure (bandgraffiti vs. graffitimalereien der gangs). a. der schweiß [lief an unserem körper herunter, sodass] unsere t-shirts an unsere rücken klebten. b. der schweiß [lief uns am körper runter, daß] uns die hemden am rücken klebten. c. the sweat [came down our bodies and] plastered our shirts to our backs. both translation varieties in example (2) convey the same information from the source sentence. however, the translation by a professional in (2-b) sounds more natural in german, whereas the student translation in (2-a) is closer to the source sentence. the direct object in the english in (2-c) cannot be directly transferred into german because of the restriction on semantic diversity of subjects and objects in german. the professional translator changes the direct object into a dative+prepositional object, whereas the student uses just a prepositional object. ## conclusion and outlook in this study, we analyzed translationese in professional and student translations using a perplexitybased approach. we modelled the source and target grammatical patterns with an lstm architecture and tested the conformity to the source and target language conventions of the two translation varieties through pos perplexity. despite the relative scarcity of our data, the small vocabulary of universal pos allowed our lstms to learn the short and long-distance patterns well enough to display significantly higher perplexities when confronted with translations instead of original texts. through this method, we found that, surprisingly, professional translators elicit higher perplexity scores from the target language model than students, which is against our first hypothesis. nonetheless, in the analysis of convergence, we tested the extent to which professional and student translations of various registers conform to the target language model. we found more convergence in student than in professional translations, confirming our second hypothesis. we then tried to understand such results by analyzing pos n-gram patterns in both translation varieties and conducting a qualitative analysis of translation divergence in the data. overall, we found that such higher perplexities are an artifact of higher register variation in professional translations. we are not observing interference, but rather professionals' essential ability to be more daring with their language use. student translators converge more, displaying a lower register sensitivity and a tendency to overuse the most general structures of the target language, while professional translators display more diversity and creativity in their structures, behaving in this way more similar to native writers. the qualitative analysis of the examples suggest that the source of this diversity may originate from the cross-lingual differences between the source and the target languages: faced with a construction that has no direct or obvious equivalent in the target language, students might tend to choose less brilliant, more standard constructions across registers, whereas professionals might attempt to recreate the original domain's diversity. at the same time, we realize that our analyses may have some limitations. for instance, due to the absence of metadata in professional translations, we fail to control for individual variation in the data. for students, we know that the texts of various registers were sometimes translated by the same translators. the results of our analyses provide an empirical contribution to the understanding of human translation. they show evidence of variation between texts generated by different translators in terms of language patterns and shed more light on the phenomenon of translationese. studying variation in human translations of the same source texts across various registers is valuable for translation studies and multilingual language processing, especially for mt. as shown in @xcite , human translation variation plays a great role in mt evaluation. in the future, we plan to deepen our understanding of how students over-normalize by aligning source and target texts, allowing for qualitative analyses of their translating behavior. we also want to explore whether other factors beyond the level of expertise influence translation convergence. moreover, we would like to connect these results with the growing field of automatic translation quality estimation. finally, although it is hard to find appropriate datasets containing comparable texts in terms of registers and different degrees of expertise, it would be interesting to expand this work on the opposite translation direction (german-english) and other language pairs to see if such tendencies are universally valid. multilinguality would introduce more variance, and thus more factors to consider to avoid the risk of overclaiming and misunderstanding the complex phenomenon of translationese.