On the differences between human translations (2020)

Abstract: Many studies have confirmed that translated texts exhibit different features than texts originally written in the given language. This work explores texts translated by different translators taking into account expertise and native language. A set of computational analyses was conducted on three language pairs, English-Croatian, German-French and English-Finnish, and the results show that each of the factors has certain influence on the features of the translated texts, especially on sentence length and lexical richness. The results also indicate that for translations used for machine translation evaluation, it is important to specify these factors, especially if comparing machine translation quality with human translation quality is involved.

## introduction many studies have demonstrated that translated texts (human translations, hts) have different lexical, syntactic and other textual features than texts originally written in the given language (originals). these special traits of hts are result of a compromise between two often antagonised aspects of the translation process: fidelity to the source text and naturalness of the generated target language text. although all studies confirm the existence of unique ht features, two categories of these features are distinguished in the literature. one category, "translation universals", represents a general set of features shared by all translations, independent of the characteristics of involved languages c 2020 the authors. this article is licensed under a creative commons 3.0 licence, no derivative works, attribution, cc-by-nd. @xcite . another category, "interference", reflects the impact of the source language, the "trace" which the source language leaves in the translation @xcite . some studies investigate and demonstrate the existence of both categories, sometimes called "source universals" and "target universals" @xcite . our research aims to find out whether differences between translators have any influence on the text features. we investigate impact of the translator's expertise and native language. we present results of a computational analysis of a set of hts originating from the news domain and involving three distinct language pairs, english-croatian, german-french and english-finnish. the analysis is guided by the following research questions: rq1 are there differences between hts related to translator's expertise? rq2 are there differences between hts related to translator's native language and translation direction? (from or into translator's native language) the main contribution of this work is empirical, showing evidence of differences between text features of hts produced by different translators. we expect our findings to motivate and drive future research in this direction in order to better understand these differences by identifying and analysing underlying linguistic phenomena. moreover, differences between hts may have practical impact on evaluation of machine translation (mt) systems. several recent studies @xcite @xcite have shown that the translation direction has impact on the results of evaluation of mt outputs, so that it is important to specify whether originals or hts were used as source texts for mt systems. taking into account these studies and the findings reported in this work, potential effects of translators' backgrounds on mt should be investigated too. ## related work analysis of translated texts a lot of work has been done exploring differences between hts and originals. some studies @xcite have emphasised the existence of "translation universals", general features of translated texts, "simplification" and "explicitation" being the most wellknown. other studies @xcite have pointed out the influence of the source language, "interference", whereas some @xcite concentrate on both categories, called "s-universals" and "t-universals". since many text features can be measured quantitatively, a number of publications demonstrated that hts can be automatically distinguished from originals @xcite @xcite @xcite . the features used for the classifiers are partly motivated by the theoretical categories mentioned above, however many features are not directly related to a particular category, and many can belong to more than one category. the most common features are lexical variety (percentage of distinct words in a text), lexical density (sometimes called information density, percentage of content words in a text), sentence length, word length, as well as frequencies of certain pos categories, function words and collocations. @xcite include analysis of nonnative texts, namely texts originally written in the given language but by non-native speakers. they found that these texts generally exhibit different features than native originals and hts, thus representing yet another text category. on the other hand, their features are closer to those of hts than to native originals, indicating the influence ("interference") of the native language. in addition to analysis of hts, more and more publications report analysis of machine translated texts. @xcite compares mt outputs with hts by means of automatically calculated text features as well as by manual analysis of di-vergences (shifts) from the source text. the main finding is that mt output is much more similar to the source text than ht. another study of machine translated texts @xcite reports significantly lower lexical richness in mt outputs in comparison to originals and hts. post-editing (pe) of mt outputs has lead to yet another type of translated text which has been analysed extensively in the recent years @xcite @xcite @xcite . these studies demonstrated that pes represent an additional text category with the features lying between those of hts and of mt outputs. relations between machine and human translation as machine translation (mt) technology improves, more and more work has been done on investigating relations between different aspects of mt and ht direction. first publications on this topic @xcite demonstrated that the direction of ht plays an important role for building a statistical mt system, and recommend training on parallel corpora which were translated in the same direction as the mt system (i.e. using originals as source and hts as target). recently, several publications @xcite @xcite demonstrated that the translation direction plays an important role both for human as well as for automatic evaluation of mt systems. before these findings were published, this aspect has not been taken into account at all in the mt community. 1 afterwards, as a consequence, using only originals as source test texts and hts as reference test texts has become a common practice in the wmt shared tasks foot_1 from 2019. the main reason is to avoid all possible side effects, since @xcite have shown that the use of hts as source texts facilitates the mt process mainly because of the decreased lexical variety. on the other hand, @xcite recommend using both, albeit separated, original as well as ht source texts precisely in order to be able to take into account and better understand all effects. apart from the impact of translation direction, the impact of divergences from a source text in ht used as mt data has been investigated, too. the potential influence of different translation strategies and resulting divergences (shifts) on mt evaluation was discussed in @xcite , whereas @xcite explored automatic identification of such divergences and their effects on mt training. texts translated by different translators despite of a large body of work dealing with analysis of different translated texts in different contexts, there is, however, not much work about texts translated by different translators. @xcite explored effects of translator's expertise to some extent, and reported that texts translated by students could be automatically distinguished from originals with higher accuracy than texts translated by professional translators. this indicates that the features of professional hts are more similar to the features of originals. to the best of our knowledge, our work is the first attempt to systematically compare texts translated by different translators. ## data sets for our experiments, we used three available parallel data sets involving three different language pairs and five translation directions: @xmath0, german↔french (def r) and english↔finnish (enf i). all data sets belong to the news domain and originate from the publicly available wmt shared tasks. ideally, each data set should have been designed specifically for one particular rq, and created under the same conditions: each of the translators should have translated the same, sufficiently large source text. in addition, all source language texts should be originally written in that language, not being translated from some other language. in addition, several domains/genres should ideally be covered, whereas all our data sets come from the news domain. nevertheless, an ideal data set is, to the best of our knowledge, currently not available for any of our research questions. therefore, we carried out our first experiments on the described available texts which, despite of their flaws, represent a good starting point for this research direction. the data sets are made publicly available for further research. 3 no personal information (such as name, gender, age, working place) about the translators are shared. the details and statistics of each of the texts used are presented together with the results in the corresponding sections. ## text features the set of text features used in our experiments is inspired by the features frequently used in the literature @xcite @xcite . although they are also motivated by two theoretical categories, simplification @xcite and interference @xcite , they do not represent any of these categories exclusively. the choice of features is based on a hypothesis that the selected features might vary depending on the factors addressed in our work, namely translator's experience, native language and translation strategies. for all features, punctuation marks were separated and counted as words. pos tags for all languages are generated by treetagger. 4 the features property enhr def r enf i rq1: expertise + + -rq2: translation direction and native language -+ + same source language text for each @xmath1 sentence length: number of words in each sentence of the text. some translators might tend to generate longer sentences in the target text than others. some translators might keep the number of words in the translated sentences closer to the number of source text words than others. mean word length: the total number of characters in the text divided by total number of words. some translators might prefer longer (potentially more complex) words than others. ## experimental set-up for each feature, we calculate relative difference between the feature value of the original source text f (source) and the feature value of its translation f (ht). the main benefits of reporting relative differences are: for each text and each feature, relative difference is calculated as average value over chunks of 100 sentences foot_4 (approximately 2000 words), similarly to some previous work @xcite . the purpose of averaging over small chunks is manifold: to make sure that the length of a text does not interfere with the feature values, to avoid issues related to the small size of some texts, and to further minimise the potential effects of using comparable instead of parallel translations. for each of the research questions, the obtained values are reported and discussed in the following section. it is worth noting that the numbers differ between the data sets due to distinct properties of the language pairs. for example, relative difference between finnish and english lexical and pos varieties are much larger than those between german and french. we did not perform any text classification in this experiment, because the sizes of the currently available texts are not sufficient for training a classifier. ## conclusions this work presents results of a set of computational analyses on three data sets containing three language pairs and five translation directions with the aim of finding out whether different human translations exhibit different traits. despite certain limitations, our findings represent a good base for analysing different human translations. the main contribution of this work is empirical, showing that each of the investigated factors has certain influence on the features of translated texts. sentence length and lexical variety are affected by