Machine vs. Human: Exploring Syntax and Lexicon in G erman Translations, with a Spotlight on Anglicisms (2023)

Abstract: Machine Translation (MT) has become an integral part of daily life for millions of people, with its output being so fluent that users often cannot distinguish it from human translation. However, these fluid texts often harbor algorithmic traces, from limited lexical choices to societal misrepresentations. This raises concerns about the possible effects of MT on natural language and human communication and calls for regular evaluations of machine-generated translations for different languages. Our paper explores the output of three widely used engines (Google, DeepL, Microsoft Azure) and one smaller commercial system. We translate the English and French source texts of seven diverse parallel corpora into German and compare MT-produced texts to human references in terms of lexical, syntactic, and morphological features. Additionally, we investigate how MT leverages lexical borrowings and analyse the distribution of anglicisms across the German translations.

## introduction advanced text generation tools such as chatgpt 1 and machine translation (mt) are used by millions of people every day. with the scope of human exposure to machine-generated texts evergrowing, these tools possess the potential to have an impact on natural language. the scientific community is yet to establish a research paradigm suitable for the assessment of this impact. in the meantime, we investigate generated texts and compare them to human-produced texts. in the present paper, we focus on machine translation for the german language. human exposure to mt output is expected to increase, and the global mt market is steadily growing foot_0 . machine-translated texts are used in almost all spheres of life, from schools @xcite , to academic publishing @xcite , to governments @xcite @xcite , and even hospitals and courts @xcite @xcite . new mt engines continue to enter the market and language coverage has reached over 200 languages @xcite and tens of thousands language pairs across all mt systems 3 . several researchers already started to investigate the sociolinguistic impact of machine translation. for instance, mt use has been shown to have a direct and long-lasting effect on the syntactic production of language learners @xcite . while producing highly fluent translations, the mt output can suffer from simplification and even impoverishment @xcite . moreover, mt models are known to overgeneralize and amplify societal biases @xcite @xcite @xcite . when it comes to the analysis of commercial mt systems, however, most research focuses on the english output of google translate foot_2 with rare mentions of other translation engines @xcite @xcite @xcite . in our paper, we explore the output of three widely used engines (google, deepl, microsoft azure) and one smaller commercial system. we work with translations from english and french to german, a morphologically and syntactically complex language. we use seven different corpora (section 2) and a battery of evaluation metrics which examine the texts on lexical, syntactic, and morphological levels (section 3). moreover, in section 3.3, we scrutinize the translations from a novel angle, by looking at the distribution of anglicisms in the german texts -the process of lexical borrowing being a crucial feature of language change and evolution @xcite . ## metrics and findings we used several metrics to analyze the available translations in terms of their lexical, syntactic, and morphological features. ## conclusion this paper provides a corpus linguistic analysis of different translations, performed by humans and machines, in seven corpora from different domains. we looked at the texts mostly on a microlevel, measuring their lexical and syntactic properties, such as type-token ratio, morphological richness, and syntactic versatility. additionally, we examined the distribution of translation variants for english lexical items that have entered the german language as borrowings or loan words. previous research emphasized that machineproduced texts suffer from standardization, simplification, and monotonicity. on one hand, our results confirm these findings in terms of syntax (section 3.2). on the other hand, we show that machine translation is becoming less of a culprit when it comes to lexical impoverishment of language. some commercial mt systems are capable of generating german texts with levels of lexical and morphological richness similar to those produced by human translators (section 3.1). of course, these results reflect only one aspect of translation quality, and our automatic scores -as imperfect as they are -suggest that deepl, not mtx, is the most reliable system for german translations (see finally, we note that the standard lexical and syntactic metrics might be getting less informative for the linguistic assessment of mt as the technology continues to improve. alternatively, automatic evaluation of lexical borrowings, such as anglicisms in german, can provide a good opportunity to assess the appropriateness of mt use. the distribution of borrowings is directly related to the quality and purpose of translation. our results indicate that certain machine translation systems tend to produce fewer anglicisms compared to other systems (section 3.3). in general, human translators adjust the use of anglicisms according to the domain, while the mt systems produce mostly consistent, system-specific distributions. as machine translation improves and becomes more widespread, it will likely play a role in the (de-)anglicization of german.