Do Humans Translate like Machines? Students’ Conceptualisations of Human and Machine Translation (2023)

Abstract: This paper explores how students conceptualise the processes involved in human and machine translation, and how they describe the similarities and differences between them. The paper presents the results of a survey involving university students (B.A. and M.A.) taking a course on translation who filled out an online questionnaire distributed in Finnish, Dutch and English. Our study finds that students often describe both human translation and machine translation in similar terms, suggesting they do not sufficiently distinguish between them and do not fully understand how machine translation works. The current study suggests that training in Machine Translation Literacy may need to focus more on the conceptualisations involved and how conceptual and vernacular misconceptions may affect how translators understand human and machine translation.

## introduction recent years have seen increasing prominence of mt both inside the translation industry and in everyday settings. although predictions of "synchronous, automated translation systems" completely replacing translators (e.g. @xcite have not come to pass, mt has had an undeniable impact, not merely changing the practical realities of translation but in fact challenging the very concept of translation (e.g. @xcite . the question "is machine translation translation?" 1 https://www.hf.uio.no/ilos/english/research/news-andevents/events/conferences/2022/est22/program/est22-was, for example, the topic of a panel at the 2022 est congress. 1 analysing the ways ht and mt are described can be a useful way to investigate how translation is conceived by people and potentially provide insights into the nature of translation (see @xcite . furthermore, the way translation is discussed and described affects how it is perceived. for this reason, it is also important to examine the socially constructed narratives (see @xcite of humans and machines as translators. whether translation is conceptualised as a straightforward task consisting of mechanically replacing linguistic components or a creative task requiring cultural competence and social perception affects discussions of the automatability of translation (cf. @xcite . common narratives in the popular press about the human-like or even "super human" performance of mt systems may give rise to unrealistic expectations as well as misconceptions of translation both by humans and machines (e.g. @xcite . one of the goals of mt literacy (see @xcite , for example, is to challenge such misconceptions. to explore these issues, this paper examines short reflective texts collected from language and translation students in finland and the netherlands. we analyse how the students describe the process of translating and what these descriptions reveal about their conceptions of ht on the one hand, and mt on the other. we examine whether the students conceptualise ht and mt as the same or a different process, what differences and similarities they perceive, and what the reflections reveal about their conceptualisation of translation as a whole. furthermore, we analyse potential misconceptions of translation (human or machine) that may need addressing as part of their training in translation and the use of translation technology. ## methodology as was mentioned in section 1, we wanted to know how students conceptualise the processes involved in mt and the similarities and differences between ht and mt after having been introduced to the history and basics of mt as part of a translation module during their bachelor's or master's programme. the following subsections describe the design, methods and participants of the study. ## results ## discussion as l23, translated from dutch: i think that in principle people translate the same way as machines, because both make connections between the words of the source text and the associated translations of the target text. both have access to a vocabulary from which the right words can be chosen. when we relate the similarity judgments to the characteristics students refer to in order to support their decision, it becomes clear that they understand the differences between ht and mt predominantly through the characteristics that are typical of human translators. only four characteristics are clearly associated with machines by students in this data -uses probabilities, translates directly, always translates the same way, and uses logicand the total counts for these are low. even though the questionnaire was filled out during an introductory tutorial on mt, it is telling that after having been told how different mt systems work, only 9 out of 58 (16%) mention probabilities and only 8 (14%) remark on the fact that mt normally retains source text structures and translates wordby-word. moreover, the idea that mt would be consistent in formulating the translation (coding always translates the same way) is not true for nmt systems. the scores for the characteristics that students clearly associated with humans are much higher and a more accurate reflection of the actual differences between ht and mt. in total, eight characteristics are associated more with humans, of which considers context and the whole text appears to be "the defining characteristic" with 27 mentions (even though 5 students also associated context with machines), followed by considers target audience and situation (19 vs 0), understands meaning (15 vs 0), has world knowledge (11 vs 0) and has emotions, cognition, personality (11 vs 0). a variety of explanations are in fact brought together under these labels. for example, a number of students mention that humans understand humor, sarcasm, irony or implicit meaning, while others mention that humans understand nuances and reflect on social norms and values and take cultural differences into consideration. most students contrast the differences between humans and machines: t03, translated from finnish: when a human translates, they do quite a lot of background work. they consider the context of the translation, think about the target audience for whom the translation is being made and look at the text holistically in terms of the reading experience. this is not something a machine can do. a machine is able to do translation work that requires repetition and to process huge amounts of material, which would be laborious for a human. interestingly, only 5 students mention that humans are (more) creative, a point often made in academic research on machine translation and post-editing, especially in a literary context (see @xcite . this may be due to the students' limited experience in doing translation themselvesmany novice translators translate quite literallyand in doing mt and post-editing on different genres and text types. two of the students, in fact, mention that they are not familiar enough with the processes involved either in mt or in how humans translate to decide on the difference or similarity. for example, t15 starts their answer by saying (our translation from finnish): "to be honest, i'm not familiar enough with the principles of machine translation to give an informed answer as to the extent to which the human translation process resembles that of a machine." in those cases where students indicated that both humans and machines consider context, a difference is sometimes made between basing the decision how to translate a particular word in context on experience/instinct/feeling versus on data: l01, translated from dutch: the main difference lies in how context is understood: in case of a word with different meanings, a human can look at the sentence, and sense from experience which translation is most suitable. a machine does this not on the basis of feeling, but on the basis of data. it is also clear that different students mean different things by "context": some use it to refer to word meaning in the context of the sentence, others to the context of the whole text, and others to the context outside of the text, so situational context: l17, translated from dutch: a machine is only concerned with the text itself. although it can take context into consideration, it does not look at the underlying meaning, the purpose, or the possible audience. as for the misconceptions the students might have, our preliminary analysis suggests mainly cases of conceptual misunderstandings (a construction of an incorrect model of the phenomenon in question, @xcite , including the humanisation of objects (suprapto, 2020: 52), as well as some vernacular misconceptions (present when words are used that have one meaning in everyday life and another in a scientific context, @xcite . for example, four students (t04, l02, l21 and l28) explain that mt first creates a wordby-word translation based on a vocabulary and then applies rules (example from l02, originally written in english): "machine translation goes word for word and then attaches grammar rules and the like while most human translators go sentence per sentence." this is, of course, true for rule-based mt systems, but not for others, and can be considered (at least partially) a conceptual misunderstanding. another example of a conceptual misunderstanding as construction of an incorrect model is the idea, suggested by l12 and l33, that in translating, machines first convert text to numbers or code, after which they turn it back into text. here, the students relate the functioning of mt to the functioning of a computer in general. there is also a tendency to humanise machine behavior in several students' responses where they talk about machine "thinking" (l03 and l07), "making guesses" (t24), "having difficulty recognising" (l5), "paying attention to" something (l05), or learning (quote from l30, originally written in english): "on top of that, machines are only able to apply rules that they have either been taught to use or that they have been able to figure out from the context of translations that they have already been given". an incorrect model is constructed also by t12 who argues (our translation from finnish): "humans and machines, translation memories for example, both explore their prior knowledge and try to find the correct equivalents of words in the target language". while the exploration part is indeed in a way true, the type of pre-defined knowledge the machine and human employ can be considered fundamentally different, and the student confuses mt and translation memories. while this clearly is a conceptual misunderstanding, this might also be interpreted as a vernacular misconception based on the idea of seeking something in a "memory". relating this back to the similarity judgments, it could be argued that for most of the students who opted for "ht and mt are the same" this judgment may be based on a vernacular misconception or lack of accurate terminology. for example, l04 cited above argued that both use a database to translate. in a technical sense, neither humans nor neural mt retrieve previous translations from a database the way cat tools or translation memories do, though the answer can also be taken to suggest that "database" is used in a more metaphorical sense to mean any kind of previously stored information. similarly, l23 mentioned that both have access to a vocabulary and make connections between words, yet it is unclear whether they realize that the way human vocabularies work and the way word meaning is determined in neural mt are fundamentally different processes. similar misconceptions and technical inaccuracies can be identified in the answers from the "both similar and different" students as well. l24 appears to be aware of the lack of accurate terminology and adds quotation marks to "read" and "instinct" in their explanation (originally written in english): "in some ways, the neural mts translate the same as humans: they "read" many different texts (data) and then develop an "instinct": for humans an almost subconscious knowledge of when something (a sentence in a language) is wrong or right, and for machines a developed strategy." this use of quotation marks illustrates that the student understands that machines do not behave like humans. ## concluding remarks and future work in this paper, we have illustrated that teaching students "the basics of mt" is not a straightforward task as far as students' conceptualisations of the translation processes involved is concerned. even though most students seem to have a reasonable understanding of the ways in which mt is different from htespecially in terms of how human translators take context, purpose, audience and effect into consideration and thus have important "added human value"their answers also point to certain conceptual and vernacular misconceptions and a tendency to humanise mt when explaining how it works. one question that remains as far as we are concerned is whether it is in fact necessary for students to develop the technical competence of understanding how mt works (in terms of programming, being able to train and customise systems, and running metrics) in order to develop the technical competence of using mt systems effectively and ethically. translator training programmes appear to focus more on developing post-editing skillswhich we agree is a translation competence rather than a technical competence. the question remains then whether training should include more computational competence depending on the meaning of "basic knowledge of machine translation technologies" (emt board and competence task force, 2022: 9). while a lot of attention has been paid to training translation students how to use different mt systems and do post-editing in different genres, far less attention appears to have been paid to assessing (also formally) how students understand the different processes involved and whether misconceptions affect either their usage or their perception or both. paying attention to misconceptions is important, as they may build barriers for students to learn about mt and direct their reasoning to incorrect notions of what mt is. as future professionals, whether working in language industry or in public service positions, they need to understand the uses and limits of mt in order to be able to "implement and advise on the use of present and future translation technologies", as the emt competence framework (2022: 9) puts it. further research is still needed to uncover the best way to introduce mt to translator trainees. in our future work, we intend to continue analysing the existing data for possible differences between students in terms of their experience and background, as well as collect some more data. students may not only be struggling with difficulties in using different systems and identifying different errors, but also with conceptualising the process they are involved in and what their own role is in that process as opposed to the machine. one area for future exploration would thus be to try and determine whether translator trainees have actual misconceptions or simply lack the accurate terminology to explain how mt works and how mt is different from ht. do students actually think that google translate and other mt engines "understand", "decide" and "get confused"? in fact, the verbs they use may as well be short-hand for processes they cannot define in technical terms.