<figure><img src="image_1.png" /></figure>

HAL
open science

# Dealing with the “elephant in the classroom”:
Developing language students’ machine translation
literacy

Rudy Loock, Sophie Lechauguette, Benjamin Holt

► To cite this version:

Rudy Loock, Sophie Lechauguette, Benjamin Holt. Dealing with the “elephant in the classroom”: Developing language students’ machine translation literacy. Australian Journal of Applied Linguistics, 2022, 5 (3), pp.118-134. 10.29140/ajal.v5n3.53si2. hal-03912198

HAL Id: hal-03912198

https://hal.science/hal-03912198v1

Submitted on 15 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

# Dealing with the “Elephant in the Classroom”:
Developing Language Students’
Machine Translation Literacy

<figure><img src="image_5.png" /></figure>

RUDY LOOCKª

SOPHIE LECHAUGUETTEᵇ

BENJAMIN HOLTª

ªUniversité de Lille and Savoirs, Textes, Langage CNRS
Research Unit, France

rudy.loock@univ-lille.fr; benjamin.holt@univ-lille.fr

ᵇUniversité de Lille and ULR 40 74 – CECILLE,
France

sophie.lechauguette@univ-lille.fr

## Abstract

In this article, we discuss the need for a new digital literacy among language students, namely a literacy in relation to the use of machine translation (MT): MT literacy, following Bowker and Buitrago Ciro (2019). This article does not focus on translation students who, due to the demands of the translation market, have already received a lot of attention from researchers. Rather, we focus on undergraduate language students, and argue that because of their massive use of free online translators—either to translate or to read and write in the language they are learning—instructors need to include a discussion on such tools. We argue that this discussion should include technical and ethical considerations, with a focus on how to ensure that students are able to use MT critically, identifying and correcting errors, for future professional use. Finally, we discuss the thorny issue of evaluation for this type of digital literacy. The discussion is based on several experiments conducted with undergraduate students enrolled in their final year of a three-year program in applied languages. We also believe that the teaching of such skills should not be restricted to language students, since students of all disciplines use online translators, and may continue to do so throughout their professional lives.

**Keywords:** machine translation, online translators, digital literacy, MT literacy, language teaching
# Starting Points

## The Massive Use of Online Translators

A number of studies have shown that students massively use free online translators such as Google Translate (https://translate.google.com), Bing Microsoft Translator (https://www.bing.com/translator), or DeepL (https://www.deepl.com/translator). When asked about their use of such machine translation (MT) tools, between 8 and 9 students out of 10 admit to using them for many different kinds of language tasks. For instance, O'Neill (2019) found that 87.7% of students of Spanish or French as a foreign language at the University of Memphis (n = 310 on campus and 56 online) used online translators for graded and non-graded work, even when this was prohibited. Resende and Way (2021) found that all 90 of their students, who were beginner to advanced learners of English as a second language, were users of machine translation, while Delorme Benites et al. (2021) found that 97.2% of students enrolled in four Swiss universities (n = 1,926) had already used online translators. Our own pilot study at the University of Lille (Loock & Léchauguette, 2021) revealed that 83% of students enrolled in their third and final year of an applied languages program (n = 159) used online translators on a regular basis. Finally, Dorst et al. (2022) found that students at Leiden University's Faculty of Humanities (n = 293) used MT on a regular basis, mostly Google Translate and for educational purposes<fnref n="1" />. Note that this discussion does not include the use of MT by future translators enrolled in a professional translation program, where the teaching of MT and post-editing (MTPE) has been in place for a while and raises very different questions and issues (see below for more justification). Rather, in this article, we are interested in the use of online translators by language students in general, either within a language program or for specialists of other disciplines.

These few results clearly illustrate language students' widespread use of MT tools. It seems that the Covid-19 pandemic and the subsequent shift to online teaching might have encouraged it even further, as suggested by a pilot, small-scale study by Ban Damash (2020). Although online translators, with the advent of neural machine translation (NMT) a few years ago, now provide better results than previous statistical versions of online translators (statistical machine translation, or SMT), the widespread use of machine translation by students is not a new phenomenon. For instance, White and Heindrich (2013), Clifford et al. (2013), and Jolley and Maimone (2015) all reported that between 68% and 88% of students from different backgrounds used SMT-based online translators on a regular basis. What is new is that the quality increase in MT output has made the use of online translators more pervasive and represents a challenge of a different order. Furthermore, student use of online translators appears to be well established before the start of university: for instance, Bourdais and Guichon (2020) found that between 8 and 9 high school students out of 10 (n = 118) used online translators outside the classroom.

## A Biased, Subjective Approach to Online Translators

There appears to be a biased, polarised view of machine translation, be it in the general public, in the student population, or even on the translation market. On the one hand, some users tend to overestimate the performance of online translators, considering that automatically generated translations are “good enough.” Those users go so far as to consider that human translators have become obsolete, despite market surveys showing that this is clearly not the case. This view has been influenced by media coverage (Nunes Vieira, 2020 for example shows that most press articles written between 1986 and 2019 are actually positive about MT), as well as by “human parity claims” (e.g., Hassan et al., 2018) in scientific articles and the press in spite of regular debunking of such a myth (e.g., Toral et al.,

<footnote n="1">¹ Note that throughout the article, some references are conference proceedings, or even sometimes conference presentations. This is due to the fact that we are dealing here with a very recent topic: not a lot of scientific literature has been published and a lot of research is actually in progress. Since the advent of neural machine translation (ca. 2016) is a game-changer, all references to studies involving the use of statistical machine translation have been avoided.</footnote>
2018; Läubli et al., 2019). On the other hand, others refuse to hear about the technology and consider MT output to be full of errors and unlikely to ever be useful. As they see it, it is a tool to be used only by poor translators. So-called “funny MT fails,” found all over the internet, are brandished as evidence that the technology is neither efficient nor usable.

Such attitudes, generally based on subjective perceptions, do not allow for a principled, unprejudiced approach to the tool (Loock, 2019). “MT bashing” does happen, even among professional translators. Some language instructors have expressed reluctance to introduce online translators in the classroom by implementing restrictions on their use, outright bans, or penalties, implying that using MT is tantamount to cheating. However, paradoxically, most trainers actually fail to detect MT use by their students (Henshaw & O’Neill, 2021). At the same time, it has never been easier for students to use online translators: no technical skills are required, as using MT means simply copying and pasting text online and retrieving the output on the screen, completely free of charge (Bowker & Buitrago Ciro, 2019, p. 33; Bowker, 2020, p. 28). Consequently, online translators have sometimes become “the elephant in the classroom”: students use them, and trainers know that students use them. Yet, neither specific training nor even discussions ever take place, as exemplified by the results of the survey conducted by Delorme Benites et al. (2021) (see below). We believe that such a situation is untenable, that online translators should be discussed in class, and that such a discussion should start from students’ actual use of the tools.

The remainder of the article is organised as follows. In the next section, based on a pilot study conducted at the University of Lille, we report on our students’ actual use of online translators and the resulting need for the development of a new digital literacy in relation to MT. Then, in the following section, we make suggestions on what technical and ethical considerations to include when introducing online translators in the language classroom, and what types of activities to implement. We devote the final section to the important but thorny issue of evaluating such a new skill.<fnref n="2" />

# The Need for a New Digital Literacy in Relation to Machine Translation

## The Need to Distinguish Translation Trainees from Other Students

In translation training programs, usually at the master’s level, students have been trained on the use of machine translation and the correction of MT output (post-editing, or MTPE) for a long time now, well before the advent of neural MT (some references on MT teaching date back to the late 1980s). The presence of the technology on the market is widespread. Even though its use varies depending on the type of translation project, MT is clearly here to stay. In Europe, 2018 is considered to be a turning point, as for the first time, MT was used by more than one out of two language service providers (LSPs) (2018 European language industry survey). The Directorate-General for Translation and its European Master’s in Translation (EMT) network now consider that understanding and using machine translation must be part of any translation curriculum. The EMT competence framework states that students should have “a basic knowledge of machine translation technologies and the ability to implement machine translation according to potential needs”.<fnref n="3" /> Programs thus teach students what MT is, when it is relevant to use it, how to post-edit MT output, and how to charge clients for MTPE projects. MT teaching has been integrated into most programs, and online translators, generic or specialised, are now considered as tools among others, such as computer-assisted translation (CAT) tools, to be used with full awareness of their potentials and limits. Such awareness is a condition to being able to define one’s added value over the machine (Loock, 2019), with a view to developing a human-centred

<footnote n="2">² We would like to thank our colleagues who have also taught the translation class described in this article: Cédric Courtois, Emily Holt, Virginie Pfeiffer, Marie-Joëlle Ravit, Consuelo Sanfourche.</footnote>

<footnote n="3">³ Source: https://ec.europa.eu/info/resources-partners/european-masters-translation-emt/european-masters-translation-emt-explained_en</footnote>
approach: the human translator is placed in the centre of the translation process and uses tools, includ-
ing MT, in a relevant way to deal with translation projects. This does not mean that all issues have
been resolved (the extent to which students should be able to understand how MT works seems to be
an important current issue), or that there is no resistance from students or trainers (e.g., De Faria Pires,
2020; Torres-Simón & Pym, 2021).

For language students not enrolled in translation programs, the issues are actually very different. As
detailed below, students use online translators to complete many different tasks such as understanding
a text, writing in a foreign language, or correcting grammar. They use MT for classes or degrees not
involving translation. As they are generally not trained in the use of online translators, they are on their
own. We think that their use of MT could be optimised in order to avoid confusion with other tools such
as online dictionaries and overconfidence in MT output. Regardless of their major, students learning
foreign languages are likely to communicate in a foreign language in their future professional lives. This
is definitely the case with students in applied languages, who are the focus of this article. We consider
critical use of online translators to be absolutely necessary for them. This is a challenge for trainers, who
generally have not been trained themselves and work in environments lacking any clear policies in rela-
tion to MT use. For example, Delorme Benites et al. (2021, p. 83) found that 77.1% of trainers in 4 Swiss
universities (n = 666) did not mention MT in their classes, with only 4% explaining how the technology
works. Similarly, 83.9% of students (n = 1,926) never received any explicit instruction in relation to MT.
In this article, we put forward ideas on how to train language students to adopt an informed, principled,
and critical approach to online translators. We also discuss the difficulties in evaluating such a skill.

Assessing Student's Use of Online Translators: A Pilot Study

Perfectly aware of machine translation's likely status as an “elephant in the classroom” in our trans-
lation classes for undergraduate students (see below for a description of our students' profile) at the
University of Lille, we decided in March 2020 to launch an investigation into our students' use of
online translators, as well as into their capacity to identify and correct MT errors after 5 semesters of
(non-professional) translation training. This investigation consisted of an online survey and the correc-
tion of an English-French machine translation output provided by a generic online translator (DeepL).
Our aim was to assess our students' use of online translators in order to develop a curriculum that
would provide them with specific training based on their actual uses and needs.

The students in our pilot study (n = 159) were enrolled in the final semester of a three-year applied
languages university program in 2019–2020. Such students specialise in two languages, one of which
is compulsory English, and attend economics, law, management, and communication classes. They
attend what we call “pedagogical translation classes” from the second to the fifth semesters, in which
they translate press texts, tourism brochures, extracts from websites, or infographics, with an approach
meant to help them develop their language skills (the classes do not focus on professional translation
training), hence the term “pedagogical translation” for such classes at an undergraduate level. No
specific elements related to professional translation like the use of computer-assisted translation tools
or project management are included. In addition, the translation classes help them study professional
issues in different languages and acquire the relevant terminology. Most students will go on to work in
the tourism industry, international relations, international commerce, and for some of them, a minority,
in the translation industry. They need to be able to understand and communicate on professional issues
in their three working languages, including French, the mother tongue of most of our students, and
the translation classes are meant to help them achieve this goal (multilingual communication). For the
purpose of this exploratory study, in which students completed an anonymous survey on their use of
online translators and had to correct the French MT output of an English text, the answers and results
of international students whose mother tongue was not French were not included.
The results, presented in detail in Loock and Léchauguette (2021), revealed that 83% of our students used online translators, either systematically (3.8%), often (29.5%), or sometimes (54.5%). Most of them reported using DeepL (8 students out of 10), as well as Google Translate (3 students out of 10), while other online translators (Systran, Reverso Traduction) were only marginally used. Some students also mentioned WordReference and Linguee, mistaking them for online translators, which shows some confusion as to what machine translation really is. Students only very rarely reported copying and pasting the entire text (4.9%), reporting instead typing parts of sentences, even words, and inserting the suggestions provided by the online translator into their own translations (4 students out of 10). 1 in 4 students used MT to gain a general idea of the meaning of the text before translating it themselves. Other students preferred to translate on their own and then compare their work with the translation suggested by the online tool (19.8%). A follow-up study in September 2021 (n = 164) revealed that students used online translators for more than just translation exercises: while 8 out of 10 students reported using MT tools for this purpose, 4.5 out of 10 also used them as a writing aid (when writing an essay in a foreign language for example); 1 out of 2 used them to improve their understanding of a text, and 1 in 6 in order to help with grammar exercises. Most students reported general satisfaction with MT output (4 out of 10 often satisfied, and another 4 sometimes). A large majority of students (93.8%) in our pilot study thought that they were able to identify MT errors, either with no difficulty whatsoever (18.5%) or quite easily (75.3%).

However, our analysis revealed that students greatly overestimated their ability to correct errors in MT output. The students were presented with the French translation of an English press text, produced by DeepL with no modification whatsoever. Both texts were provided side by side to the students, whose task was to correct any error related to meaning (accuracy) or to the quality of the target text (fluency). No justification was demanded. The evaluators had identified twenty errors requiring correction: meaning issues, inappropriate lexical/structural choices based on the source text (calques), stylistically awkward sentences, incorrect use of tenses, or punctuation. Figure 1 shows the beginning of the text (examples of errors identified by the evaluators are underlined or circled). Out of the 20 errors, only

<figure><img src="image_6.png" /><figcaption>**Figure 1** *Beginning of the text submitted to the students for error identification and correction*
(left: English source text, right: French machine-translated text).</figcaption></figure>
5.29 on average were identified and corrected by the students, 2.29 errors were spotted but wrongly corrected, and over half (12.42) were not spotted at all, suggesting that students place excessive trust in what is provided by online translators and/or that their mastery of English and knowledge of their native tongue is lacking. We noticed that students were better at identifying accuracy issues than fluency issues, due to being heavily primed (see Carl & Schaeffer, 2017 for the concept of “priming”) by the MT output (for more detailed results and analysis, see Loock & Léchauguette, 2021).

Such a gap between students’ self-confidence at identifying MT errors and the reality exposed by our online test led us to conclude that students failed to use online translators effectively, and that specific training needed to be included in our classes in order to develop a new literacy and empower students to become critical, efficient users of online translators. This is where the concept of “MT literacy,” defined in Bowker and Buitrago Ciro (2019), is relevant: it refers to a series of skills in relation to users’ capacity to understand how MT systems work, when it is relevant to use them, and when and how to modify MT output (Bowker & Buitrago Ciro, 2019, p. 88). Our pilot study clearly demonstrated that our students needed to develop their MT literacy. Most of them will work outside the translation industry but might need online translators for multilingual communication. To meet this need, we offer in the next two sections a series of suggestions to empower them and broach the thorny issue of evaluation.

# Finding the Right Way to Develop Students' MT Literacy

## The Different Components of Such a Literacy

Students who will later work outside the translation industry need to be familiar with the different existing tools and be sensitised to recurring issues that require fixing for professional multilingual communication. This means that some theoretical knowledge on online translators is needed, together with some practical exercises that will help them develop their critical eye for MT output. Complementarily, students need to be sensitised to some ethical issues posed by machine translation. Below we make suggestions on what to include in such an approach that is both theoretical and practical. We borrow the MT literacy concept from Bowker and Buitago Ciro (2019) and Bowker (2020), which both explain the need to develop MT users’ critical eye. While Bowker and Buitrago Ciro (2019) provides a general framework with a number of general, language-independent suggestions for achieving such a goal for scholarly communication, Bowker (2020) provides a series of key elements to be included in a relevant MT workshop for international business students: privacy/confidentiality, academic integrity, algorithmic bias, existence of different tools, existence of different translation tasks, and the need to ensure that the input is unambiguous (see also Bowker, 2021 for a Machine Translation Literacy module for students from the Faculty of Arts at the University of Ottawa). In this article, we go further. Based on our students’ and our own use of online translators, we make example-based, specific suggestions for French learners of English enrolled in an applied languages program. While some elements of a relevant theoretical approach can be language-independent, as those mentioned in Bowker and Ciro (2019) and Bowker (2020), e.g., defining what an online translator is or raising awareness about algorithmic bias or ethical issues, we believe that focusing on recurring errors for a specific language pair, in our case English-French, is crucial to develop students’ MT literacy. This is why we adopt below an example-based approach: we believe in the use of carefully selected examples to illustrate recurring errors to be found in MT output in order to develop students’ critical eye. Also, while some of the examples below might be considered to contain minor errors that do not hinder communication, these still need to be corrected in the context of professional communication. In doing so, we follow Bowker (2020, p. 28)’s claim that “the output of neural machine translation systems is much more fluid and can often represent a viable first draft that students can take as a starting point.” This is not meant to be in contradiction with our criticism above of the use of “funny” MT fails which might discredit the technology, but rather to help users refine their use of online translators in a professional context.
## Technical considerations

Most students today know how to obtain MT output from an online translator: no technical skills are required beyond copying and pasting text in a specific area of the webpage, or activating an automatic browser extension (see e.g., Bowker, 2020, p. 28). The source language is even generally automatically detected, although detection errors for some languages remain. The issues pertaining to the technical aspects of online translators lie elsewhere: students need to understand how automatic translators work in order to adopt a critical approach.

First of all, it seems important to define what machine translation and online translators are within the context of online translation aids, which include monolingual/multilingual dictionaries, some of which provide concordance lines (e.g., www.linguee.com), corpus tools such as the Sketch Engine website (https://www.sketchengine.eu/), thesauruses, or spelling/grammar checkers. All these tools provide relevant yet different types of information for students interested in multilingual communication. Our online survey clearly revealed some confusion between online translators and other tools such as WordReference or Linguee. Once the different types of online translation aids have been mentioned, training on the use of online translators should start with a basic discussion on how MT engines work (combination of an algorithm and corpus data) and an explanation of the differences in particular with dictionaries, hence the need to provide input with a larger context (as mentioned above, our pilot study revealed that 4 students out of 10 typed words or parts of sentences when using online translators). Then, comparing results obtained from different online translators can sensitise students to the fact that such tools are not as reliable as calculators, and results may vary depending on the corpus data used by the MT engine. All of this can help students understand what an online translator is and develop a critical approach.

In doing so, trainers should insist on the fact that online translators provide linguistic transfer only, and completely overlook cultural elements in a text, as illustrated by examples in (1),<fnref n="4" /> where *rising junior* has been mistranslated by both Google Translate, which provides a literal, nonsensical translation, and DeepL, which provides an understandable but inappropriate equivalent, as a *rising junior* would be a student at the end of his second year at university, not in his first year at university.

(1) Jackson England, 20, is a *rising junior* at Columbia University.
Jackson England, 20 ans, est un junior montant à l'Université de Columbia. (Google Translate, November 2021)
Jackson England, 20 ans, est un étudiant de première année à l'Université de Columbia. (DeepL, November 2022)

Similarly, online translators do not perform very well on creative texts that include word play for instance, as the machine is not meant to be able to deal with such language use, as in (2), where *tangfastically* is an adverb derived from “Haribo Tangfastics,” the name of a tangy mix sold by the company. An equivalent in French has been provided by DeepL based on the closeness of the word to *tangibly*, while the word has not been translated by Google Translate:

(2) The lorry driver shortage could become tangfastically real for British children and their parents after the confectionery manufacturer Haribo told shopkeepers it was struggling to deliver enough bags of sweets.

<footnote n="4">⁴ All our examples have been obtained via the online versions of DeepL and Google Translate, by copying and pasting sentences not in isolation, but with the preceding and following context (this might have some influence on the MT output). Examples are provided as raw MT output, meaning that nothing has been modified in the translations provided by the two online translators. The tool and date it was used are indicated in brackets after each example.</footnote>
La pénurie de chauffeurs routiers pourrait devenir **tangiblement** réelle pour les enfants britanniques et leurs parents, après que le fabricant de confiseries Haribo a déclaré aux commerçants qu'il avait du mal à livrer suffisamment de sacs de bonbons. (DeepL, November 2021)

La pénurie de chauffeurs routiers pourrait devenir une réalité pour les enfants britanniques et leurs parents après que le fabricant de confiseries Haribo a déclaré aux commerçants qu'il avait du mal à livrer suffisamment de sacs de bonbons. (Google Translate, November 2021)

The same goes for idioms, which can be translated literally:

(3) The Conservative & Unionist party is no more. It has ceased to be. It has expired and gone to meet its maker. It's kicked the bucket, shuffled off its mortal coil, run down the curtain and joined the choir invisible. This is an ex-party.

Le parti conservateur et unioniste n'est plus. Il a cessé d'exister. Il a expiré et est allé à la rencontre de son créateur. Il a donné un coup de pied au seau, s'est débarrassé de son enveloppe mortelle, a baissé le rideau et rejoint le chœur invisible. C'est un ex-parti. (DeepL, February 2022)

Second, it is important that students be aware of recurring errors made by online translators in spite of the sometimes impressive quality of NMT output (working on MT errors should not lead to discredit-ing online translators as a whole). These are well documented and can easily be shown in class, lead-ing to necessary corrections. For example, online translators might provide mistranslations because of lexical or syntactic ambiguities in the source sentence (4a/b). They are also prone to reproducing punctuation without respecting the typographical norms of the target language. Proper names are regu-larly translated when they should not be (5a/b). Students should be aware that cases of non-translations regularly happen (2b, 6), for rare words but not only: for instance, Loock (2020) showed that adverbs ending in -ly were not translated in 3.5% of cases with DeepL, 10% with eTranslation, the European Commission's MT system. Also, online translators might "create" words (7), as shown by some stud-ies: e.g., 15% of invented words for the English-Dutch translation direction according to Macken et al., 2019; see also examples in De Clercq et al., 2021 for the English-French language pair, where French output includes words like *tonnamment* for *thunderously* or *torch-éclairé* for *torch-lit*. The provided translation is based on the noun *thunder*, and creates a non-existent word in French, instead of, for instance, *bruyamment*, which demonstrates, if need be, that MT is not based on meaning. In the case of *torch-éclairé*, students can be made to observe the word-for-word translation, with a spelling mistake since in French the word is *torche* or *lampe de poche* and a correct translation would be *éclairé par une torche/lampe de poche*.

(4) a. [T]he **line** there is prime evidence of the growing economic impact that the coronavirus (...) has had on tourism in Paris and elsewhere across Europe.

[L]a **ligne qui y est tracée** est une preuve éclatante de l'impact économique croissant que le corona-virus (...) a eu sur le tourisme à Paris et ailleurs en Europe.

b. In news that will concern millions of small children and the carers who **bribe** them, Haribo has reportedly cancelled planned promotions on its share bags as it tries to maintain availability.

Dans une nouvelle qui concernera des millions de petits enfants et les personnes qui les corrompent, Haribo aurait annulé les promotions prévues sur ses sacs de partage alors qu'il tente de maintenir la disponibilité. (DeepL, November 2021)

(5) a. Jackson England, 20, is a rising junior at Columbia University. **England** entered his freshman year as a pre-med student, with plans for a long and successful career in the medical field as a neurosurgeon. Jackson England, 20 ans, est un junior montant à l'Université de Columbia. **L'Angleterre** est entrée dans sa première année en tant qu'étudiant en pré-médecine, avec des plans pour une longue et fructueuse carrière dans le domaine médical en tant que neurochirurgien. (Google Translate, November 2021)
b. This year, the traditional <u>Black Friday</u> to focus on other sustainability initiatives.

Cette année, ils utiliseront le traditionnel <u>vendredi noir</u> pour se concentrer sur d'autres initiatives de durabilité. (DeepL, November 2020)

(6) In his address he <u>baldly</u> told his overwhelmingly climate sceptic opponents that “no challenge poses a greater threat to future generations than climate change.”

Dans son allocution, il a déclaré à ses adversaires, qui sont majoritairement sceptiques face aux changements climatiques, qu’“aucun défi ne constitue une plus grande menace pour les générations futures que le changement climatique.” (DeepL, January 2020)

(7) Though Luckhurst denounces Resident Evil as “<u>thunderously stupid</u>,” he also calls it “delirious”: isn’t that a recommendation?

Bien que Luckhurst dénonce Resident Evil comme “tonnamment stupide,” il l’appelle aussi “délire”: n’est-ce pas là une recommandation? (DeepL, October 2018)

It also seems important to sensitise students to the different kinds of biases that can come up in MT output, in particular algorithmic biases for example, mentioned in Bowker (2020: 35) and defined by Vanmassenhove et al. (2021: 1) as “an exacerbation of frequently observed patterns in combination with a loss of less frequent ones.” This means that frequent translation choices will be selected by the online translator at the expense of less frequent, maybe more creative or relevant possibilities (over-representation of most frequent equivalents), leading in turn to a lesser lexical variety and standardised language in machine-translated texts (Vanmassenhove et al., 2019). Such biases can then lead to gender bias, which is also well documented (see e.g., Salvodi et al., 2021) and can easily be reproduced with students, as illustrated in (8), where *president* is typically translated as *président* (masculine) but *nurse* is translated as *infirmière* (feminine), even when pronominal reference shows the president is female (8c). Another relevant example is (9a/b), where the order of sentences seems to have no effect on the MT output, with *directors* being translated as *directeurs* (masculine) and not *directrices* (feminine) in spite of the presence of three unambiguously female names (note the presence of *chairman* translated as *présidente*, though).

(8) a. The <u>president</u> has arrived.

Le président est arrivé.

b. The <u>nurse</u> has arrived.

L'infirmière est arrivée.

c. The <u>president</u> asked the <u>secretary</u> to help her in the procedure.

Le président a demandé à la <u>secrétaire</u> de l'aider dans la procédure (DeepL, August 2021)

(9) a. Of the permanent senior UK managers at the company—which is owned by its staff and runs 50 department stores and the Waitrose supermarket chain—three <u>directors</u> are from an ethnic minorityyy. They are the chairman, Sharon White, who joined in February, the strategy director, Nina Bhatia, who was hired by White, and Bérangère Michel, who is in charge of customer service.

Parmi les cadres supérieurs britanniques permanents de l'entreprise—qui est détenue par son personnel et gère 50 grands magasins et la chaîne de supermarchés Waitrose—trois <u>directeurs</u> sont issus d'une minorité ethnique. Il s'agit de la présidente, Sharon White, qui a rejoint l'entreprise en février, de la directrice de la stratégie, Nina Bhatia, qui a été engagée par Mme White, et de Bérangère Michel, qui est chargée du service clientèle. (DeepL, August 2021)

b. Of the permanent senior UK managers at the company—which is owned by its staff and runs 50 department stores and the Waitrose supermarket chain—the chairman, Sharon White, who joined in February, the strategy director, Nina Bhatia, who was hired by White, and Bérangère Michel, who is in charge of customer service. This means that three <u>directors</u> are from an ethnic minority.
Parmi les cadres supérieurs britanniques permanents de l'entreprise—qui est détenue par son personnel et gère 50 grands magasins et la chaîne de supermarchés Waitrose—la présidente, Sharon White, qui a rejoint l'entreprise en février, de la directrice de la stratégie, Nina Bhatia, qui a été engagée par Mme White, et de Bérangère Michel, qui est chargée du service clientèle. Cela signifie que trois directeurs sont issus d'une minorité ethnique. (DeepL, August 2021)

Regular issues related to specific language pairs, in our case English-French, can also be mentioned, such as the translation of compounds (10), which requires consideration of the semantic relationship between the different elements. Machine translation tends to use the preposition *de* by default, but in (10), this use is infelicitous; a complex preposition such as *dans le cadre de* would be appropriate, as would the use of the plural. The use of pronouns is also an issue (11), as neutral pronouns do not exist in French, and using *il* to refer to a company that was previously designated with a proper name is not relevant (here, a lexical description such as *la marque* or *l'industriel*, which allows for the use of the pronouns *elle* or *il* later in the text, would be better). Another important linguistic feature is the need for lexical variation in French texts as opposed to English texts, for example for reporting verbs to introduce quotations in press texts or dialogue in fiction texts: while in English *say* is by far the most frequent, its direct equivalent *dire* is much less frequent, with 6,355.42 occurrences per million words for *say* in original English press texts vs. 946.91 for *dire* in original French press texts according to Loock (2018). And yet, French MT output shows a very significant overrepresentation of the verb *dire* (Loock, 2018: 795), requiring corrections to introduce verbs such as *déclarer*, *affirmer*, *expliquer*, etc. depending on the context (note however that in examples (2) and (6), the verb *déclarer* has been used as an equivalent of *tell*, which shows that literal translations are not systematic).

(10) Student exchange host family FAQ (brochure title)
FAQ sur les familles d'accueil d'échange étudiant (DeepL, February 2022)

(11) Haribo told wholesale and retail customers *it* had a number of problems in the supply chain and was “working flat out to manage the situation.”

Haribo a déclaré aux clients de gros et de détail qu'il avait un certain nombre de problèmes dans la chaîne d'approvisionnement et qu'il “travaillait à fond pour gérer la situation.” (DeepL, December 2021)

Such technical limitations of online translators should make students aware of the existence of what has been called “machine translationese,” which requires human intervention. Our discussion thus illustrates the necessity for trainers to be aware of the scientific literature on the subject: with the advent of neural MT, researchers have published an impressive number of studies on what MT systems can and cannot do, as well as their progress. The information contained in such publications can quite easily be made accessible to students in class in order to sensitise them to the limits of the tools. As the technology is constantly improving, it is important to provide dates when using examples obtained via online translators, as we are doing here: results change over time (see MT output for example (6) in February 2022 in (12)), generally for the better. Students need to become aware of this constant evolution as part of their MT literacy development.

(12) In his address he *baldly* told his overwhelmingly climate sceptic opponents that “no challenge poses a greater threat to future generations than climate change.”

Dans son discours, il a déclaré *sans ambages* à ses opposants, majoritairement climato-sceptiques, qu’“aucun défi ne constitue une plus grande menace pour les générations futures que le changement climatique.” (DeepL, February 2022)

Finally, when defining how online translators work, it is important to show that the tool is only as good as the data behind it: errors in the corpus will be reproduced in the MT output. For example, as
shown in Figure 1, the use of the subjunctive mood was found after *après que* instead of the indicative mood (*après qu’un touriste chinois de 80 ans soit mort du virus*). This is a very frequent mistake in present-day French, so frequent that its very status as an error can be questioned, and is probably present in the data used by DeepL, hence its presence in the MT output instead of the indicative mood (*est mort*), or of a nominalization which would be more natural (*après le décès d’un touriste chinois*). We still consider this an MT error to be corrected, as normative language is generally expected in professional settings.

However—and it is important to insist on this—we do *not* believe that the discussion on the limits of online translators should eventually lead to machine translation as a whole being discredited. This is too often the case, in particular on the internet through the use of so-called “funny MT fails.” This is not helping students—or the public in general—to make an efficient, informed use of online translators. Inputting verbless sentences, song lyrics, idioms, word plays, etc. will necessarily lead to errors in the MT output, which does not mean that online translators are useless. It is therefore important that trainers not restrict themselves to uncovering recurring MT errors, but discuss interesting, well-written suggestions, leading to the use of online translators as a supplement to online dictionaries or glossaries. Developing one’s MT literacy means discriminating between relevant and irrelevant output. With the substantial—and ongoing—progress of neural systems, using MT output as a starting point has now become very relevant for many users. Still, for a professional use where excellent communication skills are required, human intervention remains necessary.

## Ethical considerations

Using machine translation also comes with ethical issues, of which students should be made aware. In some institutions, using MT for graded or even non-graded work is akin to plagiarism. In particular, as students use online translators as writing tools to help them write essays for example (this use was reported by half of the students in our survey), they need to be aware that using an extract from a document, having it translated with an online translator and using it without mentioning the source and/or the fact that it has been machine translated, is unethical. In Bowker (2020)’s survey, language trainers mentioned academic integrity in relation to MT use as an important issue to be addressed (Bowker, 2020: 33–34).

Confidentiality issues also need to be discussed: free online translators should never be used for sensitive, confidential data. There is a (proven) risk that the data might end up publicly available on the internet. The data breach experienced by a Norwegian oil company following the use of a free online translator can be used to illustrate this point (https://slator.com/translate-com-exposes-highly-sensitive-information-massive-privacy-breach/). The ecological issues related to the use of deep learning necessary for the development and use of MT tools should not be overlooked either (Strubell et al., 2019); using online translators has an environmental impact. Finally, students should be aware that machine translation cannot exist without corpus data provided by human translations, and should therefore discuss the extent to which human labour is exploited. In a nutshell, students should be sensitised to “fair use” practices of MT (see Moorkens et al., 2020). Teaching institutions need to write and implement clear institutional policies to guide students and let them know what they are and are not allowed to do.

## Examples of activities

In addition to the previously mentioned online tests based on identifying and correcting the recurring errors found in MT, different types of exercises can be introduced in the classroom to develop learners’ MT literacy and critical stance on online translators. First of all, some new functionalities can help them
understand that the MT output on the screen is but one possibility among many, and that online translators are not calculators. For example, DeepL provides the possibility to click on one word in the target text in order to see other possible equivalents in a drop-down list. Although this is limited to lexical choice and not sentence structure, it can help students select a more appropriate translation when needed, as they would with a dictionary. In Google Translate, clicking on a sentence in the MT output provides alternative translations for the whole sentence, and the user can select the most suitable one. This dynamic approach to online translators places students in a decision-making position instead of a passive one. Above all, making a choice conveys the idea that what appears on the screen is not the only possible equivalence.

At the University of Lille, following the results of our pilot study, different types of activities were introduced in our translation classes alongside the translation of texts. Students first need to be given a definition of online translators in comparison with other online translation aids (see above). For example, comparing the MT output with the ‘human’ translation of the same text (‘human’ meaning without using MT, and obtained collaboratively with the students) can uncover similarities and differences between the two modes of translation. Table 1 provides an example of such comparisons, which reveal some lexical divergence, e.g., *enseignement en distanciel* vs. *cours à distance*, *ressources pédagogiques* vs. *ressources scolaires*, as well as structural divergence (use of an impersonal structure *il s’agit* in the human translation). As mentioned above, the idea is to identify relevant and irrelevant MT suggestions (note that for the first sentence, the human translation and MT outputs are nearly identical), so it is possible that the MT output can provide better solutions than the human (the use of *période marquée par* for example is clearly a stylistically elegant suggestion from DeepL). In the end, the idea is to uncover the added value of the human decision maker. Complementarily, comparing the outputs of different MT tools is another relevant exercise for observing and evaluating their qualities and differences. Classifying translations (human, MT) from ‘best’ to ‘worst’ can also be an interesting exercise.

<table><caption>**Table 1** Comparison of ‘Human’ Collaborative Translation and MT Outputs</caption><thead><tr><td>Source text</td><td>'Human' translation with students</td><td>Google Translate (1 September 2021)</td><td>DeepL (1 September 2021)</td></tr></thead><tbody><tr><td>The coronavirus pandemic has changed the world as we used to know it before long months of quarantine and social distancing.</td><td>La pandémie de coronavirus a changé le monde tel que nous le connaissions avant une période de quarantaine et de distanciation sociale qui aura duré plusieurs mois.</td><td>La pandémie de coronavirus a changé le monde tel que nous le connaissions avant de longs mois de quarantaine et de distanciation sociale.</td><td>La pandémie de coronavirus a bouleversé le monde tel que nous le connaissions avant de longs mois de quarantaine et de distanciation sociale.</td></tr><tr><td>For many college students, this has been a time of uncertainty and worry.</td><td>Pour de nombreux étudiants à l'université, il s'agit d'une période d'incertitude et d'inquiétude.</td><td>Pour de nombreux étudiants, cette période a été une période d'incertitude et d'inquiétude.</td><td>Pour de nombreux étudiants, cette période a été marquée par l'incertitude et l'inquiétude.</td></tr><tr><td>Many schools have switched to remote classes, a lot of school resources aren't available, and the job market is one of the worst in decades.</td><td>De nombreux établissements ont basculé vers l'enseignement en distanciel, de nombreuses ressources pédagogiques ne sont pas disponibles, et le marché de l'emploi est l'un des moins favorables depuis des décennies.</td><td>De nombreuses écoles sont passées à des cours à distance, de nombreuses ressources scolaires ne sont pas disponibles et le marché du travail est l'un des pires depuis des décennies.</td><td>De nombreuses écoles sont passées à des cours à distance, beaucoup de ressources scolaires ne sont pas disponibles et le marché du travail est l'un des pires depuis des décennies.</td></tr></tbody></table>
Identifying and correcting MT errors is another relevant task, which we have used both in class and for evaluation (see below), even though this means focusing on what online translators do wrong at the expense of the relevant translation solutions that they can provide. We have experienced different types of exercises: texts vs. isolated sentences; underlining or not the errors produced by MT. Table 2 provides an extract from an exercise where elements have been underlined in the machine-translated text for the students to correct. The text deals with the new Turing scheme in the United Kingdom, meant to replace the Erasmus+ study abroad program.

Another relevant—but not yet implemented—activity would be to have two groups of students translating the same text, one with access to an online translator and another without. This would allow for a comparison of the two translations, and an analysis of their respective qualities. Our pilot study showed for example that students overlooked obvious cases of calques in the MT output that they would not have produced themselves, e.g., *dictait le mandarin* (*<dictating mandarin>*), overlooked by 8 students out of 10 (see Figure 1).

## The Thorny Issue of Evaluation

In this final section, we report on our attempts at finding the best way to evaluate this new skill within a translation class. Although MT-based activities in the classroom have proven to be interesting—and even motivating—for students, who have the feeling that we are addressing their actual practices, evaluating student’s MT literacy remains difficult. As mentioned above, the test in our pilot study clearly showed that students were unable to identify and correct MT errors (more than 12 errors out of 20 were simply overlooked). In spite of the specific training and activities mentioned above, we failed to detect a marked improvement in our students’ performance with the same test format (machine translation of an English press text into French, the mother tongue of the majority of our students, with the instruction to correct both the accuracy and fluency issues in the MT output). Some adaptation was therefore needed to empower students to make an informed use of MT.

First, in April 2021, we provided sentences in isolation, for both translation directions (English-French and French-English). This hinted that there was at least one error to be corrected per sentence, and did improve our students’ performance. For instance, for the French-English translation direction, which is more difficult for native speakers of French, students (n = 196) were able to correctly identify an average of 10.2 MT errors out of 23, a 44% success rate, as opposed to 37% for our pilot study (text not segmented into sentences). Still, 12.72 errors on average were overlooked (55% vs. 62% in the pilot study).

<table><caption>**Table 2** MT Output for Correction Exercise</caption><thead><tr><td>Source text</td><td>DeepL (16 January 2022)</td></tr></thead><tbody><tr><td>"We also don't know whether these placements have been arranged and confirmed and, crucially, how much funding will be allocated to each participant." The DfE says the new scheme means that young people "will be funded to take up work and study placements" in 150 countries. However, many of the countries listed by the DfE have border entry restrictions because of the Covid pandemic that will continue to hamper participation for the coming next year.</td><td>"Nous ne savons pas non plus si ces <strong>placements</strong> ont été organisés et confirmés et, surtout, combien de fonds seront alloués à chaque participant."<br/>Selon le <strong>DfE</strong>, le nouveau programme signifie que <strong>les jeunes</strong> "seront financés pour effectuer des stages de travail et d'études" dans 150 pays. Cependant, bon nombre des pays énumérés par le DfE <strong>ont</strong> des restrictions d'entrée aux frontières en raison de la pandémie de Covid qui continuera à entraver la participation pour l'année prochaine.</td></tr></tbody></table>
Then, in December 2021, we decided to give the students a hybrid exam in which they had to translate the beginning of an English text into French and correct five isolated sentences extracted from the remainder of the text that had been translated automatically by DeepL. The elements in need of correction were underlined, meaning that the students (n = 158) were aware of what needed to be rewritten (see Table 3 for some examples). This provided much better results, as the errors were adequately corrected 67% of the time. We can therefore conclude that students are better at correcting MT errors that have been pre-identified, as this prevents them from “correcting” error-free elements of a sentence or text and allows them to focus on one translation issue at a time. This also means that their MT literacy needs further development: being able to spot errors in MT output is necessary for a professional use of the tool.

Also, what we gleaned from these different examinations is that students seem to focus primarily on lexical choices, and much less on syntactic organisation. Exercises in the classroom should therefore insist on the need to rearrange the structure of sentences in order to avoid literal translations that are either ungrammatical (like *dictait le mandarin* above) or stylistically inelegant (e.g., use of a passive voice to translate an English sentence with the passive voice, while an active voice is often a better option) in the target language.

## Conclusions and Future Work

Starting from the observation that language students were in need of developing their machine translation literacy and their use of online translators for future professional communication, we have suggested a theoretical and practical approach to MT literacy training. Experiments with French students at the University of Lille revealed that this is no easy task: while our students may have become better at correcting errors in the MT output, they still struggle with their identification. The improvement in our students' results for the different tests actually tells us nothing about their (in)ability to analyse MT output independently in real-life situations. And yet, for a professional use of such tools, even outside the translation market, this is a key issue: students must not only be able to correct MT errors, but identify them first. Further studies are therefore necessary to uncover how best to help students understand how MT tools work and identify their errors. In the short term, we would like to conduct the aforementioned experiment with two groups of students, one with MT and one without, so that students can become aware of their added value over the machine and further develop their scepticism towards MT output.

Another aim is to pursue the preparation of pedagogical material. Some research projects have made material publicly available, e.g., the European MultiTraINMT project (Machine Translation training for multilingual citizens, http://www.multitrainmt.eu) or the Machine Translation Literacy project (https://sites.google.com/view/machinetranslationliteracy/). Trainers are in dire need of textbooks and activities, and we would like to help meet this demand. Finally, we must develop specific training for language teachers if we are ever to free the elephant in the classroom.

<table><caption>**Table 3** MT Output for Correction Examination</caption><thead><tr><td>Original sentences</td><td>MT output (DeepL)</td></tr></thead><tbody><tr><td>“Do you get them from supermarket bins?” I asked them. They told me they regularly collected and redistributed the contents of the big skip-like bins behind supermarkets.</td><td>« <b>Les obtenez-vous</b> dans les poubelles des supermarchés? » leur ai-je demandé. Ils m’ont dit qu’ils récupéraient et redistribuaient régulièrement le contenu <b>des grandes bennes à benne</b> derrière les supermarchés.</td></tr><tr><td>I had heard of people bin-diving before and I was captivated by their story.</td><td>J’avais déjà entendu parler de personnes <b>faisant de la plongée sous-marine</b> et j’ai été captivé par leur histoire.</td></tr></tbody></table>
References

Bin Dahmash, N. (2020). "I can't live without Google Translate": A close look at the use of Google Translate app by second language learners in Saudi Arabia. *Arab World English Journal*, *11*(3), 226-240.

Bourdais, A., & Guichon, N. (2020). Représentations et usages du traducteur en ligne par les lycéens, *Alsic* 23(1). https://doi.org/10.4000/alsic.4533

Bowker, L. (2020). Machine translation literacy instruction for international business students and business English instructors. *Journal of Business & Finance Librarianship*, 25(1-2), 25-43.
https://doi.org/10.1080/08963568.2020.1794739

Bowker, L. (2021). Promoting linguistic diversity and inclusion: incorporating machine translation literacy into information literacy instruction for undergraduate students. *The International Journal of Information, Diversity, & Inclusion, 5*(3). https://doi.org/10.3313/ijidi.v5i3.36159

Bowker, L., & Buitrago Ciro, J. (2019). *Machine Translation and global research: Towards improved machine translation literacy in the scholarly community*. Bingley: Emerald Publishing.

Carl, M., & Schaeffer, M. (2017). Why translation is difficult: A corpus-based study of non-literality in post-editing and from-scratch translation. *HERMES–Journal of Language and Communication in Business*, *56*, 43–57. https://doi.org/10.7146/hjlcb.v0i56.97201

Clifford, J., Merschel, L., & Munné J. (2013). Surveying the landscape: What is the role of machine translation in language learning? *@tic revista d'innovació educativa, 10*, 108-121. https://doi.org/10.7203/attic.10.2228

De Clercq, O. Gert de Sutter, Rudy Loock, Bert Cappelle, Koen Plevoets (2021). Uncovering machine translationese using corpus analysis techniques to distinguish between original and machine-translated French. Special issue of *Translation Quarterly, 101*, 21–45.

De Faria Pires, L. (2018). Intégration de la traduction automatique neuronale à la formation universitaire des futurs traducteurs: pistes d'exploration. *Myriades, 4*, 53–65.

Delorme Benites, A., Cotelli Kureth, S., Lehr, C., & Steele, E. (2021). Machine translation literacy: a panorama of practices at Swiss universities and implications for language teaching. In N. Zoghlami, C. Brudermann, C. Sarré, M. Grosbois, L. Bradley, & S. Thouësny (Eds.), *CALL and professionalisation: short papers from EUROCALL 2021*. (pp. 80–87). Research-publishing.net

Dorst, A. G., Valdez, S., & Bouman, H. (2022). Machine translation in the multilingual classroom. How, when and why do humanities students at a Dutch university use machine translation? *Translation and Translanguaging in Multilingual Contexts, 8*(1), 49–66. https://doi.org/10.1075/ttmc.00080.dor

Hassan, H., Aue, A., Chen, C., Chowdhary, V., Clark, J., Federmann, C., Huang, X., Junczys-Dowmunt, M., Lewis, W., Li, M., Liu, S., Liu, T-Y., Luo R., Menezes, A., Qin, T., Seide, F., Tan, X., Tian, F., Wu, L., Wu, S., Xia, Y., Zhang, D., Zhang, Z., & Zhou, M. (2018). Achieving human parity on automatic Chinese to English news translation. https://arxiv.org/abs/1803.05567

Henshaw, F., & O'Neill, E. M. (2021). Is there a place for online translators in language courses? *Panel discussion with Dr. Florencia Henshaw. IALLT (International Association for Language Learning Technology)*, Virtual Conference, June 17, 2021.

Jolley, J. R. & Maimone, L. (2015). Free online machine translation: Use and perceptions by Spanish students and instructors. In A. J. Moeller (Ed.). *Learn Languages, Explore Cultures, Transform Lives* (pp. 181–200). Minneapolis: 2015 Central States Conference on the Teaching of Foreign Languages.

Läubli, S., Sennrich, S., & Volk, M. (2018). Has machine translation achieved human parity? A case for document-level evaluation. *Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31-November 4, 2018*, pp. 4791–4796.

Loock, R. (2018). Traduction automatique et usage linguistique: une analyse de traductions anglais-français réunies en corpus. *Meta: le journal de traducteurs/Meta: Translators' Journal* 63(3), 785–805. https://doi.org/10.7202/1060173ar
Loock, R. (2019). La plus-value de la biotraduction face à la machine. *Traduire, 241*, 54–65. https://doi.org/10.4000/traduire.1848

Loock, R. (2020). No more rage against the machine: How the corpus-based identification of machine-translationese can lead to student empowerment. *The Journal of Specialised Translation, 34*, 150–170.

Loock, R., & Léchauguette, S. (2021). Machine translation literacy and undergraduate students in applied languages: report on an exploratory study, *Revista Tradumàtica. Tecnologies de la Traducció, 19*, 204–225. https://doi.org/10.5565/rev/tradumatica.28

Macken, L., Van Brussel, L., & Daems, J. (2019). NMT’s wonderland where people turn into rabbits. A study on the comprehensibility of newly invented words in NMT output. *Computational Linguistics in the Netherlands Journal, 9*, 67–80.

Moorkens, J., Kenny, D., & do Carmo, F. (Eds). (2020). Fair MT: Towards ethical, sustainable machine translation. Special issue of *Translation Spaces, 9*(1).

Nunes Vieira, L. (2020). Machine translation in the news: a framing analysis of the written press. *Translation Spaces, 9*(1), 98–122. https://doi.org/10.1075/ts.00023.nun

O'Neill, E. M. (2019). Online translator, dictionary, and search engine use among L2 students. *CALL-EJ: Computer-Assisted Language Learning–Electronic Journal, 20*(1), 154–177.

Resende, N., & Way, A. (2021). Can Google Translate rewire your L2 English processing? *Digital, 1*(1), 66–85. https://doi.org/10.3390/digital1010006

Savlodi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021). Gender bias in machine translation. *Transactions of the Association for Computational Linguistics, 9*, 845–874. https://doi.org/10.1162/tacl_a_00401

Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. *Paper presented at the 57th Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy, July 28–August 2, 2019*, pp. 3645–3650.

Toral, A., Castilho, S., Hu, K., & Way, A. (2018). Attaining the unattainable? Reassessing claims of human parity in neural machine translation. *Proceedings of the Third Conference on Machine Translation (WMT), Volume 1: Research papers*, Brussels, Belgium, October 31–November 1, 2018, pp. 113–123. https://www.aclweb.org/anthology/W18-6312/

Pym, A., & Torres-Simón, E. (2021). Is automation changing the translation profession? *International Journal of the Sociology of Language, 270*, 39–57. https://doi.org/10.1515/ijsl-2020-0015

Vanmassenhove, E., Shterionov, D. & Way, A. (2019). Lost in translation: Loss and decay of linguistic richness in machine translation. *Proceedings of Machine Translation Summit XVII v. 1: Research Track*, Dublin, Ireland, August 2019, 222–232. https://www.aclweb.org/anthology/W19-6622/

White, K. D. & Heidrich, E. (2013). Our policies, their text: German language students’ strategies with and beliefs about web-based machine translation. *Die Unterrichtspraxis, 46*(2) v. 46, 230–250. https://doi.org/10.1111/tger.10143

## Author Biodata

**Rudy Loock** is Professor of English Linguistics and Translation Studies in the Applied Languages department of the University of Lille, France, where he is in charge of a specialized translation master’s program, and affiliated with the CNRS laboratory ‘Savoirs, Textes, Language’. His research interests include corpus-based translation studies, the use of electronic corpora as translation tools, and translation teaching. He has published a number of articles and book chapters on the topic in English and French. He is currently a board member of the *European Master’s in Translation* network.

**Sophie Léchauguette** is Associate Professor of English and Translation Studies in the Applied Languages department of the University of Lille, France. She is a specialized translator and teaches
pragmatic translation in various translation master’s programs. She is affiliated with the laboratory ‘Centre d’Études en Civilisations Langues et Lettres Étrangères’ *CECILLE*. Her research interests include translation teaching in undergraduate curricula, multisemiotic translation and training in pragmatic translation. She has published a number of articles and book chapters on the topic in English and French.

**Benjamin Holt** is an associate professor of English and translation studies in the Applied Languages department of the University of Lille. He teaches specialized translation and professional language in various undergraduate and master’s programs. He is affiliated with the CNRS laboratory ‘*Savoirs, Textes, Langage*.’ His research interests include foreign language teacher education, multimodality, gesture studies and translation.