Literary translation as a three-stage process: machine translation, post-editing and revision (2022)

Abstract: This study focuses on English-Dutch literary translations that were created in a professional environment using an MT-enhanced workflow consisting of a three-stage process of automatic translation followed by post-editing and (mainly) monolingual revision. We compare the three successive versions of the target texts. We used different automatic metrics to measure the (dis)similarity between the consecutive versions and analyzed the linguistic characteristics of the three translation variants. Additionally, on a subset of 200 segments, we manually annotated all errors in the machine translation output and classified the different editing actions that were carried out. The results show that more editing occurred during revision than during post-editing and that the types of editing actions were different.

## introduction with the current quality of neural machine translation (nmt) systems, the question arises whether post-editing nmt output is a viable alternative to human translation for real large-scale translation tasks. in this paper, we present the results of a case study on literary translations. we collaborated with nuanxed, a book translation company, which uses an mt-enhanced workflow consisting of a three-stage process of automatic translation followed by post-editing and revision. in this case study, we compare three successive versions of the target texts as they proceed through the translation process: the machine translation, the post-edited and the (mainly) monolingually revised translation. we used different automatic metrics to measure the (dis)similarity between the consecutive versions and to analyze the linguistic characteristics of the three translation variants. to assess the quality of the mt output and to get an insight into the editing actions that were carried out, a fine-grained manual annotation was carried out on a subset of 200 segments. ## related research although employing machine translation (mt) for more creative text types such as literature may not seem to be a natural fit, several researchers looked into the feasibility of using mt for literary texts, first with statistical @xcite and later with neural machine translation systems @xcite @xcite . to assess the usefulness of mt for literary texts, researchers often compare raw (unedited) machine translations of literary texts with their humantranslated (ht) counterparts. three successive studies were conducted to assess the quality of generic nmt systems for english-dutch literary texts, the language pair we also focus on in this study @xcite @xcite . according to these studies, the main issues found in literary nmt are different types of mistranslations, coherence issues, and style & register problems. the percentage of nmt sentences that were free of errors varied and averages ranged from 44% to 25% in different studies, with a notable exception of the nmt version of jane austen's sense and sensibility in which only 5% of all machine-translated sentences were errorfree. it thus seems that nmt quality is highly de-pendent on the source text and that some literary texts are more challenging for automatic translation systems than others. when comparing linguistic characteristics of nmt and ht, the machine translations were less lexically rich, showed a lower level of lexical and semantic cohesion and tended to follow the structures of the source sentences more closely, whereas the human translations showed the ability to deviate from the source structure @xcite . it is thus clear that in order to obtain high-quality literary translations, human intervention in the form of a post-editing (pe) step is needed. daems and colleagues investigated whether post-edited mt output differs from ht in english-to-dutch texts (2017), and called this (dis)similarity between pe and ht 'post-editese'. the authors did not find proof of this. neither humans nor computer systems were able to distinguish between the two types of translation, although the authors note that this may be due to a rather limited dataset size. they considered features such as average word and sentence length, average tf-idf, perplexity, type-token ratio, number of verb phrases/passives, parse tree depth, and so on. working with different language combinations and architectures, @xcite came to a different conclusion. he found that pe is indeed notably different from ht in terms of a limited set of features, namely lower lexical variety (typetoken ratio) and density (content words ratio), sentence length inference of st, and pos sequence perplexity. it must be noted however, that not only the language pairs differed in the studies of @xcite and @xcite , and hence the mt quality, but also the proficiency level and the degree of postediting that was requested (light or full). it is thus difficult to draw conclusions about the existence of post-editese. neither daems and colleagues nor toral investigated post-editese in literary texts. @xcite , however, found some evidence for post-editese in literary translation of english into brazilian portuguese but note that such observations depend on the literary genre. statistical differences between ht and pe were found, especially in the thriller genre (the girl on the train; tgott) and only barely in children's literature (alice's adventures in wonderland; aw), which is explained by the emphasis on the author's figurative style in the latter book. post-editese effects for lexical density (simplification), length ratio (text length of pe vs ht; explicitation), personal pronoun ratio (explicitation), and convergence (translated texts are more similar to each other than original texts are to each other) (partially) were found for tgott, but only evidence for convergence was discovered in aw. guerberof-arenas and toral (2020) focused on creativity, one of the distinguishing features of literary texts. they analyzed both creativity and acceptability in mt, pe and ht texts. the translation and post-edited version were created by two professional translators specialized in literary translation. to quantify acceptability they counted the number of errors in the different translations. interestingly, they found that the ht translations contained slightly more errors than the pe translations, with ht having lowest number fluency errors and pe having the lowest number of accuracy errors. to measure translational creativity they selected 48 english source sentences that contained units of high creativity potential (in which translators most likely depart from the source text structure): metaphorical expressions, imagery and abstraction, idioms, comparisons, verbal phrases or complex syntactic structures. they quantified creativity by investigating creative shifts, which can be defined as "abstracting, modifying or concretising source text ideas in the target text" @xcite . when comparing the three types of shifts in the ht and pe condition, no major differences were found for abstractness and modification, but the ht contained more instances of concretisation. the work of @xcite mentioned above built on earlier work on 'translationese' @xcite . in the field of translation studies, it is generally accepted that a translated text is different from an original text in the same language, almost as if it is a genre on its own. @xcite discusses six "universal features of translation" that may mark translated texts: explicitness, disambiguation and simplification, a focus on grammaticality (especially in interpreting), avoiding repetitions by omission or rewording, exaggeration of target language features, and finally unexpected distributions of certain language features with respect to the source text (st) and original texts in the target language. this phenomenon where translation is considered different from original text is often referred to as 'transla-tionese', and researchers have investigated its existence, both via human perception and computer models. @xcite , however, made an interesting point by suggesting that some of these translationese features might also be the consequence of the editorial intervention subsequent to translation. evidence for features commonly denoted as translationese such as increased explicitness, simplification and normalisation were also found in a parallel corpus of monolingual edited texts and their unedited counterparts. it thus seems that translation and linguistic editing share certain similarities. in the publishing sector, it is quite common that many actors play a role in the production of a translation. for example, @xcite explain that in slovenia language revisors correct the grammar and style of translations, usually without having access to the source texts. they may change the text's structure, syntax and word order and replace words and phrases to make the text more suitable. different terms are used to refer to this process: linguistic revision/editing, copyediting and translation revision. mossop states that both editing and revision "involve checking linguistic correctness as well as the suitability of a text's style for its future readers and for the use they will make of it" @xcite . translation revision can be considered the broader term as it also comprises a bilingual component, although different revision procedures exist @xcite and the process can be predominantly monolingual (the revisor focuses on the target text and only refers back to the source text if a passage is problematic) or bilingual (the revisor systematically compares the source and target text). ## discussion in this paper, we examined the possibility of using an mt-enhanced translation workflow for the translation of literary texts in a real-life professional translation scenario. we examined three different versions of the target texts as they proceed through the translation process: the mt output, the post-edited version and the revised translation. deepl was used as mt engine to translate an english novel into dutch. mt quality was in line with expectations with 27.5% error-free sentences. the three main error types were various kinds of mistranslations, disfluent sentence constructions and different types of spelling and punctuation problems. deepl failed to correctly copy quotation marks from source to target, a problem that can potentially be fixed by applying a number of post-processing rules. mistranslations and disfluent constructions have been reported in earlier research as the main error types and require more attention from the post-editor. forty-four percent of all post-editing actions were corrections of mt errors, 24% of all postedits were preferential changes, 9% of all postedits were labelled as 'undesirable'. apart from adding missing punctuation marks, the post-editor mainly carried out lexico-semantic changes (replacing words with better alternatives or synonyms) and stylistic operations (restructuring mt fragments or coming up with shorter translation solutions). most mt errors were solved in the post-editing step. only 5.6% of all editing actions during revision were related to mt errors; another 5.5% were corrections of problems introduced during post-editing. the majority of the revisor's edits (86%) were thus preferential in nature. the revisor made slightly more edits than the post-editor. the revisor, just like the post-editor, mainly made lexico-semantic changes, but the subcategories were different. the revisor often made information and relations that the reader might be able to infer from the context explicit as can be seen from subcategories 'explicitation' and 'coherence marker' in figure 4 . the revisor also made a lot of stylistic changes and restructured fragments and even split sentences in 23% of all segments. post-editing and revision can be considered two different cognitive processes. post-editing is by nature a bilingual process in which the post-editor can be primed both by the mt suggestion and the source segment. moreover, as the post-editor worked in a traditional cat tool, in which the text is segmented at sentence level, it might be more difficult to focus on the flow of the target text. revision was mainly a monolingual process, carried out in microsoft word, in which it is easier to focus on the translated text as a standalone product. it is remarkable, however, that the revisor carried out many edits that fall within two subcategories that are often considered as 'translationese', e.g. increased explictness (subcategories 'explicitation' and 'coherence marker') and simplification (subcategory 'split sentence'). we consider this as an indication that monolingual editing and translation indeed share certain similarities as @xcite suggested. the automatic evaluation confirmed that more editing took place in the revision step than in the post-editing step. the degree of similarity between the mt, the pe and the rev version was assessed based on the amount of editing, and semantic and syntactic similarity measures. all measures confirmed that the degree of similarity between mt and pe was higher than the degree of similarity between pe and rev. the lowest similarity scores were obtained when comparing the mt with the revised version. as a side note we would like to point out that in mt research it is common practice to use automatic evaluation metrics to compare the mt output with an independent reference translation, often without knowing how this reference translation was created. it might as well be that the reference translation being used is the output of a two-stage process of human translation followed by revision, which, depending on the amount of editing that took place, may have altered the human translation to a large extent. another feature that has been widely studied in previous research is lexical richness. in this study, we quantified lexical richness by means of ttr, mass index and average word translation entropy. some results were inconclusive (higher ttr values, but lower or similar mass index values). average word translation entropy showed a clearer picture, with the revised version having the highest values. it thus seems that the revised version exhibits many characteristics that have been attributed to human translations: a higher degree of explicitation and simplification, more lexical variety and translations that deviate more from the source structure (compared to mt). this study, however, cannot provide a conclusive answer to the question of whether the implemented threestage process of automatic translation followed by post-editing and revision is a viable alternative to human translation followed by revision. this can only be measured by means of comparative translation reception studies in which the reading (or listening) experience is measured. one of the major limitations of this study is that we only had data of one post-editor and one revisor. moreover, the post-editor and the revisor had different experience levels, with the posteditor having less experience in the literary domain. studying the edits of two different persons most probably changes the distribution of the edits. it would therefore be interesting to replicate this study with more post-editors and more revisors and on different language pairs. in future work we also intend to zoom in on the sentences with high creativity potential as was done by guerberof-arenas and toral (2020) and examine in more detail the creative shifts in the post-edited and revised version.