Metaphors in Literary Machine Translation: Close but no cigar? (2025)

Abstract: The translation of metaphorical language presents a challenge in Natural Language Processing as a result of its complexity and variability in terms of linguistic forms, communicative functions, and cultural embeddedness. This paper investigates the performance of different state-of-the-art Machine Translation (MT) systems and Large Language Models (LLMs) in metaphor translation in literary texts (English->Dutch), examining how metaphorical language is handled by the systems and the types of errors identified by human evaluators. While commercial MT systems perform better in terms of translation quality based on automatic metrics, the human evaluation demonstrates that open-source, literary-adapted NMT systems translate metaphors equally accurately. Still, the accuracy of metaphor translation ranges between 64-80%, with lexical and meaning errors being the most prominent. Our findings indicate that metaphors remain a challenge for MT systems and adaptation to the literary domain is crucial for improving metaphor translation in literary texts.

## introduction in 2015, toral and way carried out two landmark studies on literary machine translation (litmt) challenging 'the perceived wisdom [...] that mt is of no use for the translation of literature @xcite and the claim that literature remains 'the last bastion of human translation @xcite . despite recent improvements in mt quality, they doubted whether mt would be able to tackle what has been called 'perhaps the most creative task a human translator can take on' @xcite ). yet @xcite convincingly showed that mt has potential in assisting human literary translators, especially in the translation of fiction novels between closely related languages. their best-performing system equalled professional human quality almost 20% of the time, and a human evaluation with native speakers indicated that over 60% of the translations were considered of the same or even higher quality. a small but steadily growing number of studies has been conducted in litmt for different genres and languages @xcite @xcite @xcite , showing significant quality gains of literaryadapted nmt systems over general-purpose mt. nevertheless, several challenges remain in litmt. the time is not yet ripe to admit defeat and concede mt's triumphant victory over the human translator. the gap between litmt and publishable translations is still large, with mt systems lacking in terms of adequacy, style and tone, and the translation of figurative language @xcite . one characteristic of literary texts that continues to pose difficulties is the use of metaphors, which are problematic for nlp @xcite and notoriously hard to translate, even for humans, because of their linguistic and cultural embeddedness. recently, large language models (llms) have demonstrated remarkable performance in several linguistic tasks, including mt @xcite . unlike traditional encoder-decoder models, llms have shown potential in translating long documents, in performing style transfer in a zero-shot manner and have even been tested as aids in creative processes @xcite . these new abilities lead to the questions: can llms address key challenges in litmt? how well do they perform on metaphor translation, a hallmark of creative expression? to date, only dorst (2024) and @xcite have studied metaphor in litmt, but each performed a qualitative analysis on a single text and engine. no studies have systematically com-pared how different mt systems translate different types of metaphor and whether llms offer new possibilities in addressing this persistent problem. in this paper, we investigate metaphors in litmt by analysing (i) the performance of state-of-theart mt systems in translating metaphor and ii) the kinds of errors the different systems make when translating metaphor in literary texts. our contributions are as follows: ## how do mt systems translate metaphors? the automatic evaluation ( for the form errors, the situation is slightly different: deepl and s3big make fewer form errors than the other systems. here, the contrast is quite striking between google translate (with 22 form errors) and s3big (with only 9 form errors). this suggests that deepl and s3big may be better at correctly translating multi-word metaphors such as idiomatic expressions. for example, both systems correctly translated 'keep your voice down' as 'praat niet zo hard' [lit. talk not so loud] rather than the incorrect (unidiomatic) direct translation 'houd je stem laag' produced by google translate and gpt. the translation produced by tower -'houd je stem maar eens in' -is particularly puzzling because it sounds idiomatic but is in fact meaningless, since 'inhouden' is something you can do with your breath (e.g. hold your breath) but not with your voice. the combination 'je stem inhouden' simply does not exist in dutch (but 'je adem inhouden' does). something slightly different happened with the expression 'get under his skin' (s89, g0l), where deepl, google translate and gpt all have the incorrect direct translation 'onder zijn huid kruipen' (which does not exist as a conventional metaphorical expression in dutch), while tower outputs a correct and idiomatic alternative 'van streek te maken' [to upset] and s3big outputs the idiomatic but incorrect (wrong meaning) 'overdonderen @xcite . more examples need to be collected and analysed to determine whether these patterns are consistent across larger datasets but this first exploration may suggest that as nmt and llms become better at avoiding incorrect direct translations of multi-word expressions they may start making more meaning errors (which will be harder to spot, especially for readers without access to the source text). litmt is advancing to a stage in which the number of obviously incorrect and unidiomatic translations is decreasing, and some of the metaphor translations are indistinguishable from human translation. however, the big question is whether the remaining errors -both form and meaning -affect the readers' understanding of the metaphors and their role in the narrative. if the cost for obtaining idiomatic metaphor translations is a shift in meaning is that a price we are willing to pay? in this paper, we addressed the translation of metaphor in literary mt from english into dutch by comparing different transformer-based architectures. we investigated how the different systems translate metaphors and determined what type of errors they tend to make, asking whether llms provide new opportunities in tackling this longstanding challenge in nlp. regarding the performance of state-of-the-art mt systems in metaphor translation, our conclusion is that they come close, but no cigar. the automatic evaluation showed that different types of metrics favoured different systems and no single system consistently outperformed the others. no clear evidence was found indicating that llms in their current setting outperform nmt systems in litmt and metaphor translation, at least for this language pair and type of literary content. commercial nmt systems produced the overall highest quality output, followed by closed-source llms. notably, the open-source, literature-adapted nmt system s3big remained competitive despite having been trained on significantly less data, demonstrating the effectiveness of domain adaptation. additionally, the human evaluation revealed that the accuracy of the sys-tems for metaphor translation specifically ranged between 64-80%, highlighting that metaphors remain a challenge for mt systems. a closer look at the errors identified in human evaluation revealed that most were meaning errors (i.e. lexical) rather than form errors (i.e. grammatical) and most of the errors concerned highly conventional metaphorical expressions. our current findings suggest that further research is needed to assess whether the errors made by mt -both in form and meaning -affect readers' understanding of the metaphors and the narrative. in addition, the structural similarities between english and dutch may result in more "false friend" metaphor translations, which may appear to be fluent and correct while the metaphor technically does not exist in dutch. the next phase of this project is therefore to also extend it to more languages. another continuing objective of the current project is to develop a clearer taxonomy to identify and label different types of errors and shifts in metaphor translation, especially given the difficulties in deciding where (which word) the error occurs, what type of error it is, and whether it should be counted as an error or as a creative solution. we are therefore also conducting a reader-response study that investigates how readers respond to the different mt metaphor translations that were classified as errors in our human evaluation.