The Translator’s Canvas: Using LLM s to Enhance Poetry Translation (2024)

Abstract: We explore the potential of LLMs to enhance the translation process of rhymed and non-rhymed poetry. We examine LLMs’ performance in terms of lexical variety, lexical density, and sentence length compared to human translations (HT). We also examine the models’ abilities to translate sonnets while preserving the rhyme scheme of the source text. Our findings suggest that LLMs can serve as valuable tools for literary translators, assisting with the creative process and suggesting solutions to problems that may not otherwise have been considered. However, if the paradigm is flipped, such that instead of the systems being as tools by human translators, humans are used to post-edit the outputs to a standard comparable to the published translations, the amount of work required to complete the post-editing stage may outweigh any benefits assocaiated with using machine translation in the first place.

## introduction the translation of poetry has long been a contentious issue in the field of literary translation @xcite . the debate stems from the challenges inherent to translating poetry, which, depending on the specific poetic form in evidence, may require a delicate balancing act of content, style, tone, various types of phonetic devices, such as rhyme. differences in language and poetic tradition may necessitate compromises and creative solutions with many competing constraints making the translation of poetry a highly complex activity. literary translation has historically been regarded as the "last bastion of human translation" @xcite , and poetry translation could be thought of as the most extreme example of this phenomenon. however, recent advances in the widespread availability of large language models (llms) have shifted the conversation to ask in what ways human translators might make use of electronic tools in the negotiating of literary translation's stylistic and technical complexities. much of this work to date has focused entirely on prose, and while advancements have been substantial in this respect, much less attention has fallen onto poetry in general, and formal poetry in particular. thus, formal poetry, simultaneously combining as it does many of the stylistic features that are known to complicate machine translation, remains an extreme challenge. nonetheless, the emergence of web-based llms offers new opportunities. these models, such as chatgpt and google gemini, enable the customisation of translated outputs through prompt engineering @xcite , whereby users can specify in detail aspects of a text to focus on, change or omit. this capacity sets llms apart from the web-based neural machine translation (nmt) systems which have been the mainstay of the machine translation systems (mt) widely available to literary translators for the past decade or so. generally speaking, such nmts are limited to one or a small number of similar outputs for any given input, with little or no functionality to tailor the translation process around, for example, register, addressee, style, or tone. when it comes to complex operations such as rhyming or counting syllables, specialised systems would be required. the focus in this discussion falls predominantly on web-based systems which are either free to use or financially accessible, because the funding associated with any given literary translation project is so limited (des associations de traducteurs littéraires , ceatl). thus, it is unrealistic to imagine literary translators being in a position to invest in expensive bespoke tools which may require training for the facilitation of their work, and therefore, the most realistic use cases when it comes to literary translation centre on systems which are easily and cheaply accessible. for these reasons, the widespread emergence of llms such as chatgpt and google gemini, with their free entry points, represents a significant opportunity for analysing whether such systems might be useful tools for literary translators. here there is an important distinction to be made between literary machine translation (lmt) and computer-assisted literary translation (calt). on the one hand, lmt conceptualises the machine at the centre of the process of producing translated outputs, generally with one or more human beings supporting its work through pre-editing or post-editing. in this view, quality assessment reaches for the ultimate goal of producing outputs of the same standard as human translators @xcite . on the other hand, calt conceptualises the human translator as the primary agent, who makes use of the machine as a tool. in this view, the human translators may use the machine to translate only individual or isolated parts of the text, may use iteration to produce multiple versions of the same passage, and may wholly disregard the outputs of the machine if a better solution is found elsewhere. the focus of each perspective is reversed. in lmt, the goal is to maximise the quality of the output to minimise editing work by the human. in calt, the goal is to support human translators in their own idiosyncratic workflows, identifying and trailing possible solutions to translation challenges, and further stimulating human translators' creativity. thus, whereas in an lmt workflow, producing multiple outputs of the same text, may be perceived as wasteful, because this would imply that each output would also need to be post-edited, in the context of calt, producing multiple outputs of the same text or text fragment could perceivable be useful for a human translator who may use the machine's outputs more as inspiration than as something approaching a product to be refined. while there is evidence supporting llms in the translation of prose works, especially novels @xcite , their impact on poetry translation remains under-explored. thus, it is unclear how machine translations produced with the help of llms compare stylistically to human translations. asking about these comparisons is fundamental to assessing whether and how llms might be made useful by practising translators of literature, and especially poetry. to address this question, our initial step involves extracting and examining linguistic features at both the syntactic and lexical level from poems, as well as from translations of them produced by humans and by llms in portuguese and spanish. ## related work the methodological approach used here is one which analyses and compares the stylistic features of translated text using natural language processing (nlp) techniques. in each case, the candidate translations by each of the llms is compared side by side with previously published human translations of the same text. this approach is grounded in a body of literature that has developed since the 1980s to explore the distinctive stylistic characteristics of translated texts primarily on statistical terms. this literature is theoretically rooted in toury's translation norms @xcite which posits that translation is a culturally-bound phenomenon which functions different in different human contexts, and baker's translation universals @xcite , which identify aspects of texts which anecdotal experience can allow us to identify translated from non-translated work. the approach responds to these two somewhat subjective theoretical constructs with corpus linguistics and nlp methods, which allow for the results to be statistics-based, and repeatable @xcite @xcite . the research conducted with these methods, has consistently shown that translated text does indeed tend to exhibit simpler syntax and less varied vocabulary than non-translated text @xcite @xcite . this phenomenon, often referred to as translationese in the literature, is frequently associated with lower quality text, characterized by foreign-sounding and awkward wording and structure @xcite . with the advent of mt systems, attention has shifted towards the stylistic features of machine-translated and post-edited texts @xcite @xcite . this research has gone on to show significant differences in style and content richness between human-translated text and machine-translated text, especially that produced by nmt systems @xcite . recent research has begun evaluating the translation capabilities of large language models (llms) in both literary and technical texts, often in comparison with nmt systems @xcite @xcite . preliminary studies @xcite have explored llms' ability to preserve the rhyme schemes in poetry translation, focusing on qualitative analysis of a single poem. however, there is a lack of research on how the range of stylistic features which come together to embody poetic texts are managed by llms, and how or whether llms might be made useful to practising human translators of poetry. this study represents an initial attempt to address this gap in the literature. given the exploratory nature of the present study, the focus is on a limited corpus to provide a preliminary assessment of the the place these tools could have in a poetry translation workflow, setting the stage for more extensive research in the future.