Optimising C hat GPT for creativity in literary translation: A case study from E nglish into D utch, C hinese, C atalan and S panish (2025)

Abstract: This study examines the variability of ChatGPT’s machine translation (MT) outputs across six different configurations in four languages, with a focus on creativity in a literary text. We evaluate GPT translations in different text granularity levels, temperature settings and prompting strategies with a Creativity Score formula. We found that prompting ChatGPT with a minimal instruction yields the best creative translations, with Translate the following text into [TG] creatively at the temperature of 1.0 outperforming other configurations and DeepL in Spanish, Dutch, and Chinese. Nonetheless, ChatGPT consistently underperforms compared to human translation (HT). All the code and data are available at Repository URL will be provided with camera-ready version.

## introduction the intersection of artificial intelligence (ai) and creativity in the domain of translation presents a fascinating and challenging field for research. even if the development of machine translation (mt) technologies, especially through the advent of large language models (llms) like chatgpt, has reshaped the landscape of the language industries, there remains a notable gap in the creative capacities of mt outputs in comparison to that of professionals @xcite . this type of translation, often applied to literary texts, requires not just the accurate conveyance of meaning but also the preservation of style, tone, and creative nuances inherent in the source text to create an effect on the reader that is not purely information driven. since new models offer a dialogic capacity, we explore in this paper the best set of variables to generate the most creative translations using chat-gpt. ## methodology in this section, we explain the source text (st) used, how the target texts (tts) were generated in the different phases of our experimentation, as well as the data annotation and analysis process. ## results the following subsections contain the results obtained in each of the phases explained in the methodology. detailed annotations per language are provided in appendix a. ## conclusions we wanted to explore chatgpt mt for the best possible setting for creativity. the results show that there is indeed variability per configuration and per language. the first observation, perhaps obvious for a translator but not so obvious for others, is that creativity is seriously affected by using chatgpt in any setting. not only is the number of css in the tts provided by all chatgpt models (but also deepl, gt and softcatalà) significantly lower than in ht, but the number of errors is also significantly higher. even the most creative setting does not come close in three out of the four languages analysed (for zh we did not have an ht reference). further, it is important to note that the ci for ht is not only higher but it might also not be representative of the overall creativity of the ht tts, since we are only analysing the solutions provided by the translators to the annotated ucps but not the entire segment where translators use other techniques, e.g. compensations, to create the desired overall effect of the text. the second observation is that less appears to be more when prompting chatgpt to output a creative translation. overall the best result is the one provided by prompt 3: "translate the following text into [tg] creatively". although this prompt still yields a very high number of errors and very modest css, it still outperforms the others in es, nl and zh while in ca even less information is needed as prompt 1: "translate the following text into [tg]" outperforms the others. these results are in line with the previous results obtained for chinese in @xcite . the different prompts have somewhat similar results across different languages, with better outputs for temperature 1.0 (2b) and with prompt 3 (3b) for es, nl and zh, although there were differences when providing chatgpt with paragraphs or the whole document and between ca and the other languages. moreover, it is interesting to see that there is a level of randomization in the output that is quite unpredictable and that requires many iterations to find the optimal solution. we wonder how this fits in a context where mt is supposed to be used to increase translator's performance. trying these different alternatives and still obtaining a sub-optimal result does not seem the best solution for practicing translators, although it is impossible to predict if some mt suggestions might spark creativity. as this case study is of an exploratory nature, there are limitations, notably, we selected a reduced number of ucps that were annotated by one single annotator, with a limited number of prompts. however, the striking differences in the performance in literary translation in comparison to what is reported in the media, i.e. singularity (translated, 2025), merits urgent attention.