Synthetic Fluency: Hallucinations, Confabulations, and the Creation of I rish W ords in LLM -Generated Translations (2025)

Abstract: This study examines hallucinations in Large Language Model (LLM) translations into Irish, specifically focusing on instances where the models generate novel, non-existent words. We classify these hallucinations within verb and noun categories, identifying six distinct patterns among the latter. Additionally, we analyse whether these hallucinations adhere to Irish morphological rules and what linguistic tendencies they exhibit. Our findings show that while both GPT-4.o and GPT-4.o Mini produce similar types of hallucinations, the Mini model generates them at a significantly higher frequency. Beyond classification, the discussion raises speculative questions about the implications of these hallucinations for the Irish language. Rather than seeking definitive answers, we offer food for thought regarding the increasing use of LLMs and their potential role in shaping Irish vocabulary and linguistic evolution. We aim to prompt discussion on how such technologies might influence language over time, particularly in the context of low-resource, morphologically rich languages.

## introduction since the emergence of neural machine translation (mt), hallucinations have been recognised as a significant challenge in the field @xcite . llms also hallucinate, and despite efforts to mitigate this, hallucinations remain common-especially in low-resource settings @xcite . hallucinations produced by llms are claimed to be "qualitatively different from those of conventional translation models" which include "off target translations, overgeneration, and even failed attempts to translate" @xcite . this study focuses on specific types of hallucinations, namely instances where the system invents new words during translation. our goals are to identify which word classes are affected when open llms generate hallucinations in a low-resource language like irish (gaeilge) and to assess whether these hallucinated words follow irish linguistic rules or diverge from established conventions. to the best of our knowledge, this is the first study to examine the morphology of hallucinations generated when translating into irish. ## analysing hallucinations in irish as noted previously, some characteristics of the irish language, such as the heavy reliance on the use of suffixes and prefixes, and the great number of compound words, pose a great challenge for automatic translation. we observed a significant number of hallucinations related to verbs and nouns, adverbs and with fewer involving adjectives. due to space and time constraints, we focus on hallucinations related to verbs and nouns. the number of invented hallucinated words is greater for nouns, with only a few instances for verbs. the mini model shows a greater number of invented hallucinated words in comparison with gpt4, showing a rate of 2.14 hallucinations of this type, against 0.86 hallucinations for the latter. this is an expected result regarding the model's performance, since mini is a smaller and less robust version of gpt4. smaller models generally have fewer parameters, which can impact their ability to accurately handle complex linguistic phenomena, such as irish morphology and inflection. nonetheless, since our objective is not to compare the models' outputs but rather to analyse the patterns in which ## discussion and conclusions this study examined the types of hallucinations involving the creation of new words in llmgenerated translations into irish and evaluated whether these hallucinations adhered to irish linguistic rules, and therefore classified as confabulations. our findings indicate that both gpt4 and mini exhibit similar patterns of word invention, though the latter produces hallucinated words at a significantly higher frequency. while both models demonstrate a tendency to confabulate, that is, apply irish morphological rules to these hallucinated words, gpt4 adheres to these rules more consistently (71%) than the mini model (40%) (tables 3 and 6). this difference likely reflects the mini model's smaller size and reduced robustness. nonetheless, both models produce plausible but non-existent lexical items that raise intriguing ques-tions about their potential influence of confabulations on the irish language. many of the confabulations resemble patterns made by learners of irish, such as code switching and borrowings. this suggests that the models might not be generating entirely arbitrary forms but are instead applying irish word formation rules in a way that mirrors natural language learning processes. these confabulation patterns are particularly relevant in the context of what fhlannchadha and @xcite describe as a 'posttraditional variety of irish'-a variety adopted by non-native speakers who do not align with any particular dialect of irish. the authors note that established ideologies rooted in native and traditional models of irish are being disrupted by new speakers, creating a notable tension between linguistic groups in the era of language revitalisation. they caution that the expansion of post-traditional irish could lead to the erosion of crucial aspects of the language, particularly in lexicon and grammar. similar concerns arise in other morphologically rich, low-resource languages, such as scottish gaelic, and welsh, where language change and revitalisation efforts interact with evolving speaker communities. in this light, llm-generated confabulations raise further questions about the role of ai in reinforcing or reshaping these dynamics across such languages. but what does it mean when an ai model exhibits patterns akin to human learners? could these errors, if encountered frequently in machinegenerated content, influence the way irish is written or even spoken over time? two particularly noteworthy categories of confabulations observed in this study, which we term 'lazy gaelicisation' and 'good confabulation', involve the adaptation of english words into irish-like phonetics, often by modifying their spelling to align with irish orthographic rules. this phenomenon is not exclusive to llms; similar strategies have been observed among irish speakers themselves. the phonetic adaptation of english words into irish structures has long been a feature of the language, seen both in historical borrowings and in contemporary informal speech. does this suggest that such hallucinations are merely an extension of a natural linguistic process? or should they be viewed as problematic, reinforcing patterns of language shift rather than supporting authentic irish usage? models invented words that follow morphological rules when no official translation is available ('good confabulations'). the introduction of novel, non-standard words could be seen as either a sign of language erosion or a potential source of linguistic innovation. while some have found the replacement of existing irish words with english-derived forms as a form of 'detrimental change' @xcite , others see partial or non-standard irish as a step toward broader engagement. if llmgenerated forms gain traction, could they help fill lexical gaps in technical domains where irish terminology is scarce? or would they risk further undermining existing irish vocabulary? these are not straightforward questions, and rather than offering definitive answers, they highlight the need for continued observation and discussion. finally, it is important to highlight the specific context in which these hallucinations and confabulations occur. in our study, most invented hallucinated words appeared in technical and specialised domains, where even fluent speakers may struggle with terminology. in more general texts, where irish has a more established lexicon, the models produced fewer invented words, although overall grammatical accuracy and fluency remained an issue. this suggests that while hallucinations in llm-generated translations may be concerning in certain contexts, their broader impact on irish will likely depend on how these models are used and integrated into real-world workflows. future research should explore these issues further with larger datasets and more extensive replication resources, particularly regarding the long-term implications of llm-assisted translation for minority languages. moreover, future work should look into how speakers perceive and react to these hallucinations and confabulations. nonetheless, this work serves as a foundation for further investigations into the implications of llm errors in morphologically rich, low-resource languages. our goal is to encourage discussion on the long-term impact of these technologies on language, especially in the case of low-resource, morphologically rich languages.