Vuyko Mistral: Adapting LLM s for Low-Resource Dialectal Translation (2025)

Abstract: In this paper we introduce the first effort to adapt large language models (LLMs) to the Ukrainian dialect (in our case Hutsul), a low-resource and morphologically complex dialect spoken in the Carpathian Highlands. We created a parallel corpus of 9852 dialect-to-standard Ukrainian sentence pairs and a dictionary of 7320 dialectal word mappings. We also addressed data shortage by proposing an advanced Retrieval-Augmented Generation (RAG) pipeline to generate synthetic parallel translation pairs, expanding the corpus with 52142 examples. We have fine-tuned multiple open-source LLMs using LoRA and evaluated them on a standard-to-dialect translation task, also comparing with few-shot GPT-4o translation. In the absence of human annotators, we adopt a multi-metric evaluation strategy combining BLEU, chrF++, TER, and LLM-based judgment (GPT-4o). The results show that even small(7B) finetuned models outperform zero-shot baselines such as GPT-4o across both automatic and LLM-evaluated metrics. All data, models, and code are publicly released at: https://github.com/woters/vuyko-hutsul.

## introduction despite recent advances in large language models (llms), most research and applications remain centered on high-resource languages and their standard variants @xcite . this imbalance has significant consequences for linguistic diversity, particularly for underrepresented dialects that lack sufficient textual resources and standardized orthographies @xcite . despite being an integral part of the linguistic identity of many communities, dialects are often excluded from nlp tools and research, limiting their accessibility and risking further marginalization and extinction @xcite . language technologies and especially llms are playing a growing role in the preservation of en-dangered and underrepresented languages. while much attention has focused on major indigenous languages (e.g., māori, quechua, inuktitut) @xcite , dialects of national languages are often overlooked despite facing similar pressures of attrition and assimilation. dialectal variants, particularly in post-soviet contexts, often carry suppressed cultural identities that are not reflected in the standard language. these dialects are not only linguistically rich but also culturally vital and deserve computational attention. ukrainian, a language low in resources according to global standards itself @xcite , exhibits rich internal variation, with dialects such as hutsul, boyko and lemko foot_0 preserving unique phonetic, lexical and grammatical characteristics. among these, the hutsul dialect, spoken in the carpathian mountains, is one of the most linguistically distinct and has the most written sources. from the culture standpoint, hutsul dialect has a great significance as it encapsulates traditions, folklore, and a unique worldview, playing a central role in community identity. however, the lack of digitized corpora, dictionaries, and processing tools makes it practically invisible to modern llms. here are some of the linguistic characteristics of hutsul dialect: we frame our task as standard-to-dialect translation, in which model has to take standard ukrainian as input and produce grammatically correct (or as close as possible) hutsul dialect. our models show that it is feasible to address such translation with limited parallel data and targeted augmentation strategies. to our knowledge, this is the first work that tries to adapt llms to a ukrainian dialect and among the few globally addressing dialect-to-standard generation using synthetic augmentation. ## fine-tuning to adapt large language models (llms) to the ukrainian-to-hutsul translation task, we used parameter-efficient fine-tuning using lora @xcite . we fine-tuned two state-of-the-art open-source models in the 7b-13b parameter range (as we considered our training resources and that the model should be not too big to be able to run locally): we also considered other multilingual models such as bloomz (7.1b) 14 and nllb-200 (3.3b) foot_11 , which offer extensive language coverage. however, these models either underperformed on general language modeling tasks or lacked strong generation quality compared to selected models. recent benchmarks demonstrate that mistral-7b-instruct-v0.3 foot_12 matches or surpasses larger models in translation tasks, particularly in low-resource and instruction-tuned settings @xcite .