Bringing L adin to FLORES + (2025)

Abstract: Recent advances in neural machine translation (NMT) have opened new possibilities for developing translation systems also for smaller, so-called low-resource, languages. The rise of large language models (LLMs) has further revolutionized machine translation by enabling more flexible and context-aware generation. However, many challenges remain for low-resource languages, and the availability of high-quality, validated test data is essential to support meaningful development, evaluation, and comparison of translation systems. In this work, we present an extension of the FLORES+ dataset for two Ladin variants, Val Badia and Gherdëina, as a submission to the Open Language Data Initiative Shared Task 2025. To complement existing resources, we additionally release two parallel datasets for Gherdëina–Val Badia and Gherdëina–Italian. We validate these datasets by evaluating state-of-the-art LLMs and NMT systems on this test data, both with and without leveraging the newly released parallel data for fine-tuning and prompting. The results highlight the considerable potential for improving translation quality in Ladin, while also underscoring the need for further research and resource development, for which this contribution provides a basis.

## introduction in recent years, the field of machine translation (mt) and natural language processing has advanced rapidly and the transformer-based models played a key role in this development @xcite . this paradigm shift has enabled the development of high-quality mt systems for major languages as well as the adaptation of such systems to low-resource languages by leveraging pre-trained knowledge @xcite . earlier rulebased and statistical mt approaches lacked this capability as they depended on large, clean, and domain-specific parallel corpora, thus limiting the availability of these technologies for most lowresource languages. the release of large-scale multilingual systems, such as no language left behind (nllb) @xcite , has demonstrated that scalable and accurate translation is possible even for smaller languages. essentially, developing machine translation systems has actually become easier as it is no longer just about language expertise. instead, the generalizability capabilities of language models are being exploited, and the limitations now lie more in providing training data and infrastructure. however, its development depends on reliable and extensive test data, which is essential for consistent evaluation, model comparison, and tracking progress over time. the flores+ dataset provides such a benchmark and covers more than 200 languages @xcite . it is widely used in research on low-resource languages because of its broad language coverage and its ability to support evaluation across more than 19,900 language pairs. with this in mind, we release the flores+ translations for the ladin language. ladin is a rhaeto-romance language that originated from vulgar latin during the roman conquest of the alps. it evolved in isolated alpine valleys, leading to the diverse regional varieties spoken today in northern italy @xcite . as a result of the fragmented development over centuries there is no single, unified spoken variety for ladin that is characterised by its internal linguistic diversity, with five main regional variants depicted in figure 1 : val badia, gherdëina, fassa, fodom, and anpezo. in this work, we extend flores+ with translations for val badia and gherdëina. both variants are spoken in south tyrol and together representing the largest portion of ladin speakers. the dataset translation was carried out by professionals at the ladin cultural institute "micurá de rü", the primary institution dedicated to preserving and supporting the development of the language. moreover, we release parallel datasets for val badia-gherdëina and gherdëina-italian that can be used for fine-tuning or retrieval-augmented prompting for this language. these two datasets enables to provide an overview of the performance of nllb and state-of-the-art large language models (llms), including gpt-3.5, gpt-4o, llama-3.3, and deepseek-r1 on ladin (val badia and gherdëina). in summary, the contributions of our work are: with this, we aim to increase the visibility of ladin within the mt research community, highlight the ongoing need for focused research on this language, and provide resources that empower researchers and developers worldwide to advance ladin machine translation. ## the ladin language ladin should not be confused with ladino (lad), a judeo-spanish language. this confusion is common, especially in italian where ladino can refer to both; to avoid ambiguity, the term ladino delle dolomiti (dolomite ladin) is often used. ladin is identified by the iso 639-3 code lld. for a comprehensive treatment of the language's history, dialectal variation, and sociolinguistic context, we refer the reader to @xcite ; videsott et al. (2020); bauer (2022 @xcite . for a recent contribution discussing the state of the ladin language in the dolomite region, we refer to @xcite ; colcuc (2024). ## data collection the translation of the flores+ dataset into ladin was carried out in close collaboration with the ladin cultural institute "micurá de rü" foot_10 . in this section, we provide an overview of the translators involved, the tools used, the procedures followed for the different ladin variants, and the quality assurance measures implemented. ## experimental validation in this section, we provide an overview of the results achieved with a nmt system and different state-of-the-art llms on the manually created flores+ translations. the aim is to assess the current capabilities of these models in translating between val badia, gherdëina and italian, both in off-the-shelf zero-shot settings and using the accompanying parallel data as fine-tuning training data for the nmt system and as retrieval data for the llms. moreover, this serves as validation of the quality of the submitted datasets. training and retrieval corpora for italian-val badia, a dataset is already available foot_11 consisting of word usage example sentences extracted from the italiano-ladin val badia dictionary @xcite . to create training and retrieval resources for gherdëina, we extracted the word usage example sentences contained in the italiano-ladin gherdëina dictionary @xcite . since the two dictionaries are based on a common foundation and were largely coordinated with each other, they contain many common example sentences. this overlap allowed us to construct a val badia-gherdëina parallel dataset by aligning identical italian sentences. this yielded the following datasets for training (for nllb) and retrieval (for llms): note that, given their purpose of illustrating word usage, the sentences in these corpora are relatively short and simple, with an average length of approximately 25 characters. we make both the gherdëina-italian foot_12 and val badia-gherdëina foot_13 datasets publicly available under a cc by-nc-sa 4.0 foot_14 license. ## results interestingly, concerning the fine-tuned nllb model, the relatively simple training datasets proved to be effective, especially considering that the benchmark data is substantially more complex. despite the brevity and simplicity of the training samples, the model generated complete translations of the test samples. one peculiarity we observed is that the generated translations usually start with lowercase letters, reflecting the format of the training examples. another limitation is that the model struggles with morphological variations: since most training examples contained words in their infinitive form (e.g., "i go home" and never "he went home"), the model is unable to generate correct inflections or conjugations. llms are less influenced by the simple translation examples and do not "compromise" their fluency when translating into italian. for example, they also use the correct capitalization at the beginning of the sentence. when translating from italian into ladin, however, llms often lack the appropriate vocabulary, whereas the fine-tuned nllb model adapts the vocabulary more effectively (see from these results, three main observations can be drawn: (i) current llms exhibit limited coverage of ladin variants, with translation into ladin remaining a clear challenge; (ii) incorporating the parallel data released in this work yields substantial improvements in translation quality across models, but limitations remain in fluency and morphological variation due to the simplicity of the training examples; (iii) the relative advantage of the systems depends on the translation direction: when translating from a low-resource to a high-resource language, llms enhanced with retrieval-augmented generation achieve the best results, whereas for high-to-low-resource translation, the fine-tuned nllb model performs better. this conclusion is further supported by a significantly higher proportion of valid words in its generated translations. the incorporation additional data (e.g. through back-translation) would yield better results; however, the primary focus here is on validating the quality of the provided datasets. overall, these findings underscore both the potential and the necessity of advancing machine translation research for ladin, as well as the value of the datasets we contribute. ## conclusion in this work, we present our submission to the oldi shared task 2025, providing flores+ translations for ladin (val badia and gherdëina) and provide an evaluation of several llms and the nllb model on this test set, covering six translation directions between val badia, gherdëina, and italian. our results show significant performance gains from the additional parallel datasets released with this work, validating the quality of the datasets and highlighting a promising direction for future research. beyond fine-tuning neural mt models or relying on basic bm25-retrieval, our datasets opens the door to more advanced retrieval-augmented prompting strategies, where semantically or syntactically similar sentence pairs are selected to guide translations @xcite @xcite . a central question going forward is whether such carefully designed retrieval and prompting methods could not only provide a more lightweight alternative to fine-tuning for low-resource languages like ladin, but in some cases even surpass it in translation quality. it would be highly valuable if, in future work, this effort could be extended to the anpezo, fassa, and fodom ladin variants, enabling their inclusion in the flores+ dataset and thus fully representing the ladin language and accurately reflecting its internal diversity. we hope this work will inspire and encourage further research on ladin in machine translation, helping to bring the language into sharper focus within the mt community.