Overview of the First Workshop on Language Models for Low-Resource Languages ( L o R es LM 2025) (2025)

Abstract: The first Workshop on Language Models for Low-Resource Languages (LoResLM 2025) was held in conjunction with the 31st International Conference on Computational Linguistics (COLING 2025) in Abu Dhabi, United Arab Emirates. This workshop mainly aimed to provide a forum for researchers to share and discuss their ongoing work on language models (LMs) focusing on low-resource languages, following the recent advancements in neural language models and their linguistic biases towards high-resource languages. LoResLM 2025 attracted notable interest from the natural language processing (NLP) community, resulting in 35 accepted papers from 52 submissions. These contributions cover a broad range of low-resource languages from eight language families and 13 diverse research areas, paving the way for future possibilities and promoting linguistic inclusivity in NLP.

## introduction language models (lms) have been a long-standing research topic, originating with simple n-gram models in the 1950s @xcite . they are computational models that use the generative likelihood of word sequences to perform natural language processing (nlp) tasks @xcite . recent advancements in lms have significantly shifted towards neural language models due to their more robust capabilities @xcite . developing pre-trained neural language models/transformers is a key milestone in lm research that notably enhanced nlp performance @xcite . this breakthrough has also prompted the development of more advanced large language models (llms), such as gpt, which consist of vast numbers of parameters pre-trained on extensive text corpora, resulting in state-of-the-art natural language understanding and generation across various applications @xcite . there are approximately 7,000 spoken languages worldwide @xcite . however, most nlp research focuses on about 20 languages with high resources @xcite . for example, 63% of the papers published at acl 2008 focused on english @xcite , and even a decade later, 70% of the papers at acl 2021 were evaluated only in english @xcite . the remaining numerous languages that receive little research attention are commonly referred to as lowresource languages. these languages generally lack sufficient digital data and resources to support nlp tasks. they are also known as resource-scarce, resource-poor, less computerised, low-data, or lowdensity languages @xcite . since the capabilities of lms are primarily determined by the characteristics of their pre-trained language corpora, disparities in language resources are also evident within the models. for instance, many widely used transformer models (e.g., bert @xcite , roberta @xcite , bart @xcite , and t5 @xcite ) only support english. however, the crosslingual capabilities of transformers have paved the way for multilingual models (e.g., mbert @xcite , xlm-r @xcite , mt5 @xcite , and bloom (scao et al., 2022)), allowing low-resource languages to benefit from other languages through joint learning approaches. despite this progress, these models are typically limited to up to 100 languages due to the curse of multilingualism @xcite . in light of this challenge, developing monolingual models (e.g., sinbert for sinhala @xcite , and phobert for vietnamese (nguyen and tuan nguyen, 2020)) is another growing trend recently established to promote research in lowresource languages. there are several common factors which impede low-resource language research. one major issue is limited data availability, as the performance of most models depends heavily on the amount of training data @xcite . even recent neural lms with multilingual capabilities tend to perform poorly when pre-training data for a particular language is limited or unseen @xcite . data quality also plays a pivotal role in research outcomes, yet the absence of recommended guidelines hinders the quality of low-resource language data @xcite . additionally, the scarcity of benchmark datasets tailored for low-resource languages tends to bias most model evaluations towards highresource languages @xcite . interestingly, there are several ongoing efforts that aim to encourage research on low-resource languages and mitigate the bias in nlp approaches towards high-resource languages @xcite @xcite . we organised the first workshop on language models for low-resource languages (loreslm 2025) to further strengthen this trend. loreslm 2025 1 specifically focused on lm-based approaches for low-resource languages, inviting submissions on a broad range of topics, including creating corpora, developing benchmarks, building or adapting lms, and exploring lm applications for low-resource languages. section 2 provides a summary of the workshop contributions, highlighting language and task/research area coverage. we invite you to refer to the full papers available in the proceedings for more detailed information. ## workshop contributions loreslm 2025 received 52 submissions, including 40 long papers and 12 short papers. among these, we accepted 35 papers, including 28 long papers and seven short papers, to appear in the workshop proceedings, following the review process. we provide a detailed summary of the distribution of accepted papers across various languages and research areas below.