Serving the Underserved: Leveraging BARTB ahnar Language Model for Bahnaric- V ietnamese Translation (2025)

Abstract: The Bahnar people, one of Vietnam’s ethnic minorities, represent an underserved community with limited access to modern technologies. Developing an effective Bahnaric-Vietnamese translation system is essential for fostering linguistic exchange, preserving cultural heritage, and empowering local communities by bridging communication barriers. With advancements in Artificial Intelligence (AI), Neural Machine Translation (NMT) has achieved remarkable success across various language pairs. However, the low-resource nature of Bahnaric, characterized by data scarcity, vocabulary constraints, and the lack of parallel corpora, poses significant challenges to building an accurate and efficient translation system. To address these challenges, we propose a novel hybrid architecture for Bahnaric-Vietnamese translation, with BARTBahnar as its core language model. BARTBahnar is developed by continually training a pre-trained Vietnamese model, BARTPho, on augmented monolingual Bahnaric data, followed by fine-tuning on bilingual datasets. This transfer learning approach reduces training costs while effectively capturing linguistic similarities between the two languages. Additionally, we implement advanced data augmentation techniques to enrich and diversify training data, further enhancing BARTBahnar’s robustness and translation accuracy. Beyond leveraging the language model, our hybrid system integrates rule-based and statistical methods to improve translation quality. Experimental results show substantial improvements on bilingual Bahnaric-Vietnamese datasets, validating the effectiveness of our approach for low-resource translation. To support further research, we open-source our code and related materials at https://github.com/ura-hcmut/BARTBahnar.

## introduction the bahnar people, one of vietnam's 54 ethnic minorities, account for approximately 0.3% of the country's population. as one of the larger minority groups, they possess a rich cultural heritage reflected in unique traditions, festivals, clothing, cuisine, and, most notably, their distinct bahnaric languages @xcite . this linguistic diversity is a cornerstone of their identity, necessitating dedicated efforts for preservation and promotion. recognizing this, the vietnamese government has implemented various policies to safeguard the cultural and linguistic heritage of ethnic minorities, including the bahnar people. language preservation plays a pivotal role in maintaining the identity of ethnic groups worldwide. facilitating linguistic interaction between the bahnaric and vietnamese-speaking communities is essential for fostering cultural exchange, mutual understanding, and the preservation of minority identities. thus, developing an efficient bahnaric-vietnamese machine translation system would significantly enhance communication, granting vietnamese speakers access to the wealth of bahnaric cultural texts while enabling deeper cross-cultural interactions. the rapid advancements in artificial intelligence, particularly in neural machine translation (nmt), have significantly improved translation quality across various language pairs @xcite . the introduction of the transformer architecture @xcite and subsequent developments in large language models (llms) have revolutionized natural language processing (nlp) applications, including nmt @xcite . transformer-based models can be broadly categorized into three primary architectures, namely encoder-only, decoder-only, and encoder-decoder. the encoder-only type is primarily designed for powerful understanding tasks, making it unsuitable for non-trivial applications such as nmt. meanwhile, the decoder-only architecture excels in text generation but requires large-scale training datasets and lacks explicit encoder support for source language comprehension, making it sub-optimal for nmt @xcite . additionally, decoder-only models rely on autoregressive generation, which demands substantial computational resources and extensive parallel corpora-both of which are severely lacking for low-resource languages. in contrast, the encoder-decoder architecture is inherently suited for nmt, as the encoder effectively captures the semantic and syntactic structure of the source language, while the decoder generates the corresponding translation. however, despite these advantages, building an encoderdecoder-based nmt system for an extremely lowresource language such as bahnaric remains highly challenging due to severe data scarcity and vocabulary limitations @xcite . to the best of our knowledge, no prior research has been conducted on bahnaric-vietnamese translation. given these challenges, it is crucial to examine the linguistic characteristics of bahnaric and its relationship to vietnamese. both languages belong to the austroasiatic family and are considered lowresource in the linguistic landscape @xcite . moreover, as both languages coexist within the same country and share a common historical and cultural background, they exhibit notable syntactic similarities and structural overlaps. additionally, bahnaric speakers frequently incorporate vietnamese loanwords, particularly in cases where native bahnaric vocabulary lacks equivalents @xcite . these linguistic overlaps serve as critical insights for designing an effective translation system. to leverage these shared linguistic features, we adopt bartpho @xcite , a pre-trained encoder-decoder language model built upon the bidirectional and auto-regressive transformers (bart) @xcite architecture, trained on large-scale vietnamese corpora. this model effectively captures the linguistic characteristics of vietnamese, making it a strong foundation for adaptation to bahnaric. to enhance its ability to model the syntactic and lexical properties of bahnaric, we continually train bartpho on augmented monolingual bahnaric data. the model is then fine-tuned on an augmented bilingual bahnaric-vietnamese dataset, producing an optimized translation system, which we refer to as bartbahnar. to address the issue of data scarcity in bahnaric, we implement various data augmentation (da) techniques @xcite specifically designed for nmt. these techniques enrich and diversify the training data, improving translation performance. furthermore, to fully exploit the unique linguistic characteristics of bahnaric, we propose a novel hybrid approach that integrates bartbahnar with rule-based and statistical methods. this hybrid strategy enhances translation reliability, particularly in handling loanwords and resolving cases where direct model-generated translations may be inaccurate, ultimately improving translation quality and supporting linguistic preservation. our key contributions are summarized as follows. ## experimentations we conduct two main experiments. in the first, we compare our bartbahnar model against various baselines on the bahnaric-vietnamese translation task using only the original dataset, providing a fair evaluation under limited data conditions. in the second, we examine how different da techniques affect both bartbahnar's training process, introduced in section 3.2.3, and the performance of our end-to-end translation pipeline. ## conclusion in this paper, we introduced a novel hybrid architecture for low-resource machine translation, focusing on bahnaric-vietnamese and achieving promising results. alongside rule-based methods that leverage shared features, such as the frequent use of loanwords among bahnaric speakers to reduce errors and improve translation quality, our key contribution is the custom language model bartbahnar. this model undergoes a strategic training process: it is first pre-trained on vietnamese monolingual data, then adapted to bahnaric monolingual data, and finally fine-tuned for the bahnaric-vietnamese translation task. by building on the domestic language model bartpho, we substantially reduce training costs while relying on structural commonalities between vietnamese and bahnaric to maintain high performance. we also investigated various data augmentation methods to identify which techniques are most beneficial for low-resource languages like bahnaric. our findings suggest that certain augmentations significantly increase data diversity and enhance translation accuracy, while others may introduce excessive noise, underscoring the importance of carefully selecting augmentation strategies. future work could involve further customizing the language model by integrating additional bahnaric-specific linguistic properties and refining the rule-based components to handle more nuanced text. exploring additional combinations of data augmentation methods also holds potential for further improvements.