Federated Meta-Learning for Low-Resource Translation of K irundi (2025)

Abstract: In this work, we reframe multilingual neural machine translation (NMT) as a federated meta-learning problem and introduce a translation dataset for the low-resource Kirundi language. We aggregate machine translation models () locally trained on varying (but related) source languages to produce a global meta-model that encodes abstract representations of key semantic structures relevant to the parent languages. We then use the Reptile algorithm and Optuna fine-tuning to fit the global model onto a target language. The target language may live outside the subset of parent languages (such as closely-related dialects or sibling languages), which is particularly useful for languages with limitedly available sentence pairs. We first develop a novel dataset of Kirundi-English sentence pairs curated from Biblical translation. We then demonstrate that a federated learning approach can produce a tiny 4.8M Kirundi translation model and a stronger NLLB-600M model which performs well on both our Biblical corpus and the FLORES-200 Kirundi corpus.

## introduction the federated learning (fl) paradigm has drawn great interest for its inherent privacy, scalability, and performance across myriad vision and language tasks. recent works have proposed federated learning as a solution for low-resource machine translation @xcite . centralized federated learning often focuses on optimizing a global model by aggregating weights over a cluster of clients trained on identical tasks (with varying local datasets). current literature suggests a global model can also be used as a meta-model to increase model performance and convergence speed @xcite . in the meta-learning setting, clients train on similar, but heterogeneous tasks, enabling few-shot adaptation to new tasks of the same flavor. in this paper, we attempt to utilize federated learning, viewed through the meta-learning lens, to produce a seq2seq translation model for kirundi, which despite having 11.2 million speakers, is rarely considered in literature and lacks translation resources. here, the meta-task @xmath0, with varying source language. we aggregate a global model over a small cluster of parent seq2seq models. the parent models train higher-resource bantu languages, specifically luganda, bemba, and kinyarwanda. to the best of our knowledge, the flores-200 dataset @xcite is the only publicly-available parallel translation corpus of kirundi, containing roughly 2000 sentences (aligned with 200 other languages). we produce a novel corpus of 29,506 english to kirundi sentence pairs by scraping pairs from parallel corpuses of the new and old testament produced by the international bible society (available at bible.com). we demonstrate that the federated meta-learning strategy can boost performance on both the flores-200 kirundi and our bible corpus. we use our approach to construct a tiny, but performant 4 million parameter @xmath1, which has already been trained to translate kirundi (among many other languages). ## algorithm and preliminaries the perfedavg algorithm combines fedavg @xcite , reptile @xcite , and personalization to increase convergence speed and stability. to rapidly adapt to a new language for a machine translation task (in our case, @xmath2, we split our approach into 3 steps, similar to perfedavg @xcite . 1. global model training. using the fedavg federated learning algorithm @xcite , we aggregate gradients across multiple clients. each client holds data for a language exclusive to them. training is performed, and gradients are aggregated to update the global model, which is used by clients for the next epoch of training. it is well-known the hetereogeneous weighting of gradients during aggregation is required to achieve optimal performance @xcite @xcite . we use the optuna library to fine-tune gradient weighting rather than using an even average @xcite or weighing by the the amount of client data @xcite . ## conclusion in this work, we curate a dataset and develop an algorithm for english to kirundi translation. despite being a widely spoken bantu language, there were no previously existing translation resources for kirundi. despite limited sentence pairs, our work shows a translation model can be developed with certain federated learning techniques to provide support for an underrepresented language.