E din H els OW WMT 2025 C reole MT System Description: Improving Lusophone Creole Translation through Data Augmentation, Model Merging and LLM Post-editing (2025)

Abstract: In this work, we present our submissions to the unconstrained track of the System subtask of the WMT 2025 Creole Language Translation Shared Task. Of the 52 Creole languages included in the task, we focus on translation between English and seven Lusophone Creoles. Our approach leverages known strategies for low-resource machine translation, including back-translation and distillation of data, fine-tuning pre-trained multilingual models, and post-editing with large language models and lexicons. We also demonstrate that adding high-quality parallel Portuguese data in training, initialising Creole embeddings with Portuguese embedding weights, and strategically merging best checkpoints of different fine-tuned models all produce considerable gains in performance in certain translation directions. Our best models outperform the baselines on the Task test set for eight out of fourteen translation directions. When evaluated on a more diverse test set, they surpass the baselines in all but one direction.

## introduction the introduction of the first shared task for creole language machine translation (mt) @xcite is emblematic of the increased attention that creole languages have received in the field of natural language processing in recent years, both as individual languages @xcite @xcite @xcite and in multilingual modeling efforts @xcite . building on the latter, this shared task covers over 50 creole languages from a range of geographical and linguistic contexts. some are relatively high-resourced; for example, haitian creole and papiamento are supported in google translate and many others are institutionalised as official or educational languages @xcite . others are extremely low-resource languages or even critically endangered or extinct. the shared task invites submissions of data and systems serving mt between any of the creole languages and either english or french, with the existing kreyòl-mt @xcite and creole-val @xcite translation models serving as baselines. in this submission, we develop systems to translate between english (eng) and seven lusophone foot_0 creoles: angolar (aoa), annobonese (fab), guinea-bissau creole (pov), kabuverdianu (kea), papiamento (pap), principense (pre) and sãotomense (cri). 2 this set includes relatively high-resource creoles (like pap and kea) and extremely low-resource ones (like aoa, fab and pre). in our submission, we utilise known strategies for low-resource mt as well as techniques designed to leverage the linguistic relationship between our seven creoles of focus and portuguese (por). in particular, we contribute the following: while the models released in @xcite and @xcite are the strongest baselines for mt for creoles in general, some of our seven languages of focus are also included in other prior work on mt. the no language left behind (nllb) translation model excels at translation of low-resource languages, and supports pap and kea (as well as three other creoles not included in our study) @xcite . the training data curated as part of the nllb effort include less than 10 bitexts for each creole, but 28m monolingual sentences in pap and 300k in kea. the flores-200 evaluation dataset was also translated into both creoles in this context. both kea and pap are featured in panlex, 4 a massive, open-access online lexicon covering over 5,000 languages @xcite ; but only pap is supported in gatitos, a smaller, higherquality parallel lexicon for low-resource languages developed by @xcite . these lexical resources have been used to improve low-resource mt performance for creoles. following prior work using llms to post-edit machine translation system outputs to correct errors @xcite @xcite , @xcite showed that including the entire gatitos lexicon in such post-editing prompts can improve chrf scores and reduce lexical confusion, including for pap-eng mt. similarly, hus and anastasopoulos to academic researchers for non-commercial purposes upon request; please contact the lead author for license agreement and access. 4 at the time we conducted our study, panlex was not accessible online and so we did not use this resource for kea in our work. the question of how training data from related languages can improve mt for creoles remains open @xcite . @xcite showed that the speech foundation model whisper @xcite performs surprisingly well on keaeng speech translation (despite having not been trained on kea speech) when the por language code is used for decoding, which they hypothesise is due to pronunciation similarities between the two languages. conversely, @xcite demonstrated that parameter efficient fine-tuning via language adapters improves mt for three creoles (including pap) regardless of whether the adapters were trained on related languages, unrelated languages, or even random noise, indicating that language adapters improve performance due to regularization rather than cross-lingual transfer. ## data in this section, we briefly describe the data provided by the task organisers and the additional data we collect and create for model training. our novel data sources are documented in full in ## models to create our mt systems, we fine-tune the three multilingual pre-trained translation models described in section 2: kmt @xcite , mbart-50 @xcite , and nllb @xcite . we explain our approach for fine-tuning each model below, listing additional training configuration details in appendix e. ## results and discussion in this section, we report and discuss the results of our fine-tuning experiments, model merging and post-editing with llms. all results are calculated using the chrf metric foot_12 @xcite implemented in the sacrebleu library @xcite . 23 fine-tuning we find through initial fine-tuning on our dataset that the best overall models for translation into eng are kreyol-mt fine-tuned for xx-xx translation (kmt1), mbart-50 m2m finetuned for xx-eng translation (mb1) and nllb distilled-1.3b fine-tuned for xx-eng translation (nllb1). we find the best overall models for translation out of eng are kreyol-mt fine-tuned for xx-xx translation (kmt1), mbart-50 o2m finetuned for eng-xx translation (mb2) and nllb distilled-1.3b fine-tuned for eng-xx translation (nllb2). for each of these best setups, we then implement our initial set of experiments by retraining each model using por embeddings, por data or distilled data, and then the combinations of the two best settings for each base model. the results in our best model for kea, pov and cri (fine-tuned kreyol-mt) does not beat the kreyol-mt baseline in these languages, so we experiment further with fine-tuning kreyol-mt. we therefore repeat the three experiments while fine-tuning kreyol-mt only for one translation direction at a time (xx-eng or eng-xx), as well as fine-tuning on only the highest-resource languages (cri, pov, kea and pap). to further improve scores, we find each model's best checkpoint for each language pair on the validation set and then use this checkpoint to translate test kmt for that language pair. any of these new models which improve on our previous best results for a given language pair are included in ## data contamination at the end of the shared task, organisers communicated with us that the kreyol-mt model, one of the specified baseline models for the unconstrained systems track, had been trained on some of the shared task public test data; and the intended baseline was kreyol-mt-pubtrain. this explains why kreyol-mt scored so highly on the official test set for certain language pairs (kea, pov and cri), and why our models cannot beat it in these directions despite additional data and modelling efforts. for our submission, this clarification impacted our experimental baseline and our finetuned or merged models which use kreyol-mt as a base model. this means a substantial proportion of our submissions were affected. 25 to address this, we re-evaluated both the kreyol-mt baseline and our submission 1 and submission 2 models 26 on two further test sets: ## conclusion our submissions to the wmt 2025 creoles mt systems subtask utilise a range of known mt techniques, including fine-tuning three pre-trained multilingual translation models on both task data and additional data, merging best models and checkpoints and post-editing system outputs using llms. while no single fine-tuning, merging or post-editing strategy emerged as best amongst all language pairs, we observed considerable gains over the baseline kmt model performance on the test kmt dataset for pap, aoa, fab and pre by combining different approaches, including oversampling the lowest-resource languages in the training data via temperature sampling. while some of our results are unreliable due to the fact that test kmt is contaminated with kreyol-mt training data, we demonstrate the robustness of our model's performance using alternative test sets, and show that kreyol-mt appears to be overfitted to kmt-style data in general. future work could explore whether the techniques and strategies we have utilised here to improve performance are also useful for other creole language pairs and across data from a broader variety of different domains.