JHU WMT 2025 C reole MT System Description: Data for Belizean K riol and F rench Guianese Creole MT (2025)

Abstract: This document details the Johns Hopkins University’s submission to the 2025 WMT Shared Task for Creole Language Translation. We submitted exclusively to the data subtask, contributing machine translation bitext corpora for Belizean Kriol with English translations, and French Guianese Creole with French translations. These datasets contain 5,530 and 1,671 parallel lines of text, respectively, thus amounting to an 2,300% increase in publicly available lines of bitext for Belizean Creole with English, and an 370% such increase for French Guianese Creole with French. Experiments demonstrate genre-dependent improvements on our proposed test sets when the relevant state-of-the-art model is fine-tuned on our proposed train sets, with improvements across genres of up to 33.3 chrF++.

## introduction the vast majority of countries and territories throughout central and south american are hispanophone. two notable exceptions to this trend are anglophone belize and francophone french guiana, pictured in figure 1 . these are both home to a duplicity of creole languages that exist alongside english, french, and other regional languages. many thousands of belize residents speak belizean kriol and garifuna. and french guiana is home to french guianese creole, saramaccan, and ndyuka @xcite . according to the statistical institute of belize' 2022 population and housing census, belizean creole is spoken by 181k people in belize (or 49% of the overall population), making it the third-mostspoken language of the country, after english (at 278k, or 75.5% of the population) and spanish @xcite . because english and french are prominent in belize and french guiana, respectively, and because these european languages are used by populations that do not speak belizean kriol or french guianese creole in belize and french guiana, we justify the choice of these languages for translation into and out of our creole languages of focus. in this work, we contribute: they put together a belizean kriol bitext with english, including 12,085 lines of training data. using these data, they trained the creolem2m model, foot_1 which acheives 44.4 chrf @xcite for belizean kriol to english translation and 46.3 chrf for english to belizean kriol (on their own test set). however, none of the creoleval data were publicly released to the research community. the other publication to benchmark belizean kriol translation was kreyòl-mt @xcite , a project that produced a dataset and model for translation of 41 creole languages, primarily of the colonial african diaspora. the kreyòl-mt model foot_2 supports translation in 172 language directions-all of which include one creole language and one other language, usually english or french. the kreyòl-mt model achieves 53.3 chrf on the kreyòl-mt test set for belizean kriol to english translation, and 83.3 chrf for english to belizean kriol (on their own private test set). the same model achieves very similar scores on the creoleval @xcite test set: 53.3 chrf into english, and 83.5 into belizean kriol, establishing kreyòl-mt as state-of-the-art for belizean kriol translation with english. the dataset used to develop kreyòl-mt contains 31,002 total lines of bitext for belizean kriol aligned with any other language, but only 229 of these lines (all aligned with english translations) were released to the research community in the kreyòl-mt public dataset. 5 kreyòl-mt is also the only published work to address french guianese creole mt. on its own test set, kreyòl-mt achieves 71.9 chrf translating french guianese creole into french and 62.4 translating french into french guianese creole. this was accomplished using only 292 lines of bitext for training data, indicating that these high scores may be a result of over-fitting to the training domain. this entire bitext was released publicly, along with 50 lines of bitext for testing and 50 for development/validation. ## task description the first shared task for creole language machine translation (creolemt) for the tenth conference on machine translation (wmt25) @xcite was announced to further advance the state of machine translation for creole languages. the shared task is comprised of two subtasks: one for systems and one for data. we participate only in the data subtask, which solicits new bitext data for creole language mt. as part of the set up for the systems subtask of the shared task, the task organizers established the public kreyòl-mt dataset as the primary source of training data for the constrained submission track. to make the evaluation fair, they named the kreyòl-mt pubtrain model-an mbart @xcite initialization developed only with the publicly released kreyòl-mt train and development sets and hosted at https://huggingface.co/ jhu-clsp/kreyol-mt-pubtrain with the name kreyol-mt-pubtrain-as the baseline model for benchmarking. (the flagship kreyòl-mt model was trained on some proprietary data not available for public release and hence would not be a fair baseline.) since the datasets we curate in this work are publicly releasable, we compare their utility to that of other publicly available data and also adopt kreyol-mt-pubtrain as our primary base-line model for mt experiments. ## dataset creation we detail our creation approaches for all datasets. ## dataset details our datasets consist of one genre each, educational material in belizean kriol and english, and biblical text and literature in french guianese creole and french. both datasets we shuffled randomly and then split into train, dev, and test sets with an 80-10-10 split ratio. see the shared task findings @xcite for data cards of our submitted datasets. ## experimental results here we detail experiments that minimally show the utility of our datasets, as requested by the shared task organizers. we show both spbleu @xcite and chrf++ scores @xcite for the kreyol-mt-pubtrain model both out of the box, and after fine-tuning on our train sets (with our dev sets used to determine time of early stopping). we trained with a learning rate of 2 * 10 -5 , a batch size of 4, a weight decay of 0.01, and a maximum of 10 training epochs. we implemented early stopping with a stopping patience of 2 and a stopping threshold of 0.01. we fine-tuned each model on both translation directions for the language pair in question, and with early stopping we ended up finetuning for belizean kriol-english translation for 13,272 training steps, and for french guianese creole-french translation for 4,683 training steps. ## conclusion we introduce three new datasets for creole language translation of two language pairs: 1. a bitext with 5,530 aligned translations of the educational genre in belizean kriol and english, resulting in a 2,304% increase of publicly available bitext data for the language pair, and improvements of 28.9 chrf++ into-english and 33.3 chrf++ out-of-english over the state-of-the-art mt model trained on publicly available kreyòl-mt data on our ingenre test set 2. two bitexts with combined 1,671 aligned translations in french guianese creole and french: 879 biblical sentences and 792 lines of guianese folktales, resulting in a 370% increase of publicly available bitext data for the language pair; improvements of 12.6 chrf++ into-french and 14.3 chrf++ out-of-french over the state-of-the-art mt model trained with publicly available data on our biblical test set, and like improvements of 3.2 into-french and 3.8 out-of-french on our fables test set we hope these incremental contributions may forward research in mt for creole languages.