Krey-All WMT 2025 C reole MT System Description: Language Agnostic Strategies for Low-Resource Translation (2025)

Abstract: The paper is a submission to the WMT 2025 Creole MT Task, aimed at improving translation between Seychellois Creole and English by using data from other closely related Creole languages. The approach treats all Creoles as variations of the same language, allowing the model to share knowledge across them. This strategy led to better translation quality than using separate language settings. The work highlights that related languages can effectively support each other in low-resource settings, while also noting challenges such as data imbalance, differences in language domains, and the absence of human evaluation.

## introduction the work of this paper is based on @xcite which presented a one of a kind, cumulative dataset for creole language mt. this is an important topic owing to the fact that many creole language speakers live in areas where their language is in the minority and they could benefit from better machine translation systems. the main objective behind this paper is to see if a certain language pair translation could benefit from similar languages when building an mt system. to that end, i picked seychellois creole (crs)-english (eng) pair and to supplement the model training mauritian creole (mfe), french guianese creole (gcr), louisiana creole (lou) and réunion creole (rcf) were chosen. the linguistic proximity of these creoles were studied in @xcite . many creoles also have linguistic relationships with high resource languages which gives better prospects for cross-lingual transfer @xcite and hopefully can be studied in a future iteration of this paper. in addition to the shared-task submission for crs-eng, this work evaluates the broader applicability and limits of unified tagging by comparing performance across multiple french-lexifier creoles as well as tok pisin, a typologically distant creole, thereby situating the approach within a linguistically grounded framework rather than a purely engineering exercise. ## data since this work was part of the constrained track, the data and base model used were provided by @xcite . as mentioned above, the 5 creole languages used were chosen based on their linguistic proximity. seychellois creole and mauritian creole are widely regarded as the closest pair among the varieties considered here, a connection that stems largely from historical migration patterns @xcite . seychelles was primarily settled from mauritius in the late 1700s, and while there are lexical and phonological differences, several scholars have noted that they may be treated as regional variants of a single linguistic system rather than entirely separate languages (michaelis and rosalie, 2013) @xcite . réunion creole, while geographically proximate, has a distinct and earlier colonization history, resulting in greater grammatical divergence from mauritian. nevertheless, commonalities in the pronominal system, such as the use of zot for both second and third-person plural, and shared plural markers like ban(de), point to the effects of geographical continuity and inter-island contact. lexical similarities, including borrowings from malagasy and indo-portuguese, reinforce the role of shared areal features despite significant typological differences @xcite . expanding beyond the indian ocean, louisiana creole and french guianese creole also exhibit no-table structural parallels with mauritian, underscoring broader transoceanic affinities among frenchlexifier creoles. @xcite for mt research, this interconnected yet varied typological landscape offers opportunities for cross-lingual transfer learning, provided that models are designed to exploit shared structures without conflating distinct grammatical systems. @xcite as far as creoles go, i have also chosen tok pisin as a linguistically dissimilar language to seychellois creole to demonstrate how this strategy is effective only when the varieties share substantial structural overlap. ## methodology this work adopts a language-agnostic machine translation strategy for low-resource creole-english translation, building on multilingual pretraining and fine-tuning techniques that have been shown to benefit typologically related languages in low-data regimes @xcite ; @xcite ; @xcite . i call the approach "language-agnostic" because the model is not given any explicit linguistic rules or handcrafted features about creole varieties; instead, all source languages are treated under a unified tag, allowing the system to learn transfer purely from shared representations without relying on language-specific annotations. ## experiments the experiments compare baseline, all kreyols, partial krey-all, and specialized setups. performance was evaluated using bleu @xcite and chrf @xcite on the held-out seychellois creole-english test set. ## results and analysis the language-agnostic tagging strategy which was the main goal of this paper proved most effective, as shown by two clear outcomes. first, tagging crs across all creoles consistently yielded higher scores compared to runs where tags were languagespecific. second, even when crs data was oversampled by 10x, scores did not improve over the 5x runs, despite mfe data being more abundant, indicating that mfe served as a strong substitute for crs in the shared tag setup. this aligns with prior findings that shared tags encourage parameter sharing across related low-resource languages @xcite ; @xcite . no freezing continued to outperform partial freezing, suggesting that kreyol-mt-pubtrain's multilingual pretraining already aligns creole varieties in the embedding space. scheduler and label smoothing choices impacted results more than weight decay, with cosine or linear schedules and @xmath0.1 performing best overall. ## conclusion this study examined a unified tagging strategy using kreyol-mt-pubtrain to improve translation quality for the low-resource seychellois cre while absolute gains are modest (<1 bleu/chrf), they are consistent across runs and show that leveraging related creoles can stabilize low-resource training. moreover, since the constrained track provides the same pretraining data for both base and fine-tuning, the improvements are naturally limited compared to unconstrained settings. beyond crs-eng, applying the same strategy to other creole-english directions (rcf, lou, gcr,mfe) produced similar improvements, confirming that the benefits of unified tagging generalize across french-lexifier creoles. in contrast, tok pisin (tpi), a typologically distant creole, did not benefit, which corroborates the claim that this approach is most effective when applied to closely related languages. no-freezing fine-tuning proved most effective, suggesting that kreyol-mt-pubtrain's multilingual pretraining already aligns creole varieties well in embedding space. the best system, trained on all creoles without freezing, reached a bleu of 35.44 and chrf of 61.71 for crs-eng, outperforming the baseline. overall, cross-lingual transfer with linguistically related low-resource languages emerges as a promising strategy in constrained mt settings, particularly when paired with balanced exposure and a unified tagging scheme. if there is one takeaway from this paper, it is that prior linguistic research remains vital even in the age of large language models. more often than not, leveraging linguistic insight can be just as critical as scaling data volume. @xcite limitations while the results are promising, there are several limitations to this work: