WMT 2025 C reole MT Systems Description : Martinican Creole and F rench (2025)

Abstract: This paper describes our submissions to the constrained subtask of the WMT25 Creole Machine Translation shared task. We participated with a bidirectional Martinican Creole – French system. Our work explores training-time strategies tailored for low-resource MT, including LoRA fine-tuning, curriculum sampling, gradual unfreezing, and multitask learning. We report competitive results against the baseline on both translation directions.

## introduction the wmt25 creole mt shared task is the first shared task focused on machine translation involving creole languages. as part of the wmt25 creole mt shared task @xcite , we submitted constrained systems for martinican creole ↔ french. this track explores how to improve creole mt under limited resource conditions. participants were restricted to using only the official training data provided by the organizers and were allowed to initialize their models from the baseline kreyol-mt-pubtrain model on huggingface. our submission focused on the martinican creole ↔ french language pair (iso: mart1259-fra). we trained and evaluated a bidirectional system for both @xmath0. motivation our participation stems from ongoing work on nlp tools for creole languages, including martinican creole and haitian creole. despite progress in nlp, creole languages remain underrepresented and under-resourced, making them ideal candidates for constrained mt settings. we aimed to investigate how far one can push model performance without access to expanded corpora by instead exploring training strategies and finetuning configurations tailored to low-resource language modeling. this work is part of a broader effort to develop nlp tools for creole researchers, educators, and communities. in particular, we aim to build usable mt systems, including domain-specific applications (e.g., medical mt for haitian creole). we explore various configurations: custom tokenizer, curriculum sampling, lora fine-tuning, gradual unfreezing, denoising, label smoothing, weighted bleu scoring, multitask learning, and mixed training strategies. the wmt25 constrained track permits only the use of official datasets provided by the organizers. no additional monolingual or parallel data was allowed for training, tuning, or backtranslation. the baseline system is the publicly available kreyol-mt-pubtrain model on huggingface. ## data preprocessing and setup we experimented with a 70/30 training/validation split (3,094 / 1,338) to increase the size of the development set for more stable evaluation but reverted back to the original 90/10 split as it yielded slightly better results. training was conducted separately for each translation direction rather than using concatenated bidirectional data or multitask learning, as this yielded more consistent results in our preliminary experiments given the dataset provided. ## model architecture and training all systems were initialized from the publicly available jhu-clsp/kreyol-mt-pubtrain model on huggingface. we applied low-rank adaptation (lora) for efficient fine-tuning, targeting the attention on q_proj and v_proj. our final configuration used rank @xmath1, scaling factor α = 32, and dropout 0.1, though we also explored @xmath2, 32} and α ∈ {64} in preliminary runs. training was performed with a custom seq2seqtrainer that integrates curriculum sampling (ordering examples by difficulty), gradual unfreezing of encoder and decoder layers, and multitask loss weighting to balance direction-specific performance. we applied label smoothing of 0.1 to improve generalization and used weighted bleu scoring on the development set as the main model selection criterion. the final training configuration included a batch size of 32, maximum source and target lengths of 128 tokens, 50 training epochs, a constant learning rate of 2 × 10 -5 with 500 warmup steps, and the adamw optimizer. evaluation used a beam size of 4 and a maximum of 128 new tokens during generation. ## conclusion and future work under the constrained track, our lora-based systems with direction-specific training choices are close to but ultimately below the strong baseline on the organizers' blind test set. the gap is larger for @xmath3, where bleu and chrf++ are nearly tied. one plausible factor is data scarcity: with limited and relatively homogeneous training material, core architectural or training-time changes may yield muted or unstable gains, and measurement noise can obscure small effects. we will (i) explore alternative parameterefficient adapters and layer targeting paired with data-scaling, (ii) tune curriculum schedules per direction and analyze where they help/hurt, (iii) align preprocessing/normalization and decoding with the organizers' pipeline, (iv) investigate tokenizer and segmentation choices for martinican creole, and (v) add robust automatic and human error analyses (e.g., code-switching, diacritics, oovs). in the unconstrained setting, we also plan to study backtranslation and multilingual transfer to further close the gap.