NRC Systems for the WMT 2025- LRSL Shared Task (2025)

Abstract: We describe the NRC team systems for the WMT25 Shared Tasks on Large Language Models (LLMs) with Limited Resources for Slavic Languages. We participate in the Lower Sorbian and Upper Sorbian Machine Translation and Question Answering tasks. On the machine translation tasks, our primary focus, our systems rank first according to the automatic MT evaluation metric (chrF). Our systems underperform on the QA tasks.

## introduction this paper describes our systems submitted to the wmt 2025 llms with limited resources for slavic languages shared task. we focused primarily on llm-based machine translation (mt) into upper sorbian (hsb) and lower sorbian (dsb). balancing two competing tasks typically comes at the cost of performance on one task or the other. while we performed preliminary experiments on qa, our initial results were unsatisfactory and we chose to submit a system that was trained only for mt as our primary submission. ## data and models we constrained our systems to the corpora offered by the organizers. the wmt25 corpora are available in the shared task github repository 1 while the previous years' corpora are available in the wmt22 repository. 2 mt data is described in more detail in section 3.1 while qa data is described in more detail in section 4.1. as required by the shared task, we trained our models using the 0.5b, 1.5b, and 3b size qwen2.5 models as the base. our submitted system for the shared task is based on 1 https://github.com/tum-nlp/ llms-limited-resources2025 ## machine translation we focused primarily on mt performance in our submissions. in this section, we describe preliminary experiments on mt, leading to the choices we made for our final submissions (section 5). we used the doc_to_text task prompt suggested in the lm-eval test harness: @xcite translate the following german text to lower sorbian. put it in this @xmath0.\@xmath1. in addition to exploring hyperparameters for supervised finetuning using llama-factory, we tested which combinations of language pairs and directions to use for training as well as different stopping criteria and prompt templates. ## question answering we used our mt-finetuned models as the base models from which to train for the qa task. our main chrf↑ template languages chatml empty qwen de-dsb (42.4) (42.3) (42.4) de-{dsb,hsb} 80.2 80.2 80.1 de-hsb 79.4 79.2 79.3 we also ran limited experiments using supervised finetuning, but initial results showed a large drop in performance on the translation task as a result. ## submissions here, we describe the specifics of our primary submissions. we submitted a single model that was trained to translate into both upper sorbian and lower sorbian. note that our translations were obtained from llama-factory for inference while the qa output are produced using lm-eval. @xcite at submission time our best model attempting to balance between mt and qa performance, according to our dev sets for mt and qa, was based on qwen2.5-1.5b-instruct 14 . it used llama-factory's supervised finetuning (sft training stage) and trained all weights (not lora). using the chatml template, it trained on our filtered corpora from the top part of the official results for the task are shown in we note that, after the late decision to use an mt-only system, we might have benefited from @xcite our removal of newlines for the mt training data meant that we did not typically meet the expected stopping criteria for lm-eval, hence our use of llama-factory for inference. ## conclusion we submitted systems for the upper sorbian and lower sorbian portions of the shared task. due to the lack of success of our approaches to qa, we submitted systems that were trained only on mt. as expected, our systems performed well on translation (our main focus) but underperformed on qa. if we had examined the qa accuracies on our training data earlier, as shown in we had observed some small improvements in qa even when training only on mt when evaluated on dev data, but did not find the same results on the test set. we continue to explore the reasons for this.