Fine-tuning NMT Models and LLM s for Specialised EN - ES Translation Using Aligned Corpora, Glossaries, and Synthetic Data: MULTITAN at WMT 25 Terminology Shared Task (2025)

Abstract: This paper describes our participation in the WMT25 Terminology Shared Task, specifically Track 1 (Spanish to English) focused on translation within the Information Technology (IT) domain. The shared task challenges participants to improve machine translation systems by effectively incorporating terminology constraints to ensure accurate and consistent translation of specialised technical terms. We experimented with several approaches to tackle terminology and lexical constraints with both NMT systems and LLMs with a small amount of training data and a glossary. Experimental results demonstrate that systems behave differently with and without glossary. The NMT system seems rather limited in adapting to special lexicon and resizing embeddings, which is the opposite of the case with LLMs preferring structured instructions. Through this participation, our objective is to improve terminology accuracy and overall translation quality, highlight the potential of specialised terminology-aware translation models for technical domains, and explore possibilities of fine-tuning of LLMs and NMT models with domain and lexical constraints.

## introduction adaptation of the nmt model to a specific domain @xcite @xcite is a major concern in bilingual neural machine translation. among the various methods that have been proposed to tackle domain adaptation, two approaches are particularly relevant to the objective of this shared task: i) data approach that involves selecting and filtering existing in-domain parallel segments @xcite , or generating synthetic data. the latter is widely used in back-translation and in enhancing domain-specific data by reformulating or paraphrasing @xcite ; ii) system approach which aims to assign weights to segments close to the target domain @xcite . in recent years, research has increasingly focused on frugal domain adaptation strategies, emphasising the efficient use of limited resources and optimising training settings to address low-resource scenarios @xcite . with the rapid development of llms for translation purposes in recent years, several methods for llm domain adaptation have emerged, such as prompt engineering and context leaning @xcite @xcite , constrained decoding to enforce terminology or special data format @xcite , supervised fine-tuning (sft) using reinforcement learning from human feedback @xcite , etc. meanwhile, domain adaptation sheds light on the functioning, strengths, and weaknesses of llm @xcite , making these 'black boxes' more interpretable -an important objective of our participation in the shared task. in our contribution, we have chosen to use three strategies to fine-tune both nmt and llm systems to ensure accurate translation of terms tagged as 'proper_terms' in the dev set: ## experimental settings we used constrained machine translation with hard terminology control (en-es) to ensure that english terms were consistently predicted as their spanish equivalents. we implemented our nmt system with marian (junczys-dowmunt et al., 2018), mainly because it allowed constraint decoding with the additional_special_tokens parameter, and the models embeddings were resized accordingly. in the training data, we replaced english terms and their spanish equivalents with placeholders: each pair of bilingual terms was replaced by the same placeholder. consequently, english terms in the input were replaced by placeholders before decoding, and placeholders in the output were replaced by the equivalent spanish terms after generation. our system was fine-tuned with seq2seqtrainer with the following setting: training epochs: 5, learning rate: 1e-5, training batch size: 8, gradient accumulation steps: 2, max length: 128. for the llm model, we used eurollm-1.7b-instruct @xcite as a causal lm due to its relatively light weight for fine-tuning. for each pair of aligned segments (en-es), the system builds a prompt that injects the glossary before instruction as follows: the glossary was created using the english and spanish lexicons tagged as "proper_term" elements in the dev set. after filtering and deduplication, we obtained 172 unique occurrences in english (not case-sensitive). these occurrences correspond to 221 pairs of english and spanish terms. training was set as follows: training batch size: 4, learning rate: 1e-4, weight decay: 0.01, lr scheduler type: linear, warmup steps: 100, training epochs: 1. we used parameter-efficient lora @xcite adapters for light fine-tuning, using 8-bit loading to reduce gpu memory usage: task type: causal lm, r: 16, lora alpha: 32, lora dropout: 0.1. for both the marian and eurollm fine-tuning processes, we used a small number of epochs and a small batch size to reduce training time and prevent over-fitting. the main purpose was to evaluate the efficiency of our training pipelines. we also used an advanced commercial mt platform systran model studio lite 3 to finetune a generic en-es baseline model. nmt was used to ensure accurate sentence-level translations. genai, including llms, was used to develop a synthetic data set that contains domainspecific terms to fine-tune the initial system and implement specific term correspondences, improving coherence regarding terminology specifications (see section 2.3 on data augmentation techniques). ## conclusion and perspectives our contribution to the shared task reveals the strengths and challenges of nmt systems and llms in translation tasks. although nmt systems perform well with small amounts of highquality domain-specific training data, their performance can deteriorate under constrained decoding conditions. in contrast, llms can benefit from structured guidance, such as glossaries and clear instructions, which enhances their translation quality. these findings emphasise the complementary