DLUT and GTCOM ’s Large Language Model Based Translation System for WMT 25 (2025)

Abstract: This paper presents the submission from Dalian University of Technology (DLUT) and Global Tone Communication Technology Co., Ltd. (GTCOM) to the WMT25 General Machine Translation Task. Amidst the paradigm shift from specialized encoder-decoder models to general-purpose Large Language Models (LLMs), this work conducts a systematic comparison of both approaches across five language pairs. For traditional Neural Machine Translation (NMT), we build strong baselines using deep Transformer architectures enhanced with data augmentation. For the LLM paradigm, we explore zero-shot performance and two distinct supervised fine-tuning (SFT) strategies: direct translation and translation refinement. Our key findings reveal a significant discrepancy between lexical and semantic evaluation metrics: while strong NMT systems remain competitive in BLEU scores, fine-tuned LLMs demonstrate marked superiority in semantic fidelity as measured by COMET. Furthermore, we find that fine-tuning LLMs for direct translation is more effective than for refinement, suggesting that teaching the core task directly is preferable to correcting baseline outputs.

## introduction the field of machine translation is undergoing a profound paradigm shift, marked by the ascent of general-purpose large language models (llms) that challenge the dominance of specialized encoder-decoder neural machine translation (nmt) architectures @xcite . for years, nmt systems, meticulously trained on vast parallel corpora, have been honed into highly effective, specialized tools for a single task: translation @xcite . in contrast, llms, pretrained on web-scale multilingual and multimodal data, emerge as powerful generalists, possessing not only cross-lingual capabilities but also extensive world knowledge and reasoning skills (brown * corresponding author et al., 2020), which they can apply to translation with remarkable zero-shot proficiency. this dichotomy between the "specialized artisan" (nmt) and the "generalist polymath" (llm) raises critical questions about the future trajectory of machine translation research. this transition is further complicated by an evolution in evaluation philosophy. the community is increasingly moving away from lexical overlap metrics like bleu @xcite , which may unduly penalize valid, fluent translations that diverge stylistically from a single reference. the rise of semantic-aware metrics such as comet @xcite and its successor, xcomet-xl @xcite , reflects a demand for evaluations that prioritize meaning and fidelity. this shift is particularly pertinent when comparing nmt and llms, as llms often excel at producing semantically coherent and contextually appropriate outputs that might be lexically dissimilar to the reference. a core challenge, therefore, is to conduct a fair comparison that accounts for this evaluation dichotomy. in this paper, we leverage our participation in the wmt25 general machine translation task as a standardized testbed to systematically investigate this ongoing paradigm shift. our work is guided by two central research questions (rqs): to address these questions, we developed a comprehensive suite of systems. our nmt pipeline features deep transformer models trained with the fairseq toolkit, enhanced by data augmentation. our llm pipeline is built upon the powerful gemma3 model family @xcite , which we adapt using the llama-factory framework @xcite . our main contributions are: (1) a robust empirical comparison of nmt and llm systems across five language pairs, revealing a significant divergence between lexical and semantic evaluation scores; (2) a direct analysis of two distinct llm fine-tuning strategies, demonstrating the superior efficacy of direct translation; and (3) insights into the qualitative differences between the outputs of these systems, highlighting the semantic strengths of modern llms. ## task description the core of this task is bilingual text translation. the data, sourced using the 'mtdata' tool @xcite from the official wmt25 repository, consists of both parallel and monolingual corpora. ## methodology our methodology is designed as a comparative study of two distinct translation paradigms. we first establish a strong baseline representing specialized nmt systems and then build upon a generalist llm foundation, exploring different adaptation strategies. ## experimental setup our experiments were designed to ensure a fair and reproducible comparison between the nmt and llm paradigms. ## results and discussion in this section, we analyze our experimental results to answer the research questions posed in the introduction. we dissect the performance of each paradigm and discuss the implications of our findings. the comprehensive results for our nmt baselines and zero-shot llm evaluations are consolidated in ## conclusion in this paper, we presented a systematic comparison between specialized neural machine translation (nmt) systems and general-purpose large language models (llms) within the framework of the wmt25 general mt task. our work was designed to investigate the ongoing paradigm shift in the field, focusing on the divergence in performance characteristics and the efficacy of different llm adaptation strategies. our investigation yielded clear answers to our initial research questions. first (rq1), we identified a significant and consistent "bleu-comet dichotomy." while our highly optimized nmt systems remained competitive, and occasionally superior, in terms of lexical similarity (bleu), llms demonstrated a marked advantage in semantic fidelity (comet), even in a zero-shot setting. this finding underscores the limitations of traditional metrics in the age of llms and highlights the unique ability of these large models to produce fluent and semantically equivalent translations. second (rq2), our experiments on supervised fine-tuning strategies provided a decisive result: direct translation proved to be a more effective adaptation method than translation refinement. we hypothesize that teaching the model the core sourceto-target mapping task directly provides a cleaner and more potent learning signal than asking it to perform the more complex, multi-step task of identifying and correcting errors from a baseline system. our best systems, based on full-parameter finetuning of the gemma3-27b-it model, achieved highly competitive results. however, our analysis suggests that the next frontier for llm-based translation lies beyond standard sft. the path to state-of-the-art performance will require more sophisticated alignment techniques that can better bridge the gap between the llms' vast generative capabilities and the nuanced preferences of human evaluation. future work should therefore prioritize the exploration of methods such as continual pre-training on in-domain corpora and, most promisingly, preference optimization techniques like dpo.