I n2 X at WMT 25 Translation Task (2025)

Abstract: This paper presents the open-system submission by the In2x research team for the WMT25 General Machine Translation Shared Task. Our submission focuses on Japanese-related translation tasks, aiming to explore a generalizable paradigm for extending large language models (LLMs) to other languages. This paradigm encompasses aspects such as data construction methods and reward model design. The ultimate goal is to enable large language model systems to achieve exceptional performance in low-resource or less commonly spoken languages.

## introduction machine translation (mt) has long been both a high-impact application and a central research challenge in natural language processing. the advent of large language models (llms) has reshaped mt from task-specific supervised learning toward large-scale representation learning and instructionfollowing paradigms, enabling steady gains across diverse language pairs @xcite @xcite . yet, two persistent gaps remain. first, while mainstream llm training increasingly optimizes for mathematical and code reasoning, their expressive and creative language abilities-e.g., idiomaticity, stylistic naturalness, and culturally appropriate phrasing-are comparatively underdeveloped @xcite @xcite @xcite . this often leads to translations that are locally literal but globally stilted, especially for informal registers, slang, and literary text . second, model competence is unevenly distributed across languages: english receives disproportionate coverage and quality, while many non-english languages trail in both general capability and translation naturalness @xcite @xcite @xcite . com-munity findings over recent wmt cycles echo this asymmetry: despite the "llm era", mt is far from solved uniformly across directions, with larger gaps off english-centric pairs and on long-tail phenomena . this paper studies how to transfer english strength into non-english targets to improve expressive and culturally faithful translation. concretely, we focus on japanese-a language where literal adequacy is not sufficient: natural japanese requires idiomatic paraphrasing, honorific and register control, and sensitivity to genre and context. our thesis is that english can be used as a hub language to bootstrap these capabilities via curriculum design, cross-lingual alignment, and preference signals that explicitly reward naturalness. we present in2x, a japanese-focused model designed to inherit general competency from english while specializing for japanese expressiveness. at a high level, in2x operationalizes three principles: (i) english-as-hub transfer: leverage rich english data and strong english modeling to seed robust lexical/semantic priors, then transfer to japanese via bilingual and style-augmented objectives; (ii) expressiveness-first supervision: emphasize prompts and signals that drive idiomaticity and cultural appropriateness (beyond literal adequacy); (iii) evaluation beyond metrics: complement automatic metrics with human judgments targeted at idioms, slang, and stylistic naturalness. we evaluate in2x on standard wmt-style test sets and targeted japanese-focused challenge suites that stress idioms, slang, and style. according to the preliminary ranking results of wmt 2025, in2x outperforms many large-scale proprietary models, such as gemini-2.5-pro @xcite , gpt-4.1 @xcite , claude-4 @xcite , and deepseek-v3 @xcite . overall, we make three core contributions: ## continue pretraining stage to balance the capabilities of large language models (llms) in both science-oriented and humanities-oriented domains during the pretraining process, we divided the continued pretraining stage into three distinct phases. the goal of this process is to enhance the model's multilingual proficiency, improve general-purpose abilities in foundational humanities tasks, and refine its representation in specialized contexts @xcite . the training process incorporates diverse corpora, including a comprehensive 2 trillion tokens dataset comprising encyclopedic knowledge, webpages, structured information, news articles, wikipedia entries, academic papers, and stemrelated datasets @xcite . in addition, a dedicated 500 billion tokens corpus has been curated exclusively for creative writing tasks such as novel and screenplay synthesis, as well as authentic conversational datasets simulating real-life dialogue @xcite . another significant aspect of this training stage focuses on enhancing capabilities in the target language, with japanese utilized as an example. to this end, substantial japanese language-specific corpora were introduced, alongside a balanced dataset with equal distribution of chinese, english, and japanese corpora @xcite . the aim was to facilitate transfer learning from pretraining on chinese and english to the japanese language. ## post-training data the post-training dataset consists of 2 million samples, with 1.5 million used during the supervised fine-tuning (sft) process and 500,000 used in the reinforcement learning (rl) process. to ensure the japanese language (our target language) achieves a proficiency level comparable to major languages such as chinese and english, we adjusted the ratio of target language instructions to attain an equal balance across these languages. specifically, we used a 1:1:1 ratio in the instruct-to-example (in2x) setup, striving to transfer the original model's knowledge into the target language as effectively as possible @xcite . we developed a detailed pipeline for constructing the target language instructions, which can be categorized into three major synthetic processes: ## post-training sft stage the post-training supervised fine-tuning (sft) stage is a critical step to balance linguistic diversity and optimize alignment within the instruction space for target languages. below, we outline the key strategies and methods employed during this stage. ## model ensemble model ensemble techniques are employed by taking into account the orthogonality of linguistic capabilities among various models. specifically, models that exhibit strong linguistic proficiency are selected for the ensemble process to maximize overall performance. furthermore, the fusion of model tensors is conducted based on gradient information and the importance of weights. this approach ensures a robust integration of model parameters, leveraging their respective contributions to optimize the ensemble. such methodologies have been shown to enhance the effectiveness of model ensembles in complex tasks @xcite . ## conclusion this work introduces and validates a method for transferring language modeling capabilities, as demonstrated on the wmt translation task. the proposed approach significantly enhances japanese language proficiency during the cpt, sft, and rl processes. notably, without any additional language-specific fine-tuning, the large language model achieves alignment in its japanese capabilities, bringing them on par with those of mainstream languages. furthermore, this work presents a systematic pipeline for aligning language models, as well as a method for training rewards in the creative content domain. remarkably, this approach requires only around 2,000 annotated samples in the target language to achieve improved language transfer capabilities, with the remaining process relying on automated instruction-building techniques. future efforts will focus on enhancing the literary style of the model by integrating elements such as classical texts and stylistic refinements into the pipeline.