TRANS - ZERO : Self-Play Incentivizes Large Language Models for Multilingual Translation Without Parallel Data (2025)

Abstract: The rise of Large Language Models (LLMs) has reshaped machine translation (MT), but multilingual MT still relies heavily on parallel data for supervised fine-tuning (SFT), facing challenges like data scarcity for low-resource languages and catastrophic forgetting. To address these issues, we propose TRANS-ZERO, a self-play framework that leverages only monolingual data and the intrinsic multilingual knowledge of LLM. TRANS-ZERO combines Genetic Monte-Carlo Tree Search (G-MCTS) with preference optimization, achieving strong translation performance that rivals supervised methods. Experiments demonstrate that this approach not only matches the performance of models trained on large-scale parallel data but also excels in non-English translation directions. Further analysis reveals that G-MCTS itself significantly enhances translation quality by exploring semantically consistent candidates through iterative translations, providing a robust foundation for the framework’s success.

## introduction the advent of large language models (llms) witnesses a fundamental shift in machine translation (mt) paradigms from the supervised endto-end training @xcite to the sophisticated generation of fine-tuned language models @xcite . unlike various downstream tasks that gain impressive proficiency through lightweight instruction tuning, multilingual translations between languages still necessitate sufficient fine-tuning with parallel data for specific translation directions. besides external human annotations, researchers like @xcite ; @xcite obtain translation * corresponding author † work done during internship at bytedance research annotations or preferences from external llm, as long as they prove multilingual capability. either way, it faces a notable deficit in data scarcity for less popular languages. moreover, issues arise as the multilingual translation fine-tuning scales up. the reliance on one-on-one mle supervision has been criticized for potential biases that clash with natural language's inherent multilingualism @xcite and poses a risk of catastrophic forgetting. furthermore, exceeding multilingual annotations inversely dilutes the pre-trained knowledge in supported languages, thus degrading overall cross-lingual performance @xcite . @xcite proposes a mixture-of-expert with hand-crafted route across language modules. however, the route and distributed overheads increase exponentially as the number of translation directions involved increases. whereas traditional fine-tuning scaling approaches a plateau, leveraging llms' inherent knowledge rather than external supervision for selfimprovement is trending @xcite . however, adapting this approach to mt introduces two technical challenges. first, systematic cross-lingual exploration requires navigating complex semantic spaces beyond simple prompt engineering. traditional llm planning involves delicate prompt-based reasoning, even finetuning, which is generally unavailable for most scenarios. second, multilingual quality assessment must overcome the limitations of data-dependent quality estimation (qe) metrics and reward model training complexities. in this work, we introduce trans-zero. this innovative self-play framework enables llms to bootstrap their multilingualism by strategically exploring their semantic space, achieving selfimprovement for multilingual translation given only monolingual data. we start by defining a multilingual translation process (mtp) that gen-erates translations by interweaving the languages supported by llm, so the inference is scaled up to explore more potential translations. first, we implement the genetic monte-carlo tree search (g-mcts) upon mtp, exploring potential translations. second, we harness the search to assess translation preferences based on intrinsic multilingual consistency. experiments verify that trans-zero improves the lesser translations through iterative g-mcts and preference optimizations given only monolingual data. we summarize our contributions as follows: ## trans-zero in this section, we present trans-zero. first, we introduce the multilingual translation process (mtp), a novel framework orchestrating multistep translation across multiple languages ( §3.1). second, we implement genetic monte carlo tree search (g-mcts) upon mtp to explore promising translations ( §3.2), which derive preference from cross-lingual semantic consistency in the search purely through translation prompts, eliminating the need for explicit reasoning training or reward learning. finally, we utilize the search results for further preference optimization ( §3.3), enabling the unsupervised mt training given only monolingual data. ## related work the utilization of llm for machine translation has become popular, aligning with the prevailing trends in llm applications. @xcite first shifts the machine translation paradigm to fine-tuned llm with moderate parallel supervision. though llm seems more data-efficient, it does not have a big appetite for large-scale super-vision due to potential catastrophic forgetting @xcite . directly scaling up multilingual mle supervision hurts performance on resource-rich languages @xcite . therefore, xalma @xcite hand-craft a mixture-of-expert to route different translation directions through separated modules, while alma-r @xcite turn to scale up more expensive preference tuning. preference tuning offers flexibility when fitting llm with subjective human expectations for openended generations. however, the expense of preference annotation has led researchers to seek more cost-effective data sources, e.g., with additional assessments such as critic and revise modules @xcite @xcite intuitively, the cross-lingual gaps in llm offer a more scalable self-improvement preference as the proficient languages improve the lesser ones @xcite , e.g., a straightforward improvement roots in the direct mapping from the dominant linguistic ability as preference @xcite . researchers further scale the preference by iterative and competitive gaming theory @xcite , making it possible for models to selfimprove. recent work by @xcite has empirically validated its self-improving potential by employing large-scale reinforcement learning with its multilingual reasoning abilities.