ExTrans: Multilingual Deep Reasoning Translation via Exemplar-Enhanced
  Reinforcement Learning
Authors: name: Jiaan Wang; name: Fandong Meng; name: Jie Zhou
Published: 2025-05-19 11:34:47

Abstract: In recent years, the emergence of large reasoning models (LRMs), such as
OpenAI-o1 and DeepSeek-R1, has shown impressive capabilities in complex
problems, e.g., mathematics and coding. Some pioneering studies attempt to
bring the success of LRMs in neural machine translation (MT). They try to build
LRMs with deep reasoning MT ability via reinforcement learning (RL). Despite
some progress that has been made, these attempts generally focus on several
high-resource languages, e.g., English and Chinese, leaving the performance on
other languages unclear. Besides, the reward modeling methods in previous work
do not fully unleash the potential of reinforcement learning in MT. In this
work, we first design a new reward modeling method that compares the
translation results of the policy MT model with a strong LRM (i.e.,
DeepSeek-R1-671B), and quantifies the comparisons to provide rewards.
Experimental results demonstrate the superiority of the reward modeling method.
Using Qwen2.5-7B-Instruct as the backbone, the trained model achieves the new
state-of-the-art performance in literary translation, and outperforms strong
LRMs including OpenAI-o1 and DeepSeeK-R1. Furthermore, we extend our method to
the multilingual settings with 11 languages. With a carefully designed
lightweight reward modeling in RL, we can simply transfer the strong MT ability
from a single direction into multiple (i.e., 90) translation directions and
achieve impressive multilingual MT performance.