DRT : Deep Reasoning Translation via Long Chain-of-Thought (2025)

Abstract: Recently, O1-like models have emerged as representative examples, illustrating the effectiveness of long chain-of-thought (CoT) in reasoning tasks such as math and coding tasks. In this paper, we introduce DRT, an attempt to bring the success of long CoT to neural machine translation (MT). Specifically, in view of the literature books that might involve similes and metaphors, translating these texts to a target language is very difficult in practice due to cultural differences. In such cases, literal translation often fails to convey the intended meaning effectively. Even for professional human translators, considerable thought must be given to preserving semantics throughout the translation process. To simulate LLMs’ long thought ability in MT, we first mine sentences containing similes or metaphors from existing literature books, and then develop a multi-agent framework to translate these sentences via long thought. In the multi-agent framework, a translator is used to iteratively translate the source sentence under the suggestions provided by an advisor. To ensure the effectiveness of the long thoughts, an evaluator is also employed to quantify the translation quality in each round. In this way, we collect tens of thousands of long-thought MT data, which is used to train our DRT. Using Qwen2.5 and LLama-3.1 as the backbones, DRT models can learn the thought process during machine translation, and outperform vanilla LLMs as well as LLMs which are simply fine-tuning on the paired sentences without long thought, showing its effectiveness.

## introduction recently, the emergence of the o1-like llms shows great performance in reasoning tasks, e.g., math and coding tasks @xcite @xcite @xcite . with the help of long thought, llms tend to explore, reflect and self-improve the reasoning processes to achieve more accurate answers. in this paper, we explore technical routes to bring the success of long thought to mt. to this end, we introduce drt, a product of our exploration, and we hope it could facilitate the research community. there are two key points in achieving this goal: for i), inspired by @xcite , a possible scenario is translating sentences with similes or metaphors, where literal translation often fails to convey the intended semantics. given that, we decide to mine such sentences from literature books. the mining process uses an advanced large language model (llm) to first judge q1: whether each literature sentence has any similes or metaphors. if has, the llm will be asked to literally translate the sentence to a target language, and give a final judgment on q2: whether literal translation is effective for native speakers of the target language to comprehend. if the answers of q1 and q2 are "yes" and "no", respectively, the corresponding literature sentences will be reserved, and regarded as "suitable to translate via long thought". for ii), after collecting the literal sentences with similes or metaphors, the next question is how to synthesize long thought mt samples. previous work typically utilizes monte carlo tree search (mcts) @xcite @xcite or data distillation @xcite (from existing o1-like models) to collect long thought sft samples. nevertheless, mcts is typically used in math and coding tasks where multiple reasoning behaviors should be considered, and the method emphasizes complex reasoning that might not be efficient for machine translation. besides, utilizing existing o1-like models for data distillation might (1) constrain the potential quality of the long-thought data; and (2) have a data gap in mt since current o1-like models are typically optimized toward math and coding tasks. therefore, we propose a multi-agent framework to synthesize mt data with long thought. in detail, there are three agents in the framework, i.e., a translator, an advisor and an evaluator. the synthesis process is iterative, consisting of the following three steps during each iteration: (1) the translator generates a new translation conditioned on the previous step's translation and the corresponding refinement suggestions from the advisor; (2) the advisor evaluates the current translation and offers detailed feedback; (3) the evaluator assesses the current translation and gives an evaluation score using predefined scoring criteria. once the translation score provided by the evaluator reaches a pre-defined threshold or the number of iterations reaches a maximum value, the iteration will stop. after that, the translation and suggestions in every step could form the long-thought mt samples. to improve the readability and fluency of the longthought data, we employ gpt-4o (openai, 2024a) to reformulate the long-thought content. based on the collected long-thought mt samples, we train our drt-7b, drt-8b and drt-14b using the backbones of qwen2.5-7b-instruct, llama-3.1-8b-instruct @xcite and qwen2.5-14b-instruct @xcite , respectively. experimental results on literature translation verify their effectiveness. in particular, drt-14b outperforms qwq-32b-preview and deepseek-r1-distill-qwen-32b in terms of bleu, cometkiwi, cometscore and gpt-4 evaluations. moreover, human evaluation and case study show the strong translation performance of drt models. our main contributions are concluded as follows: • we propose drt aiming at building llms with long-thought machine translation ability. to achieve this, we mine literature sentences with similes or metaphors, and collect mt samples with long-thought processes. ## drt data we focus on english-to-chinese translation foot_2 , and we introduce how to collect the long-thought mt samples via three steps in this section: (1) collecting english sentences that tend to require long thoughts during translation ( § 2.1); (2) synthesizing the long-thought translation process for the collected sentences by a designed multi-agent framework ( § 2.2); (3) improving the readability and fluency of the long-thought content to form the final long-thought mt samples ( § 2.3). next, we provide data statistics and data analyses of the collected data to give a deeper understanding ( § 2.4). finally, we discuss the data quality ( § 2.5). ## conclusion in this paper, we introduce drt, an attempt to bring the success of long-thought reasoning to neural machine translation (mt). specifically, we synthesize the machine translation long-thought samples by a designed multi-agent framework and gpt-4o reformulation. to collect the source sentences that are suitable for translation via long thought, we mine sentences with similes or metaphors from existing literature books. to synthesize the long thought machine translation process for these sentences, a translator, an advisor and an evaluator collaborate to translate the source sentence iteratively. based on the synthesized data, we train drt models. extensive experiments on literature translation demonstrate the effectiveness of drt models in terms of automatic evaluation. case study and human evaluation further verify their superiority.