Ladder: A Model-Agnostic Framework Boosting LLM -based Machine Translation to the Next Level (2024)

Abstract: General-purpose Large Language Models (LLMs) like GPT-4 have achieved remarkable advancements in machine translation (MT) by leveraging extensive web content. On the other hand, translation-specific LLMs are built by pre-training on domain-specific monolingual corpora and fine-tuning with human-annotated translation data. Despite the superior performance, these methods either demand an unprecedented scale of computing and data or substantial human editing and annotation efforts. In this paper, we develop MT-Ladder, a novel model-agnostic and cost-effective tool to refine the performance of general LLMs for MT. MT-Ladder is trained on pseudo-refinement triplets which can be easily obtained from existing LLMs without additional human cost. During training, we propose a hierarchical fine-tuning strategy with an easy-to-hard schema, improving MT-Ladder’s refining performance progressively. The trained MT-Ladder can be seamlessly integrated with any general-purpose LLMs to boost their translation performance. By utilizing Gemma-2B/7B as the backbone, MT-Ladder-2B can elevate raw translations to the level of top-tier open-source models (e.g., refining BigTranslate-13B with +6.91 BLEU and +3.52 COMET for XX→En), and MT-Ladder-7B can further enhance model performance to be on par with the state-of-the-art GPT-4. Extensive ablation and analysis corroborate the effectiveness of MT-Ladder in diverse settings.

## introduction general-purpose large language models (llms) like gpt-4 @xcite have exhibited strong translation abilities @xcite * equally contributed. figure 1 : the average translation quality improvements across 8 translation directions on wmt22 test set (zh↔en, de↔en, en↔ru, en↔cs) using mt-ladder-2b or 7b. the metric scores are calculated by comet-22 (wmt22-comet-da) @xcite . @xcite , but achieving this performance requires enormous model scale, infrastructure, and deployment costs. on the other hand, translation-specific llms like alma @xcite and aya 23 @xcite have reached top-tier levels through continued pretraining on large monolingual corpora (e.g., 20b tokens from common crawl @xcite ) and fine-tuning on high-quality translation data (e.g., 10.5m translation examples from aya dataset @xcite )), which is also time-consuming and costly. these observations raise a question: can we enhance the mt performance of existing llms in a model-agnostic manner, achieving results comparable to translationspecific llms or even gpt-4, without incurring the significant costs associated with human annotations or extensive training? there are two potential approaches to achieving this goal. the first is the prompt-based method, which involves developing effective prompting strategies to better stimulate llms' translation capabilities, such as using in-context translation examples, as outlined in works @xcite @xcite @xcite . however, @xcite indicate that prompting methods overly rely on the language model, often under-translate the input and generate hallucinations. additionally, @xcite demonstrate that the same prompting strategy can lead to different performance across different models. furthermore, most of these prompting strategies like agent debating or self-correction @xcite cannot be applied to some popular neural machine translation models like nllb @xcite . these limitations make the learningfree method non-model-agnostic and unstable. another line of work employs learning-based paradigms by fine-tuning llms to adapt quality estimation @xcite and automatic @xcite tasks to refine raw translations. qe involves automatically predicting translation quality, typically using multidimensional quality metrics (mqm) datasets @xcite , where human experts annotate error spans and assign quality scores. ape aims to address systematic errors of a black-box mt system and tailor the output to the lexicon and style required in a specific application domain. ape datasets are manually collected from real-world post-editing triplets like qt21 @xcite . built on these well-defined tasks and annotated datasets, prior works @xcite @xcite have shown the promising utility and generalization of the learning-based method. @xcite trained palm2 @xcite on mqm datasets to refine translations, and @xcite trained towerinstruct on 637k translation examples, integrating ape datasets, outperforming all open models and gpt-3.5-turbo on ape tasks. however, these works heavily rely on human-annotated evaluation data and lack extensive validation in model-agnostic and multilingual scenarios. additionally, the overall refinement in translation quality, particularly for translation-specific models, remains limited. in this paper, we introduce mt-ladder, a model-agnostic and cost-effective tool for multilingual translation refinement. instead of directly finetuning a translation-target llm, we train an llm to refine translations using refinement datasets without human evaluation or post-edits, employing an instruction-following refinement task (section 2.1). we notice that the reference in existing parallel corpus can serve as a natural refined translation. by sampling a translation for the source sentence from an existing llm as the intermediate translation, we create a pseudo-refinement translation triplet [source, intermediate translation, reference], allowing us to construct training data without extra labor costs. during training, we split the training triplets into three hierarchies (easy, medium, hard) based on their comet (rei et al., 2020) scores and propose a hierarchical fine-tuning (hft) strategy to improve mt-ladder's refining performance step by step. comprehensive experiments demonstrate that effectiveness of our mt-ladder across various llms on multiple translation tasks. ## conclusion in this paper, we introduce mt-ladder, a modelagnostic and cost-effective tool for multilingual translation refinement that bridges the gap between off-the-shelf models and top-tier translation models. we sample translations from existing models to create pseudo-refinement training triplets without human annotations, which makes training cost-efficient. the proposed hierarchical finetuning strategy improves mt-ladder's refining performance step by step, following an easy-to-hard schema. our exploration of training paradigms demonstrates good performance in effectiveness and robustness, as well as promising results in weak-to-strong generalization and self-refinement, providing valuable insights to the mt area.