Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation (2025)

Abstract: Large language model (LLM) shows promising performances in a variety of downstream tasks, such as machine translation (MT). However, using LLMs for translation suffers from high computational costs and significant latency. Based on our evaluation, in most cases, translations using LLMs are comparable to that generated by neural machine translation (NMT) systems. Only in particular scenarios, LLM and NMT models show respective advantages. As a result, integrating NMT and LLM for translation and using LLM only when necessary seems to be a sound solution. A scheduling policy that optimizes translation result while ensuring fast speed and as less LLM usage as possible is thereby required. We compare several scheduling policies and propose a novel and straightforward decider that leverages source sentence features. We conduct extensive experiments on multilingual test sets and the result shows that we can achieve optimal translation performance with less LLM usage, demonstrating effectiveness of our decider.

## introduction neural models @xcite @xcite greatly boost machine translation (mt) performance while various inference speed-up strategies @xcite ensure fast translation. as model scales, large language model (llm) @xcite now is able to deliver fairly good translation results @xcite @xcite , which are even comparable to translations done by commercial translation systems. @xcite find that llm performs particularly well when translating content in particular domains. as shown in figure 1 , we find that neural machine translation (nmt) model and llm have own merits and drawbacks. as shown in ## method as shown in figure 2 , compared to the qe threshold (qet) method proposed by @xcite , our approach is more straightforward, relies on fewer model parameters, and has shorter inference time (see appendix f). the former requires using the wmt22-cometkiwi-da foot_0 @xcite model to assess the quality of nmt translation results to decide whether to continue calling the llm for translation. in contrast, our method directly decides whether to call the llm or nmt based on the input source text, which is clearly a challenging task. moreover, our method needs to meet two requirements: 1) minimize the use of llms as much as possible, and 2) if llm results are used, they should outperform the nmt results. it is evident that using qet makes it difficult to fully satisfy the second condition, which impacts the fusion effect. as shown in ## results we validate the aforementioned methods on four zh↔en test sets, as shown in interestingly, our pplt method achieved better results than the qet method, while the number of llm calls was only slightly higher. the result demonstrates that using merely source text features, i.e. text complexity, can get a desirable integration of nmt model and llm. in addition, as lm is trained on nmt-similar data, pplt significantly outperforms qet on the tech test sets. our jdm method achieves best performance on average. it seems that jdm method is able to dynamically control llm usage as it varies greatly on different test sets. for instance, llm outperforms nmt model on news and literary test sets, llm usage is thus high (even over 50% on the literary test sets). on contrary, nmt model outperforms llm on the tech test sets, so llm usage is greatly decreased (less than 10%). regarding domains where llm and nmt model's performances are equal, jdm uses llm as less as possible. on the tech and flores test sets where nmt model outperforms, we witness more llm usage on the tech test sets than on flores, because the tech test sets are more difficult to translate (6-8 points gap regarding da score), and the decider tends to pass complex sentences to the llm. similar results prevail on de2en and ja2en test sets (see appendix h). when llm and nmt each have their respective advantages, jdm can also achieve better integration results. in addition, we also discuss the generalization ability of the jdm method on different model combinations and unknown domains (see appendix i). ## conclusion this paper proposes a fast approach to integrate llm and nmt model in order to improve translation quality while ensuring fast speed and low cost. compared with previous methods that use qe models, our decider determines when to use llm based on source sentence features. we train an end-toend decider to get the desired performance, that is, use llm as less as possible and use llm for translation only when it outperforms nmt model. we test our approach on multiple test sets, including zh2en, en2zh, de2en, and ja2en. the result shows that this straightforward approach achieves almost the best performance. in addition, our experiments show that llm has a particular advantage over nmt when translating internet memes and informal expressions.