LLM s Can Achieve High-quality Simultaneous Machine Translation as Efficiently as Offline (2025)

Abstract: When the complete source sentence is provided, Large Language Models (LLMs) perform excellently in offline machine translation even with a simple prompt “Translate the following sentence from [src lang] into [tgt lang]:”. However, in many real scenarios, the source tokens arrive in a streaming manner and simultaneous machine translation (SiMT) is required, then the efficiency and performance of decoder-only LLMs are significantly limited by their auto-regressive nature. To enable LLMs to achieve high-quality SiMT as efficiently as offline translation, we propose a novel paradigm that includes constructing supervised fine-tuning (SFT) data for SiMT, along with new training and inference strategies. To replicate the token input/output stream in SiMT, the source and target tokens are rearranged into an interleaved sequence, separated by special tokens according to varying latency requirements. This enables powerful LLMs to learn read and write operations adaptively, based on varying latency prompts, while still maintaining efficient auto-regressive decoding. Experimental results show that, even with limited SFT data, our approach achieves state-of-the-art performance across various SiMT benchmarks and different evaluation metrics, and preserves the original capabilities of offline translation. Moreover, our approach generalizes well to document-level SiMT setting without requiring specific fine-tuning, even beyond the offline translation model.

## introduction simultaneous machine translation (simt) @xcite is a critical technique for enabling seamless cross-linguistic communication in realtime scenarios, such as international conferences. unlike offline machine translation (omt), where the fixed policy (conversational simulmt) reads a predetermined number of words per step, leading to translation errors due to incomplete semantic understanding (e.g., prematurely translating "avalanche at washington state" as "华盛顿州的雪崩"). in contrast, east uses a adaptive policy and recognizes the incompleteness of the semantic context and continues reading more words until "ski resort", resulting in the correct translation. the entire source sentence is available before translation begins, simt systems start translating before receiving the complete input, achieving a balance between translation quality and latency. large language models (llms) have achieved significant advances in omt, demonstrating impressive capacity when translating full sentences in offline settings @xcite . however, their application to simt remains underexplored and faces several significant challenges. first, most existing simt models @xcite @xcite are typically trained on omt data due to the scarcity of simt-specific datasets. this training setup does not align well with the demands of simt, which hinders the model's ability to learn how to translate effectively with incomplete input @xcite . second, many simt approaches focus on optimizing prompt structures to simulate simt for llms @xcite @xcite @xcite , as shown in figure 2 changes continuously with the update of the source and target. this recomputation significantly increases the computational cost and inference latency, limiting the efficiency of simt systems @xcite . lastly, llms-based methods typically employs fixed policies @xcite @xcite @xcite , such as the wait-k, for their simplicity. however, these methods fail to adaptively adjust its read/write actions based on sentence structure and context, leading to suboptimal translation quality, as shown in figure 1 . in this paper, we introduce east, an efficient and adaptive simultaneous machine translation method with llms, which aims to achieve highquality simt as efficiently as offline translation. specifically, we first leverage the instructionfollowing capability of llms to generate the simt data @xcite with different latency levels (low, medium, and high). two simt datasets are constructed for supervised fine-tuning (sft), including the german-english dataset simt-de-en-660k and the multilingual dataset simt-multi-90k. we structure the sft data by alternating between the source and target segments of the generated simt data and introducing two special tokens (<|end-of-read|> @xmath0. by performing sft on this structured data on the llms, it can learn to effectively determine when to read more source input and when to generate the translation. a two-stage fine-tuning process is conducted to enhance its multilingual translation capabilities, with full-weight fine-tuning on the simt-de-en-660k dataset, followed by lora @xcite finetuning on a combination of the simt-multi-90k and off-multi-120k datasets. during inference, east employs an adaptive read-write policy that aligns with its sft recipe. the model predicts token-by-token and then dynamically switches between read and write actions based on whether the predicted token is a read or write signal. due to the auto-regressiveness of our token input/output (i/o) sequence, where new source input and target translations are incrementally appended, east can efficiently reuse the kv cache without modifying historical sequence. this significantly reduces computational costs and inference latency, improving the overall efficiency of simt. a comparison to highlight the main difference between the previous llm simt and ours is shown in figure 2 . experimental results show that east achieves highquality simt across eight translation directions and near-offline decoding speeds, without compromising offline translation performance, and generalizes well to document-level simt. our contributions are summarized as follows: traditional simt simt requires starting translation before the full source sentence is available, aiming to balance translation quality and latency. to achieve this, a read-write policy is introduced to determine whether to wait for more input or begin translating. traditional simt models are often built on encoder-decoder architectures, with fixed or trainable policies. the widely studied fixed waitk policy @xcite @xcite @xcite is simple but struggles with complex contexts or non-monotonic language pairs @xcite . adaptive policies, which dynamically determine when to read and write, offer improved translation quality. to enable models to learn effective read-write decisions, a variety of techniques have been applied, including reinforcement learning @xcite @xcite , dynamic programming @xcite @xcite , data augmentation @xcite , information transport theory @xcite , hidden markov models @xcite , and decoder-only architecture @xcite . ## methods in this paper, we propose east, an efficient and adaptive simultaneous machine translation method with llms, which involves three key components: the construction of simt data, the training of the llms on the sft data, and the inference with adaptive read-write policy. ## conclusion in this paper, we introduce an efficient and adaptive simultaneous translation method using llms, east, designed to achieve high-quality simt with the efficiency of offline systems. by constructing sft data, leveraging an interleaved token structure with explicit read-write signals and incorporating latency-aware prompts, east enables llms to perform adaptive reading and translation based on varying latency requirements. our experimental results demonstrate that east not only achieves state-of-the-art performance on simt benchmarks but also maintains high-quality translations in offline settings. additionally, east shows excellent generalization to document-level simt, highlighting its suitability for streaming translation in real-world scenarios.