Simultaneous Translation with Offline Speech and LLM Models in CUNI Submission to IWSLT 2025 (2025)

Abstract: This paper describes Charles University submission to the Simultaneous Speech Translation Task of the IWSLT 2025. We cover all four language pairs with a direct or cascade approach. The backbone of our systems is the offline Whisper speech model, which we use for both translation and transcription in simultaneous mode with the state-of-the-art simultaneous policy AlignAtt. We further improve the performance by prompting to inject in-domain terminology, and we accommodate context. Our cascaded systems further use EuroLLM for unbounded simultaneous translation. Compared to the Organizers’ baseline, our systems improve by 2 BLEU points on Czech to English and 13-22 BLEU points on English to German, Chinese and Japanese on the development sets. Additionally, we also propose a new enhanced measure of speech recognition latency.

## introduction in this paper, we describe the submission of the charles university (cuni) system to @xcite . our system is built on top of whisper @xcite with alignatt @xcite simultaneous policy. to achieve higher translation quality, we apply beam search and prompting for in-domain terminology. in our end-to-end system for the czech-to-english translation, we also exploit previous translations as a context. for the translation into german, chinese, and japanese, we adopted a cascaded approach consisting of whisper for english asr and eurollm @xcite for translation. we validate our systems' latency in computationally unaware simulation. our czech-to-english systems work both in 2-second and 4-second latency regimes required by @xcite . the english-to-german, chinese and japanese systems are available only in the high-latency regime of 4-5 seconds. for an overview of our systems, see our main goal in this submission is to create a robust and straightforward implementation that can be used in further research as well as in many realistic use cases. we name the implementation simulstreaming and publish it at https://github.com/ufal/simulstreaming . among the strengths of our submitted system is very high quality, because of using highperforming foundation models, and very high multilinguality. whisper allows direct translation from 99 speech source languages to english, and eu-rollm allows english translation into 35 languages. our systems are also adaptable; the prompts and in-context learning allow injecting specific in-domain terminology. moreover, although we primarily focus on computationally unaware latency, our system is practically usable in real time only with feasible hardware resources. it requires hosting whisper large 1.6b parameters model and and the eurollm 9b parameters model. our second goal is to evaluate the state-of-the-art methods in combination. our results show improvements by 2 bleu points on czech to english over the organizers baseline, and 13-22 bleu points on english to german, chinese, and japanese, which highlights the effectiveness of our systems. we conclude we have reached both goals. the original contributions of this work is the simul-streaming implementation and evaluation, and also a new enhanced method for measuring asr latency using continuous levenshtein alignment (see section 5.1). ## background whisper is among the top-performing asr and speech translation models for 99 languages. it has the ability to use initial and context prompts, which makes it adaptable for in-domain terminology. works such as @xcite and @xcite show that whisper is adaptable to simultaneous mode, although it is primarily designed for offline mode. whisper is available in multiple model versions that differ in size and quality. we use the large-v3 model, which achieves the highest quality. alignatt @xcite is a simultaneous policy. given an offline translation model, partial source and previous target, it detects where to stop generating the partial target, which is when the most attended source frame by the decoder is behind a threshold. @xcite shows that this policy outperforms all previously proposed policies. @xcite later showed that alignatt also works with whisper. localagreement @xcite is a simultaneous policy that considers the target prefixes of two subsequent updates, each processing a newly incoming source chunk. it emits their longest common prefix as confirmed and uses it in forced decoding of the latter chunks. simul-whisper @xcite is an implementation of the simultaneous mode with whisper using alignatt. it is an extension of the original openai whisper inference using the torch deep learning framework. simul-whisper supports only asr of speech that is segmented into individual sentences, and computationally unaware simulation, while the iwslt 2025 simultaneous task focuses on a more realistic case of unbounded speech @xcite without any explicit sentence boundaries. on the other hand, whisper-streaming @xcite is our imple-mentation of whisper with the localagreement simultaneous policy and reprocessing the audio buffer from the beginning with every incoming source chunk, which is less computationally effective than alignatt. on the other hand, whisper-streaming supports both computationally aware and unaware simulations, as well as unbounded speech. it integrates silero vad @xcite that incrementally detects silence and non-voice sounds vs. voice. eurollm @xcite is a recent large language model for text-to-text translation between 35 eu and non-eu languages, including english, german, japanese, and chinese. it is a decoderonly model of the llama family. we use its 9b parameter version with instruction tuning. it supports a system prompt, which can be used to suggest the domain, and a maximum context of 4096 tokens, which spans over 10 minutes of english source and german target of acl 6060 @xcite dev set reference (see estimation in appendix a). we use eurollm with the fast inference framework ctranslate2. it enables fast computation and efficient memory usage; however, it currently does not provide access to attention weights. therefore, we can not apply the alignatt policy to eurollm, so we use it with the localagreement simultaneous policy, which is the best-performing policy that does not require attention weights. our system uses our implementation called simulstreaming which merges the simul-whisper alignatt policy with the whisper-streaming interface to support unbounded speech processing. moreover, we extend the original simul-whisper with the following enhancements. first, we added support for the whisper large-v3 model. second, in addition to transcription, we enabled translation. then, to improve quality, we implement beam search decoding. finally, we incorporate support for initial prompts and contextual information from the preceding audio buffers. our simultaneous speech-to-text pipeline consists of the prototypical processing steps that are described in @xcite , section 3.1. 1. audio acquisition. ## simultaneous translation with eurollm we implement eurollm simultaneous translation using a chat template. we design a system prompt asking the model to perform simultaneous interpreting at a conference and specifying the translation direction. the chat is followed by the user's message containing the source prefix, and by the assistant's reply that contains the previous target prefix to continue. the chat is initiated with one sentence pair as an in-context example because we observed that without that, the model tends to produce text that is not a translation, especially for a short source. our simultaneous translation consists of steps that are analogous to the prototypical ones in papi et al. ( 2025): 1. source acquisition: the punctuated text produced by the simultaneous whisper english asr. additionally, this strategy does not require processing a slow parallel wordalignment model. ## conclusion in this paper, we presented our submission to the simultaneous speech translation task of the iwslt 2025. using the combination of the direct approach for czech-to-english translation and the cascaded approach for english to german, chinese, and japanese, we cover all language pairs of the task. to leverage the strong offline whisper speech model and the large language model eurollm, we applied state-of-the-art onlinization techniques and