NAIST Simultaneous Speech Translation System for IWSLT 2025 (2025)

Abstract: This paper describes the NAIST submission to the English-to-German, Japanese, Chinese Simultaneous Speech-to-Text track at IWSLT 2025. Last year, our system was based on an end-to-end speech-to-text translation model that combined HuBERT and mBART. This year, the system consists of a Whisper encoder, the DeCo compressive projector, and the Qwen large language model. The simultaneous translation (SimulST) system is implemented by applying a local agreement policy to an offline-trained translation model. For the streaming translation (StreamST) system, we integrate an online version of the SHAS segmenter into our SimulST architecture. Our results demonstrate that adopting LLMs as the backbone architecture for speech translation tasks yields strong translation performance. Additionally, leveraging robust segmentation capability of SHAS for StreamST achieves good quality-latency trade-off when processing unbounded audio streams.

## introduction simultaneous speech-to-text translation (simulst) aims to mimic human interpreters by providing real-time translation with low latency while maintaining high translation quality. in simulst, the system generates translation before receiving the full source utterance. a decision policy is required to determine whether to generate partial output or wait for additional source context to improve reliability. some prior studies train dedicated models for simulst using specialized training strategies and architecture designs to learn a data-driven decision policy @xcite @xcite @xcite . however, their performance heavily depends on the design of training strategies, which is a complex and challenging task. furthermore, achieving different latency regimes typically requires training multiple separate models, substantially increasing computational requirements and complicating practical deployment. due to the aforementioned reasons, approaches using a single model for different simultaneous scenarios have become popular @xcite . these methods train the speech translation (st) model using offline translation data and then apply a manually designed decision policy to this offline st model for simulst inference. in this way, a single st model can adapt to different latency requirements in practical use. designing an optimal decision policy is significant to their performance. among several existing decision policies @xcite @xcite , local agreement (la) @xcite is one of the most popular method and won the simulst track of iwslt 2022 @xcite . it makes decisions by establishing an agreement between two consecutive chunks and only emitting their longest common prefixes. additionally, the attention-based decision policies, edatt @xcite and alignatt @xcite , have been proposed for encoder-decoder st models. they leverage the cross-attention mechanism to make decisions based on the idea that if the model attends to the tail end of the incomplete input speech, the generated hypothesis is unreliable and more context is needed. these attention-based decision policies have shown good performance and have been widely adopted for simulst tasks @xcite . most recently, several studies have explored the use of pre-trained large language models (llms) for simulst, capitalizing on their powerful generative and zero-shot transfer capabilities. @xcite proposes a cascaded architecture combining an asr model with a decoder-only llm to perform simulst. however, this cascaded approach is hindered by error propagation and additional latency. a few works have instead focused on end-to-end llm-based simulst systems. xu et al. (2024) trains an offline llm-based st model and extends it to simulst using the hold-n (liu et al., 2020 this paper describes the naist submission for the english-to-{german, japanese, chinese} simultaneous speech-to-text track at iwslt 2025. in our last year's system @xcite , we applied the la policy to an encoder-decoder model to do simulst. for the iwslt 2025 evaluation campaign, we explore employing llm in our system to conduct translation in real time. we construct an end-to-end llm-based st model, trained on offline data, and-similar to our previous system-enable it to perform simultaneous translation using the la policy. to handle the unbounded audio stream in real-world settings, we adopt an online version of the shas segmentation method @xcite to identify the speech segments in the audio stream and present the shasbased streamst. ## system description in this section, we first describe the model architecture of our system and its training methodology. then we present the detailed implementation of our simultaneous speech-to-text translation and streaming speech-to-text translation approaches. ## conclusion this paper presents our streamst system developed for the iwslt 2025 simultaneous speech translation track. experimental results demonstrated the effectiveness of employing an large language model (llm) as the backbone for the speech translation tasks. our system also showed the effectiveness of applying shas segmentation method in real time to handle unbounded audio stream during streaming speech translation. this time, we used the local agreement (la) for our llm-based system, which results in a higher computational latency in real condition. in the future, we will investigate better decision policy methods for the llm-based streamst system.