B eaver T alk: O regon State University’s IWSLT 2025 Simultaneous Speech Translation System (2025)

Abstract: This paper discusses the construction, fine-tuning, and deployment of BeaverTalk, a cascaded system for speech-to-text translation as part of the IWSLT 2025 simultaneous translation task. The system architecture employs a VAD segmenter for breaking a speech stream into segments, Whisper Large V2 for automatic speech recognition (ASR), and Gemma 3 12B for simultaneous translation. Regarding the simultaneous translation LLM, it is fine-tuned via low-rank adaptors (LoRAs) for a conversational prompting strategy that leverages a single prior-sentence memory bank from the source language as context. The cascaded system participated in the English-German and English-Chinese language directions for both the low and high latency regimes. In particular, on the English-German task, the system achieves a BLEU of 24.64 and 27.83 at a StreamLAAL of 1837.86 and 3343.73, respectively. Then, on the English-Chinese task, the system achieves a BLEU of 34.07 and 37.23 at a StreamLAAL of 2216.99 and 3521.35, respectively.

## introduction this paper covers oregon state university's simultaneous translation system, beavertalk, for iwslt 2025. the system constructed takes in a speech stream input and outputs text translation in a cascaded manner for two language pairs, those being @xmath0. unique to iwslt 2025's simultaneous translation task @xcite , this system generates translation for unsegmented audio. architecture-wise, the system includes a vad speech segmenter @xcite , breaking a speech stream into segments, whisper large v2 @xcite performing automatic speech recognition (asr), and a fine-tuned gemma 3 12b model @xcite that performs context-aware conversational prompting to generate a simultaneous translation. the simultaneous translation portion of this cascaded system is fine-tuned on opensubtitles v2018 @xcite across both language pairs. given the unsegmented source for this task, leveraging additional context is possible and likely to improve results, based on prior work @xcite . as such, our system utilizes a singlesentence memory bank for the source language as context. this memory bank required modifying the typical conversational prompting structure for simultaneous translation @xcite . although a fine-tuned gemma 3 12b leveraging conversational prompting is a powerful model for simultaneous translation, its application in a cascaded architecture suffers from typical issues of error propagation @xcite . as such, maximizing the capabilities of a powerful simultaneous translation llm requires minimizing these errors in the preceding steps, consisting of the vad segmenter and whisper asr model. as such, we conduct an extensive inference time hyperparameter search aimed at minimizing error propagation. from the joint contributions of our cascaded simultaneous translation system and minimization of error propagation, we achieve impressive results on the acl 60/60 development set. for example, on the @xmath1, our cascaded system achieves a bleu of 24.64 and 27.83 at a streamlaal of 1837.86 and 3343.73. furthermore, on the @xmath2, our system achieves a bleu of 34.07 and 37.23 at a streamlaal of 2216.99 and 3521.35. ## task description simultaneous translation, generally speaking, is the process of taking in some source context and making translation decisions to another language figure 1: depiction of the cascaded system described in this technical paper. unsegmented source audio is taken in and fed into an asr pipeline that segments the audio and then transcribes it into a hard text data modality. this segmented, running transcription is then fed into a simultaneous translation pipeline powered by gemma 3. the transcription and current translation are fed into a conversational prompt constructor, adapted from prior work. in a manner that does not rely on that source context being complete. for example, typical neural machine translation (nmt) might act on a sentenceto-sentence basis, taking in a source sentence and outputting a target sentence. in comparison, a simultaneous translation must balance the lagging factor of output translations (i.e., the time it takes from a piece of source context to a corresponding piece of the output translation) with translation quality, making translation decisions with only partial context. as previously mentioned, the iwslt 2025 simultaneous translation task @xcite is fundamentally a speech-to-text task with two tracks governing what systems participants are expected to build: text-to-text where participants only construct a simultaneous agent for text data and prepend this system with an asr model and speech-to-text where the simultaneous system takes in raw speech and outputs target translations in text without the need for a conversion to a text data modality. our constructed system targets the text-to-text track, and since it is applied to the @xmath3, it is restricted to predefined high and low latency regimes specified by the task. these two latency regimes, as specified below, are governed by non-computationally aware streamlaal in seconds (s): the required development set for @xmath4. a blind test set is employed for final evaluations. ## beavertalk: a system description our simultaneous translation system consists of a cascaded architecture, which is divided into a vad segmenter utilizing a silero vad model @xcite , whisper large v2 @xcite , and a fine-tuned gemma 3 @xcite for simultaneous translation. in the system, gemma 3 was fine-tuned for a conversational prompting strategy @xcite , which is designed to mimic a streaming setting. our complete system is provided in figure 1 . our choice of a cascaded architecture rather than an end-to-end system hinges on our desire to (1) leverage the language modeling capabilities of an llm to overcome contextual obstacles faced during simultaneous translation, (2) take advantage of an llms context understanding capabilities to harness prior sentence context in a stream of data, and (3) benchmark the fine-tuning and multilingual capabilities of the recent gemma 3 @xcite . to provide a deeper understanding of our designed system, we will first explain the finetuning approach for our translation llm to enable conversational prompting, followed by a deeper explanation of our cascaded translation system. ## experimental setup the dataset of choice for fine-tuning is opensubtitles v2018. this corpus is particularly noisy (e.g. some chinese translations are almost entirely english, mismatched translations to transcriptions, etc.), rendering it difficult to achieve reasonable initial results. given that, some cleaning of the dataset is required. this occurs in four steps, the third of which only occurs for the @xmath5: the reference translation. this is meant to minimize the likelihood of a mismatched transcription and translation. the fine-tuning pipeline that employs the aforementioned dataset is based on frameworks for simultaneous translation with llms provided in prior work @xcite , which were then adapted for unsegmented fine-tuning and evaluation. fine-tuning occurred on gemma 3 4b/12b via loras with quantization @xcite . the lora adapters were applied to all attention projections and all the feed-forward network linear projections. we chose a lora r of 64, a lora α of 16, and a lora dropout of 0.1. our quantization quantized to 4-bit floating point via normalfloat with a compute data type of bfloat16. we used the paged 32-bit adamw optimizer and an inverse sqrt learning rate scheduler with an effective batch size of 64, a learning rate of 2e -4 , a weight decay of 0.1, a max gradient norm of 1, and a warm-up ratio of 0.03. we evaluate the translation quality of our models using bleu score with sacrebleu @xcite . the latency is reported using streamlaal @xcite . for the @xmath6, the latency and bleu scores are reported at the word level using the 13a tokenizer. alternatively, for the @xmath7, the latency and bleu scores are reported at the character level. our fine-tuning and evaluation for the gemma 12b models was conducted on an nvidia h200. alternatively, the gemma 4b models were trained on a nvidia a40 and evaluated on a nvidia v100. ## conclusion in this paper, we provide oregon state university's system, beavertalk, designed for the low and high latency regimes of the @xmath8. our system consists of a cascaded architecture composed of a vad speech segmenter, a whisper asr model, and a gemma 3 translation llm using conversational prompting. we provide an extensive inference hyperparameter search for our system and demonstrate its performance utilizing a 4b and 12b translation llm. our final submitted model, composed of the 12b translation llm, demonstrates strong results on the @xmath9.