Findings of the IWSLT 2025 Evaluation Campaign (2025)

Abstract: This paper presents the outcomes of the shared tasks conducted at the 22nd International Workshop on Spoken Language Translation (IWSLT). The workshop addressed seven critical challenges in spoken language translation: simultaneous and offline translation, automatic subtitling and dubbing, model compression, speech-to-speech translation, dialect and low-resource speech translation, and Indic languages. The shared tasks garnered significant participation, with 32 teams submitting their runs. The field’s growing importance is reflected in the increasing diversity of shared task organizers and contributors to this overview paper, representing a balanced mix of industrial and academic institutions. This broad participation demonstrates the rising prominence of spoken language translation in both research and practical applications.

## introduction the international conference on spoken language translation (iwslt) stands as the leading annual scientific conference dedicated to advancing all aspects of spoken language translation (slt). operating under the auspices of the special interest group on spoken language translation (sigslt), the conference receives support from three prestigious organizations: the association for computational linguistics (acl), the international speech communication association (isca), and the european language resources association (elra). maintaining its 22-year tradition, the 2025 conference was preceded by a comprehensive evaluation campaign designed to address critical scientific challenges in slt. this paper presents the outcomes of the 2025 iwslt evaluation campaign, which comprised seven distinct shared tasks organized into three primary research areas: the shared tasks drew participation from 32 diverse teams (detailed in ## evaluation the evaluation campaign features both automatic and human evaluation. to support automatic evaluation, we developed a dedicated evaluation server this year, as detailed in section 2.1. the server was piloted in the offline, model compression, and instruction following tracks. for the other tracks, submission and evaluation processes were managed by the respective organizers, following the procedure used in previous campaigns. in addi-tion, we performed a human evaluation across several tracks as described in 2.2 ## offline track the offline speech translation task at iwslt, a cornerstone of the conference's tradition, aims to establish a robust evaluation framework for monitoring advancements in spoken language translation. its core focus lies in unconstrained speech translation, distinguishing it from tasks with inherent temporal and structural limitations such as simultaneous translation or subtitling. while maintaining a consistent task formulation, the emphasis over time has incrementally shifted towards increasing task difficulty to better align with realworld demands, encompassing the translation of ## simultaneous track simultaneous speech translation focuses on translating speech in real-time, in a manner similar to simultaneous interpreting. the system is designed to begin translating before the end of an utterance. this technology is particularly useful in scenarios such as international conferences, personal travel, or public emergency events. the task included two tracks: cascaded and direct. submissions to the cascaded track contain systems that produce intermediate text, i.e. the transcription of the source audio, that is imme-diately consumed by a simultaneous text-to-text agent. in contrast, direct, or end-to-end, systems avoid any intermediate text and directly generate target-language (text) translations from the source audio. both tracks covered four language directions as in the previous year: english to german, english to chinese, english to japanese, and czech to english. ## subtitling track in recent years, the task of automatically creating subtitles for audiovisual content in another language has gained a lot of attention due to the rapid increase in the global distribution and streaming of movies, series, and user-generated videos. reflecting these trends, the automatic subtitling track was introduced for the first time in 2023 @xcite and proposed again in 2024 @xcite as part of the iwslt evaluation campaigns. the automatic subtitling task has been continued this year. 16 participants were asked to gen-erate subtitles in german and/or arabic from english speech in audiovisual documents. ## model compression track the model compression track, introduced for the first time at iwslt 2025, addresses a growing concern in the nlp community: how to reconcile the impressive capabilities of foundation models with the practical constraints of real-world deployment. as a matter of fact, while large-scale text and speech models have revolutionized tasks such as end-to-end speech-to-text translation, their substantial size and computational demands introduce significant challenges in resource-constrained settings-including mobile devices, embedded systems, and edge computing environments. this is particularly problematic when low-latency, ondevice inference is required. model compression offers a promising path forward, enabling reductions in model size and complexity while striving to minimize performance degradation as much as possible. by foregrounding this challenge, the track aims to establish a shared evaluation framework for monitoring future advancements in the development of more efficient, accessible, and deployable slt systems. ## low-resource slt the 5 th edition of the low-resource spoken language translation track focused on the translation of speech from a variety of data-scarce languages. the target language is typically a higher-resource one, generally of similar geographical or historical linkages. the goal of this shared task is to benchmark and promote speech translation technology for a diverse range of dialects and low-resource languages. while significant research progress has been demonstrated recently, many of the world's languages and dialects lack the parallel data at scale needed for standard supervised learning. recognizing that the biggest bottleneck towards truly language-inclusive speech translation systems is data availability, this year's edition included a data track, inviting participants to contribute newly collected speech translation datasets for under-resourced language pairs. 7.1 challenge systems track this year's task significantly expanded the typological and geogrpahical diversity of the languages, language families, and scripts represented. the ten subtasks were: data track this track aimed to empower language communities to contribute datasets. such datasets are essential for expanding the reach of spoken language technology to more languages and varieties. participants of this track were encouraged to get creative with data creation strategies, while also ensuring data quality. as such, data track instructions included the following: • translations should be performed, wherever possible, by qualified, native speakers of the target language. we strongly encouraged verification of the data by at least one additional native speaker. ## indic languages track the growing demand for inclusive digital access has highlighted the need for seamless crosslingual communication, especially in linguistically rich regions like india. while english dominates global technology and information spheres, millions of speakers of indic languages such as bengali, hindi, and tamil still lack adequate speech and language technologies. despite their large combined speaker base of over 700 million and significant cultural and economic importance, these languages remain underrepresented in nlp and speech research due to limited high-quality parallel text and audio data. compounding this challenge are the inherent complexities of indic languages including rich morphology, high inflection, and frequent codemixing in real-world discourse, which make spoken language translation (slt) development especially difficult @xcite . addressing this gap, the indic shared task track at iwslt 2025 focuses on slt for bengali, hindi, and tamil in both @xmath0. the latter is emphasized due to its higher complexity and the inclusion of stem and broadcast media domains, demanding systems capable of handling technical vocabulary and varied speech styles. by releasing the first benchmark dataset tailored to these low-resource languages across critical do-mains, this task aims to drive research that tackles real-world multilingual challenges. it seeks to advance digital inclusion, foster equitable access to global knowledge, and support the preservation and technological integration of indic languages. ## instruction-following track in recent years, large language models (llms) have redefined the landscape of natural language processing by demonstrating the ability to perform a wide range of tasks without requiring taskspecific architectures or fine-tuning. these models offer a single, unified interface for diverse applications such as translation, summarization, and question answering, simply by conditioning on textual instructions @xcite . initially restricted to textual input, llms are now evolving into multimodal systems, incorporating modalities such as vision and speech to expand their applicability beyond the text domain @xcite . in parallel, speech foundation models (sfms) have emerged as powerful architectures capable of processing spoken language at scale @xcite . when combined with the instructionfollowing capabilities of llms @xcite , they open new opportunities for building general-purpose speech models that are not lim-ited to handling a pre-defined set of tasks @xcite . this integration, often referred to as speechllm or sfm+llm @xcite , promises to deliver very versatile systems, making it possible to interact with spoken language in flexible and controllable ways. to explore this promising direction, this year we introduce, for the first time at iwslt, a new shared task focused on evaluating instructionfollowing models for the speech modality. the goal is to assess models that can perform multiple speech-to-text tasks-such as automatic speech recognition, speech translation, spoken question answering, and summarization-by following natural language prompts, using either short audio segments or long-form spoken content as input.