FINDINGS OF THE IWSLT 2023 EVALUATION CAMPAIGN (2023)

Abstract: This paper reports on the shared tasks organized by the 20th IWSLT Conference. The shared tasks address 9 scientific challenges in spoken language translation: simultaneous and offline translation, automatic subtitling and dubbing, speech-to-speech translation, multilingual, dialect and low-resource speech translation, and formality control. The shared tasks attracted a total of 38 submissions by 31 teams. The growing interest towards spoken language translation is also witnessed by the constantly increasing number of shared task organizers and contributors to the overview paper, almost evenly distributed across industry and academia.

## introduction the international conference on spoken language translation (iwslt) is the premier annual scientific conference for all aspects of spoken language translation (slt). iwslt is organized by the special interest group on spoken language translation (sig-slt), which is supported by acl, isca and elra. like in all previous editions @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite ,this year's conference was preceded by an evaluation campaign featuring shared tasks addressing scientific challenges in slt. this paper reports on the 2023 iwslt evaluation campaign, which offered the following 9 shared tasks: the shared tasks attracted 38 submissions by 31 teams (see ## offline slt offline speech translation is the task of translating audio speech in one language into text in a different target language, without any specific time or structural constraints (as, for instance, in the simultaneous, subtitling, and dubbing tasks). under this general problem definition, the goal of the offline st track (one of the speech tasks with the longest tradition at the iwslt campaign) is to constantly challenge a technology in rapid evolution by gradually introducing novelty aspects that raise the difficulty bar. ## simultaneous slt simultaneous speech translation means the system starts translating before the speaker finishes the sentence. the task is essential to enable people to communicate seamlessly across different backgrounds, in low-latency scenarios such as translation in international conferences or travel. this year, the task included two tracks: speechto-text and speech-to-speech, covering three language directions: english to german, chinese and japanese. ## multilingual slt the nlp and speech communities are rapidly expanding with increasing focus on broader language coverage and multilinguality. however, despite the community's efforts on asr and slt, research is rarely focused on applying these efforts to the data within the scientific domain. it is clear from recent initiatives to caption technical presentations at nlp and speech conferences that transcription and translation in the technical domain is needed, desired, and remains a disproportionate challenge for current asr and slt models compared to standard datasets in these spaces. motivated by the acl 60-60 initiative 18 to translate the acl anthology to up to 60 languages for the 60th anniversary of acl, which will be reported on at this year's acl conference co-located with iwslt, this year's multilingual task evaluates the ability of current models to translate technical presentations to a set of ten diverse target languages. ## dialect slt the dialect speech translation shared task is a continuation of last year's task. we use the same training data as 2022 and evaluated systems on the 2022 evaluation set to measure progress; in addition, we added a new 2023 evaluation set as blind test. from the organizational perspective, we merged the call for shared task with the the low-resource tasks (section 8) in order to encourage cross-submission of systems. ## low-resource slt the low-resource speech translation shared task focuses on the problem of developing speech transcription and translation tools for low-resourced languages. ## formality control for slt different languages encode formality distinctions in different ways, including the use of honorifics, grammatical registers, verb agreement, pronouns, and lexical choices. while machine translation (mt) systems typically produce a single generic translation for each input segment, slt requires adapting the translation output to be appropriate to the context of communication and target audience. this shared task thus challenges machine translation systems to generate translations of different formality levels.