ON - TRAC Consortium for End-to-End and Simultaneous Speech Translation Challenge Tasks at IWSLT 2020 (2020)

Abstract: This paper describes the ON-TRAC Consortium translation systems developed for two challenge tracks featured in the Evaluation Campaign of IWSLT 2020, offline speech translation and simultaneous speech translation. ON-TRAC Consortium is composed of researchers from three French academic laboratories: LIA (Avignon Université), LIG (Université Grenoble Alpes), and LIUM (Le Mans Université). Attention-based encoder-decoder models, trained end-to-end, were used for our submissions to the offline speech translation track. Our contributions focused on data augmentation and ensembling of multiple models. In the simultaneous speech translation track, we build on Transformer-based wait-k models for the text-to-text subtask. For speech-to-text simultaneous translation, we attach a wait-k MT system to a hybrid ASR system. We propose an algorithm to control the latency of the ASR+MT cascade and achieve a good latency-quality trade-off on both subtasks.

## introduction while cascaded speech-to-text translation (ast) systems (combining source language speech recognition (asr) and source-to-target text translation (mt)) remain state-of-the-art, recent works have attempted to build end-to-end ast with very encouraging results @xcite @xcite @xcite . this year, iwslt 2020 offline translation track attempts to evaluate if endto-end ast will close the gap with cascaded ast for the english-to-german language pair. another increasingly popular topic is simultaneous (online) machine translation which consists in generating an output hypothesis before the entire we participated in: this paper goes as follows: we review the systems built for the offline speech translation track in §2. then, we present our approaches to the simultaneous track for both text-to-text and speech-to-text subtasks in §3. we ultimately conclude this work in §4. ## offline speech translation track in this work, we developed several end-to-end speech translation systems, using a similar architecture as last year @xcite and adapting it for translating english speech into german text (en-de). all the systems were developed using the espnet @xcite end-to-end speech processing toolkit. ## simultaneous speech translation track in this section, we describe our submission to the simultaneous speech translation (sst) track. our pipeline consists of an automatic speech recognition (asr) system followed by an online machine translation (mt) system. we first define our online asr and mt models in §3.1 and §3.2 respectively. then, we outline in §3.3 how we arrange the two systems for the speech-to-text subtask. we detail our experimental setup and report our results on the text-to-text subtask in §3.4 and on the speechto-text in §3.5. ## conclusion this paper described the on-trac consortium submission to the iwslt 2020 shared task. in the continuity of our 2019 participation, we have submitted several end-to-end systems to the offline speech translation track. a significant part of our efforts was also dedicated to the new simultaneous translation track: we improved wait-k models with unidirectional encoders and multi-path training and cascaded them with a strong asr system. future work will be dedicated to simultaneous speech translation using end-to-end models.