Maastricht University’s Multilingual Speech Translation System for IWSLT 2021 (2021)

Abstract: This paper describes Maastricht University’s participation in the IWSLT 2021 multilingual speech translation track. The task in this track is to build multilingual speech translation systems in supervised and zero-shot directions. Our primary system is an end-to-end model that performs both speech transcription and translation. We observe that the joint training for the two tasks is complementary especially when the speech translation data is scarce. On the source and target side, we use data augmentation and pseudo-labels respectively to improve the performance of our systems. We also introduce an ensembling technique that consistently improves the quality of transcriptions and translations. The experiments show that the end-to-end system is competitive with its cascaded counterpart especially in zero-shot conditions.

## introduction in this paper, we describe our systems for the multilingual speech translation track of iwslt 2021. speech translation @xcite is the task of converting speech utterances to their translation in other languages. while "endto-end" modeling @xcite of the speech translation pipeline has become the dominant approach, an open challenge remains in terms of data scarcity. as the amount of speech directly paired with translation is lower compared to speech transcription or text-to-text translation, it is especially crucial for models to be data-efficient. in this context, multilingual speech translation @xcite presents itself as a promising direction to alleviate data scarcity by leveraging commonalities across languages. in this multilingual translation track, we submit: 1) an end-to-end system ( §5.2) that directly translates from speech and 2) a cascaded system ( §5.1) that consists of a multilingual speech transcription module ( §3) followed by a multilingual text translation module ( §4). our efforts to improve the speech translation system can be categorized as follows. when training, on the source side, we augment the speech data by speed perturbation. on the target side, we apply pseudo-labeling foot_0 by translating the asr transcriptions. furthermore, we train multilingual systems for both speech transcription and translation to alleviate the scarcity of training data. when testing, we use different ensembling techniques to increase the diversity of output distribution and improve output quality. the main findings from our experiments are: ## automatic speech recognition (asr) the asr performance is summarized in ## machine translation (mt) when translating from speech, the mt module ingests asr outputs. to assess the quality of the mt component alone, we first report the performance of directly translating from the ground truth transcriptions in ## speech translation (st) in ## results on blind test set we submitted systems c3 and e5 for evaluation on the blind test set. the results are summarized in ## conclusion this paper summarizes our participation in the iwslt 2021 multilingual speech translation track. we improved our end-to-end speech-to-text systems from different angles. on the source side, we augmented the input utterance. on the target side, we created pseudo-labels from asr transcriptions. furthermore, at test time we used different ensembling approaches to improve the performance of trained models. by experimenting under different data scenarios, we showed the benefit of multilingual training and the joint training speech transcription and translation. we note a few directions to further improve our systems: first, we expect that utterances augmented by specaugment @xcite could improve the quality of the asr and st systems. second, our mt module can be improved by synthetic data from back-translation @xcite , especially for the zero-shot directions. regarding upcoming work, since the source languages all belong to the same family, an interesting next step is to investigate how to better utilize the relatedness between these languages.