ON - TRAC ’ systems for the IWSLT 2021 low-resource speech translation and multilingual speech translation shared tasks (2021)

Abstract: This paper describes the ON-TRAC Consortium translation systems developed for two challenge tracks featured in the Evaluation Campaign of IWSLT 2021, low-resource speech translation and multilingual speech translation. The ON-TRAC Consortium is composed of researchers from three French academic laboratories and an industrial partner: LIA (Avignon Université), LIG (Université Grenoble Alpes), LIUM (Le Mans Université), and researchers from Airbus. A pipeline approach was explored for the low-resource speech translation task, using a hybrid HMM/TDNN automatic speech recognition system fed by wav2vec features, coupled to an NMT system. For the multilingual speech translation task, we investigated the us of a dual-decoder Transformer that jointly transcribes and translates an input speech. This model was trained in order to translate from multiple source languages to multiple target ones.

## introduction in the two last editions of the iwslt evaluation campaigns, the on-trac consortium focused on end-to-end offline speech translation and simultaneous speech translation @xcite . in 2021, we chose to focus on low-resource speech translation and multilingual speech translation by using two different kinds of approaches: a cascaded speech-to-text translation (combining source language automatic speech recognition (asr) and source-to-target text translation) to process the low resource speech translation tasks, and a neural end-to-end model for the multilingual speech translation task. for the low resource task, we investigated the use of speech features extracted by a neural model pretrained by self supervision the wav2vec xlsr-53 model @xcite in order to process swahili lan-guages by a classical hybrid markovian/neuronal asr system. the asr outputs were processed by neural machine text-to-text translation systems dedicated to the two targeted language pairs. for the multilingual speech translation task, we investigated the use of a dual-decoder transformer that jointly transcribes and translates an input speech. this model was trained in order to translate from multiple source languages to multiple target ones. the on-trac consortium is composed of researchers from three french academic laboratories and an industrial partner: lia (avignon université), lig (université grenoble alpes), lium (le mans université), and researchers from airbus. ## low resource speech translation the task of the low resource speech translation track was to build the speech transcription/translation system for transcribing and/or translating between the two language pairs: ## multilingual speech translation speech-to-text translation (st) consists in translating a speech utterance in a source language to a text in another target language (e.g., english audio to french text). in this section, we describe a multilingual st system that can translate from multiple source languages to multiple target ones. ## conclusion this paper described the on-trac consortium submissions to the low-resource translation task and to the multilingual speech translation task. our unique asr system for both swahili and congolese swahili languages uses xlsr-53 wav2vec features as speech representation input. it got the best results on both swahili languages (respectively 31.25% and 36.75% of wer). the nmt systems used to translated these transcription into respectively to english and to french got bleu scores of 12.9 (@xmath0.1 (@xmath1. the dualdecoder transformer we used in the multilingual speech translation got promising results. we did not try a specific strategy to handle language pairs without training data. the low results we got on such language pairs confirm that a specific treatment must be applied in these conditions.