IMS ’ Systems for the IWSLT 2021 Low-Resource Speech Translation Task (2021)

Abstract: This paper describes the submission to the IWSLT 2021 Low-Resource Speech Translation Shared Task by IMS team. We utilize state-of-the-art models combined with several data augmentation, multi-task and transfer learning approaches for the automatic speech recognition (ASR) and machine translation (MT) steps of our cascaded system. Moreover, we also explore the feasibility of a full end-to-end speech translation (ST) model in the case of very constrained amount of ground truth labeled data. Our best system achieves the best performance among all submitted systems for Congolese Swahili to English and French with BLEU scores 7.7 and 13.7 respectively, and the second best result for Coastal Swahili to English with BLEU score 14.9.

## introduction we participate in the low-resource speech translation task of iwslt 2021. this task is organized for the first time, and it focuses on three speech translation directions this year: coastal swahili to english (@xmath0, congolese swahili to @xcite and congolese swahili to english (@xmath1. working on under-represented and low-resource languages is of special relevance for the inclusion into technologies of big parts of the world population. the masakhane initiative @xcite has opened the doors for large scale participatory research on languages of the african continent, to which swahili belongs to. our speech-to-text translation systems aim to contribute to this global effort. a common problem for these languages is the small amount of data. this is also true for the language pairs of the shared task: the provided data contains a small amount of translated speech samples for each pair, but the participants are allowed to use additional data and pre-trained models for the sub-tasks of asr and mt. we utilize most of the suggested additional data resources to train and tune sequence-to-sequence asr and mt components. our primary submission is the cascaded system built of conformer end-to-end asr model and transformer mt model. our contrastive system is end-to-end st system utilizing parameters transfer from the encoder part of asr model and the full mt model. both asr and mt components of the cascaded system initially yield good results on their own, but the discrepancy between language formats (spoken vs. written) in asr and mt corpora causes degradation by 47% in resulting scores. to adapt the mt system to the output of the asr, we transform the swahili source data to output similar to one of an asr system. to further increase the performance of our mt system, we leverage both source formats (original swahili text and simulated asr output swahili) into a multi-task framework. this approach improves our results by 17%, mostly for the english target language. our system outperforms the next best system on @xmath2.4 bleu points, but got outperformed by 10.4 bleu for @xmath3, being the second-best team. our team was the only participating for @xmath4.7 bleu. the results of end-to-end system consistently appear to be about twice worse compared to the pipeline approach. ## final systems ## conclusion this paper described the ims submission to the iwslt 2021 low-resource shared task on coastal and congolese swahili to english and 6 406 of 2124 hypothesis are empty.