End-to-End Offline Speech Translation System for IWSLT 2020 using Modality Agnostic Meta-Learning (2020)

Abstract: In this paper, we describe the system submitted to the IWSLT 2020 Offline Speech Translation Task. We adopt the Transformer architecture coupled with the meta-learning approach to build our end-to-end Speech-to-Text Translation (ST) system. Our meta-learning approach tackles the data scarcity of the ST task by leveraging the data available from Automatic Speech Recognition (ASR) and Machine Translation (MT) tasks. The meta-learning approach combined with synthetic data augmentation techniques improves the model performance significantly and achieves BLEU scores of 24.58, 27.51, and 27.61 on IWSLT test 2015, MuST-C test, and Europarl-ST test sets respectively.

## introduction the goal of the iwslt 2020 offline speech translation challenge @xcite is to check the feasibility of end-to-end models for translating audio speech of one language into text of a different target language. the success of end-toend neural models for asr @xcite and mt @xcite inspired to build end-to-end neural models for the more challenging speech-to-text translation (st) task @xcite . traditionally the st systems are built by cascading asr and mt systems @xcite . however, the cascaded system suffers from error propagation, latency, and memory requirement issues. although these issues can be addressed using end-to-end st models, it is hard to collect such data for training these models. in this work, we build an end-to-end st system which not only addresses the issues of a cascaded system but also works with limited training data. the proposed system is fine-tuned towards * the two authors contributed equally to this paper iwslt 2020 offline speech-translation task foot_0 . however, the proposed training strategies and the data augmentation techniques can be adopted into existing and future st models. we adopt the metalearning approach proposed for st task @xcite to train our system. the meta-learning based training approach not only allows us to leverage huge amounts of training data available in asr and mt tasks but also helps to find a good initialization point for the target st task. we conduct several experiments involving asr, mt, and st corpora to test our model performance on the iwslt 2020, must-c, and europarl-st english-german (en-de) st tasks. our experiments reveal that the proposed model trained using the meta-learning approach achieves significant performance gains over the model which only utilizes the st data for training. our model achieves 4.81, 5.37, and 8.46 bleu score improvements on iwslt test 2015, must-c test, europarl-st test sets compared to the models trained without using the meta-learning approach for training. our best system attains 24.58, 27.51, and 27.61 bleu scores on iwslt test 2015, must-c test, and europarl-st test sets, respectively. ## model architecture we use the transformer model as a base sequenceto-sequence (seq2seq) model to train the asr, mt, and st tasks. in this section, we describe briefly about the transformer architecture and how it is adopted to asr and st tasks. in section 2.2, we describe the meta-learning algorithm used to train our seq2seq model. ## conclusion in this work, we improve the performance of endto-end speech translation system based on the data available from the iwslt2020 offline speech translation task. we train end-to-end models to solve the complex task of speech translation. we leverage the large out-of-domain training data from the asr, mt tasks to improve the performance of the st task. we adopt model agnostic meta-learning(maml) and data augmentation techniques to achieve a performance of 24.58, 27.51, 27.61 bleu scores on iwslt test 2015, must-c test, and europarl-st test sets respectively.