SRPOL ’s System for the IWSLT 2020 End-to-End Speech Translation Task (2020)

Abstract: We took part in the offline End-to-End English to German TED lectures translation task. We based our solution on our last year’s submission. We used a slightly altered Transformer architecture with ResNet-like convolutional layer preparing the audio input to Transformer encoder. To improve the model’s quality of translation we introduced two regularization techniques and trained on machine translated Librispeech corpus in addition to iwslt-corpus, TEDLIUM2 andMust_C corpora. Our best model scored almost 3 BLEU higher than last year’s model. To segment 2020 test set we used exactly the same procedure as last year.

## introduction this paper describes the submission to iwslt 2020 end-to-end speech translation task by samsung r&d institute, poland. we propose a few improvements to our previous system. introducing additional training data gave us 0.7 bleu improvement. spectrogram augmentation techniques increased quality by 0.2 bleu. encoder layer depth of 12 layers gives an increase of bleu by 1 point and 1.8 points when combined with additional training data and two spectrogram augmentation techniques. replacing simpler convolutions with resnet-like convolutional layers gave around 0.5 bleu improvement. combining all of these and increasing embedding size to 512 resulted in almost 3 bleu improvement compared to our last year's model. document structure is as follows. firstly we describe data preparation and augmentation. then we provide system specification and training procedure used in our experiments. we describe data segmentation algorithm used to segment test sets ted 2019 and ted 2020. we show results of our experiments. finally we conclude our results. ## training data to train our system we used only iwslt 2020 permissible audio corpora -iwslt-corpus, tedlium2 @xcite , must c corpus @xcite and machine translated librispeech @xcite corpus. we did not do any further data preparation in case of iwslt-corpus and tedlium2, we used the same data as in 2019. in case of must c corpus, this year we ran a training with half of translations being synthetic. this improved the score by 0.35 bleu. we used the same text translation models as last year to generate synthetic translations. both models scored around 33.8 and 31.1 bleu on tst2010 and tst2015 sets respectively. ## e2e speech translation system in this section we will describe the architecture and training techniques of our end-to-end spoken language translation system. some of these were used in our 2019 system. ## final model in this paper we presented improvements on simpler models. the final model uses a combination of all the promising techniques together with a larger embedding size. it would be unfeasible to perform all of these experiments on ## segmentation this year we used the same segmentation technique as last year. it relies on dividing the audio input densely using silence detection tool. these small fragments are then joined together up to a certain length depending on the length of the silence between them. shorter distances between segments are joined earlier. this procedure is repeated until further joining results in segments longer than maximal length. last year we determined such length should be 11s. however, for our current best model this distance turned out to be 15s. we used tst2015 to optimize the process. we present the result in ## evaluation we have improved our score on tst2019 by 4 bleu compared to our last year's submission. it is importnat to notice the difference between given and our custom segmentation. our method produces longer segments than the ones in the given segmentation. our models seem to work much better on these longer segments giving around 3.9 bleu higher scores. ## conclusions in this paper we have presented a significant improvement of translation quality of our end-toend model. we have shown that despite limited parallel training data, end-to-end systems can compete with traditional pipeline systems. using a longer segmentation, our model outscored the best iwslt2019 pipeline system on tst2019(iws, 2019).